Hosting & Infrastructure Case Study:
NSW Government

Scaling Wordpress

The majority of Launch Lab’s work involves developing products for startups using React, Python / Django and Amazon Web Services.

When working on these startup projects we offer a full solution including initial scoping, wireframing, web design, front-end development, back-end development, digital marketing and devops / infrastructure.

So, when we were approached to work on a pure deployment and infrastructure project for NSW Government it meant a slight change of pace.

The website had already been designed and developed by their digital agency and made use of Wordpress. Launch Lab was tasked with:

  • Hosting the Wordpress site
  • Ensuring the website could withstand massive spikes in traffic

Wordpress at scale

Wordpress has an abundance of features, and a very large and passionate community. However, it is very old in internet years. Wordpress comes from a different era, long before best practices in cloud computing had emerged. Nowadays, it is considered best practice to use “stateless” servers, where any persistent data is stored in a backing service, such as an S3 bucket on Amazon Web Services (AWS). This allows the site to scale easily as extra servers can be spun up or down as required.

However Wordpress expects files to exist on the server itself, which makes it challenging to build a scalable Wordpress website.

For the products Launch Lab develops in-house we use one of two main infrastructure setups on AWS. Our main workhorse over the last year and a half has been our own Launch Lab Cloud which is a highly available solution using Deis, Docker and Kubernetes. With newer projects we take the ‘serverless’ approach, using Zappa and AWS Lambda. However, both of these approaches are geared towards stateless “12-factor” apps. For the scalable Wordpress deploy we needed to use a different setup.

If you have a Wordpress website where most of the traffic is viewing static content, then you can easily put the site behind a CDN such as Cloudfront. But if you’re expecting a lot of traffic to a dynamic site, then you may need to look for a scalable Wordpress solution. There are several ways to set up Wordpress to use multiple servers to help with scalability. One option is to use NFS for a shared filesystem between server, but this approach is notorious for having issues in production with syncing and latency issues between the servers.

We followed the approach set out in an AWS whitepaper “Deploying WordPress with AWS Elastic Beanstalk

The most important part of that technique is using the W3 Total Cache Wordpress plugin to store static assets from Wordpress in an S3 bucket, which, along with using an external SQL server, effectively makes the Wordpress setup stateless. With the stateless setup, we could then apply auto-scaling rules via AWS Elastic Beanstalk to automatically spin up or down the number of servers depending on how much traffic was coming through to the site.

It is easy to see the benefit of having a scalable setup when you compare it to the alternative. A year ago we took over a different Wordpress site which was already being hosted using a standard, stateful setup running on a single EC2 server. This site occasionally received very large traffic spikes. If a traffic spike overloaded the server then the server would need to be upgraded to a larger EC2 instance, which resulted in a couple of minutes downtime right when the site was most active. To have a large enough EC2 server running permanently would have been wasteful and beyond the client’s budget.

If we were to to do this in 2017

It has been over a year since we deployed the stateless Wordpress setup described here, and web hosting infrastructure has been evolving very rapidly. Would we take the same approach if we needed to deploy another scaleable Wordpress application?

Probably not. While the scalable Wordpress setup didn’t give us any technical difficulties, we encountered dissatisfaction from Wordpress developers who had quite rigid expectations of how a Wordpress server should behave. The main frustrations from their point of view were needing to wait for deployments before they could see their changes in production, and making sure all of their work was compatible with the W3 Total Cache plugin.

How would we do it now? We could try a statefull setup using the new AWS Elastic File System to share the filesystem between servers, however this is not yet available in the Sydney region, which is a deal-breaker for many of our clients.

Another option is Kubernetes, which, over the last year, has emerged as the leader in container management. In the last few months Helm has become production-ready. Helm is a package manager which simplifies the task of deploying apps like Wordpress on a Kubernetes cluster. While not stateless, a Wordpress deploy on Kubernetes would be scaleable with containerised servers having a shared data storage. This would allow us to use a vanilla Wordpress deploy, removing any hard dependency on plugins.

View some of our other portfolio work including startup application development, corporate websites and more

How can we help you?

We're web application developers in Sydney & Canberra