Server side minify leads to PHP process bottleneck on high traffic site. What are my options?

Question

I am currently tasked with finding a solution for a serious PHP bottleneck which is apparently caused by server-side minification of CSS and JS when our sites are under high load.

Some details and what I have found out so far

I inherited a web application running on Wordpress and which uses a complex constellation of Doctrine, Memcached and W3 Total Cache for minification and caching. When under heavy load our application begins to slow down rapidly. So far we have narrowed part of the problem down to the server-side minification process. Preliminary analysis has shown that the number PHP processes start to stack up under load, and when reaching the process limit of 500 processes, start to slow everything down. Something which is also mentioned by the author of the minify library.

Solutions I have evaluated so far

Pre-minification

The most logical solution would be to pre-minify any of the files before going live. Unfortunately our workflow demands that non-developers should be able to edit said files on our production servers (i.e. after the web app has gone live). Therefore I think that pre-processing is out of the question, as it limits the editability of minified files.

Serving unminified files

75% of our users are accessing our web application with their mobile devices, especially smartphones. Unminified JS and CSS amounts to 432KB and is reduced by 60-80% in size when minified. Therefore serving unminified files, while solving the performance and editability problem, is for the sake of mobile users out of the question.

I understand that this is as much a technical problem as it is a workflow problem and I guess we are open to working on both as long as we end up with a better overall performance.

My questions

1. Is there a reasonable compromise which solves the PHP bottleneck problem, allows for non-devs to make changes to live CSS/JS and still serves reasonably sized files to clients.

2. If there is no such one-size-fits-all solution, what can I do to better our workflow and / or server-side behaviour?

EDIT: Because there were some questions / comments regarding the server configuration, our servers run Debian and are equipped with 32GB of RAM and 24 core CPUs.

Answer 1

I would solve this with 2 solutions - first, removing any WP-CRON operation that runs every time a user runs the application and move that to actual CRON on the server. Second I would use load balancing so that a single server is not taking the load of the work. That is your real problem and even if you fix your perceived code issues you are still faced with the load issue.

I don't believe you need to change your workflow at all or go down the road of major modification to your existing system.

The WP-CRON tasks that runs each time a page is loaded causes significant load and slowness. You can shift this from the users visiting running that process to your server just running it at the server level. This reduces load. It is also most likely running these processes that you believe are slowing down the site.

See this guide: http://www.inmotionhosting.com/support/website/wordpress/disabling-the-wp-cronphp-in-wordpress

Next - load balancing. Having a single server supplying all your users when you have a lot of traffic is a terrible idea. You need to split the webservers load.

I'm not sure where or how you are hosted but I would move things to AWS. Setup my WordPress site for load balancing @ AWS: http://www.mornin.org/blog/scalable-wordpress-amazon-web-services/

This will involve:

• Load Balancer
• EC2 Instances running PHP/Apache
• RDS for your database storage for all EC2 instances
• S3 Storage for the sites media

For user sessions I suggest you just setup stickiness on the load balancer so users are continually served the same node they arrived on.

You can get a detailed guide on how to do this here: http://www.mornin.org/blog/scalable-wordpress-amazon-web-services/

Or at server fault another approach: https://serverfault.com/questions/571658/load-balancing-wordpress-on-amazon-web-services-managing-changes

The assumption here is that if you are high traffic you are making revenue from this high traffic so anytime your service responds slowly it will turn away users or possibly discourage them from returning. Changing the software could help - but you're treating the symptom not the illness. The illness is that your server comes under heavy load. This isn't uncommon with WordPress and high traffic, so you need to spread the load instead of try and micro-optimize. The difference is the optimizations will be small gains while the load balancing and spread of load actually solves the problem.

Finally - consider using a CDN for serving all of your media. This loads media faster and it removes load from your system by reducing the amount of requests to the server and it's output to the clients. It also loads pages faster consistently for people wherever they are visiting from by supplying media from nodes closest to them. At AWS this is called CloudFront. WordPress also offers this service free via Jetpack (I believe) but it does not handle all media from my understanding.

Answer 2

I like the idea of using GulpJS. One thing you might consider is to have a wp-cron or even just a system cron that runs every 5 minutes or so and then runs a gulp task to minify and concatenate your css and js files.

Another option that doesn't require scheduling but is based off of watching the file system for changes and then triggering a Gulp build to happen is to use incron (inotify cron). Check out the incron man page. Incron is great in that it triggers actions based on file system events such as file changes. You could use this to trigger a gulp build when any css file changes on the file system.

One caveat is that this is a Linux solution so if you're hosting on Windows you might have to look for something similar.

Edit: Incron Documentation

Answer 3

You can run a css/javascript compilation service like Gulp or Grunt via Node.js that minifies all your js and css assets on change.

This service can run in production but that is not recommended without some architectural setup ( having multiple versioned compiled files and auto-checking them via gulp or another extension ).

I emphasize that patching features into production and directly editing it is strongly discouraged as it can present live issues to your visitors reducing your credibility.

http://gulpjs.com/

Using Gulp/Grunt would require you to change how you write your css/javascript files.