I have another WordPress site (aside from this one) which consists of mostly static content and uses Cloudflare for DDoS protection and as a CDN. The server is running PHP 7.0 and has all updates installed but was taking a minimum of 400ms to generate each page; this delay was combined with any other delays common with HTTP requests such as TCP handshakes, DNS lookup times, and latency between the visitor and the server. This meant that users far away from the server location were experiencing delays of up to a second before the page evenĀ started loading and as the page was around a megabyte with around 50 or so additional resources, it wasn’t uncommon to take multiple seconds to finish loading.

My main focus was on the 400ms delay on every page load. As this was ‘wait’ time (aka, time to first byte), I knew it was most likely to be caused by WordPress’ rather large PHP codebase and that server-side caching would likely be the best way forward. I initially started by testing various caching plugins for WordPress but the improvements gained were negligible and some of the plugins were altering the cache headers which were actually instructing Cloudflare to not cache some resources which meant I was using the Apache config to overwrite those headers.

Apache to the rescue

Apache has a built-in cache function which I decided to try out instead of WordPress plugins. As the server I was using for this particular website has a mechanical hard drive, I set up a RAM disk and configured Apache to use it with the following snippet in the global Apache config (my RAM disk is mounted at /tmp/apachecache).

<IfModule mod_cache.c>
    LoadModule cache_disk_module /usr/lib/apache2/modules/mod_cache_disk.so
    <IfModule mod_cache_disk.c>
        CacheRoot /tmp/apachecache/
        CacheEnable disk /
        CacheDirLevels 5
        CacheDirLength 3   
        CacheHeader on
        CacheDefaultExpire 3600
        CacheMinExpire 60
        CacheIgnoreHeaders Set-Cookie
        CacheLastModifiedFactor 0.5
        CacheStaleOnError On
        CacheStoreExpired On
        CacheIgnoreNoLastMod On
        CacheQuickHandler Off
        CacheMaxFileSize 400000

This instructs Apache to cache all responses up to 400000 bytes for a minimum of 60 seconds, even pages without a Last Modified header and with the Set-Cookie header stripped. In addition, it is instructed to keep cached resources after their expiry time and serve them if WordPress encounters an error. Any resources without an expiry time are cached for 3600 seconds otherwise the expiry time is set by the cache headers (generated by WordPress in this instance).

By default WordPress sends rather cautious cache headers and therefore with this config alone very little was being cached. I found a WordPress plugin for setting sensible cache headers for dynamic pages based on their age, type, etc. This meant that most pages were being cached, but the number of cache hits was disappointingly low and users were seeing the logged out version of the site after logging in.

Optimising the cache

I needed to tell Apache not to serve cached pages to logged in users and to do that I used the following Apache VirtualHost configuration:

SetEnvIf Cookie wordpress_logged[\S\s]* logged_in
Header set Cache-Control "max-age=0, no-cache, no-store" env=logged_in
RequestHeader unset Pragma
RequestHeader set Pragma "no-cache" env=logged_in
RequestHeader unset Cache-Control
RequestHeader set Cache-Control "no-cache" env=logged_in

This tells Apache to set a variable named logged_in whenever a cookie beginning with wordpress_logged was sent in a request which matches all WordPress auth cookies. That variable is then used to alter the HTTP headers for that request; logged in users have Pragma and Cache-Control set to no-cache in their request headers (which bypasses all server-side caching) whereas logged out users have it removed entirely which prevents the browser refresh button from bypassing the cache. This fixes the problem where authenticated users were seeing the logged out version of the website and improved the number of cache hits slightly, but there were still a lot of improvements to make.

Disabling Compression

You’re probably wondering why disabling compression on a web server would speed anything up. As my website is behind Cloudflare, they are acting as a reverse proxy which means all requests go through their servers. Cloudflare sends the Accept-Encoding header from the client without modification and thanks to the Vary: Accept-Encoding header (which is very important, by the way) multiple caches for each page were being generated; one per each type of compression. The problem was made worse by the fact that Vary doesn’t care what order the header contents are in, so there are a huge number of combinations which are cached separately. Any of gzip, compress, deflate, br, identity, and * can be used, and combined, in any order. I was seeing around 10 different Accept-Encoding values from clients on a regular basis.

I discovered that Cloudflare will actually decompress responses and recompress them again with a better algorithm when possible so I was often compressing a page only for them to decompress it again before a visitor even receives it. I instructed Apache to remove the Accept-Encoding header from all requests which meant that it would never compress responses and would have just one cache for every request. This resulted in reduced loading times because my server was doing less work and Cloudflare continued to compress responses efficiently for clients anyway. As my server is hosted in a datacenter with a very fast internet connection (and so are Cloudflares servers), compression barely made a difference on that hop of the journey anyway; the time saved by not compressing the content negated the additional time it takes to transfer uncompressed data over a fast connection.

I also found out that Cloudflare makes good use of Keep Alive, so I raised the 5 second limit (the default) to 120 seconds. This means that Cloudflare remains connected to my server even after visitors finish loading the page which has the benefit of completely removing the TCP handshaking phase from Cloudflare and my server if another request comes in from the same location within 120 seconds.

The result

The configuration changes I made meant that all pages were cached for an appropriate amount of time and clients would always get the cached version if it existed and they were logged out which bypasses all of the WordPress logic and means that responses are now extremely quick. Unfortunately I can’t make use of Cloudflares cache everything mode which can be enabled from Page Rules as that will serve logged out pages to logged in users, however, by caching everything in RAM on my server the initial delay caused by thousands of lines of PHP executing is now gone.

The ‘Wait’ time for requests can now be as low as 80ms instead of always being at least 400ms. Cloudflare continues to cache all static resources which means their fast servers all around the world are serving most of the resources and my server only needs to be contacted once per page

Table and chart showing number of requests cached vs uncached

The pie chart and table above shows the caching result for dynamic pages over the last 5 days. “cache miss: attempting entity save” in green shows the number of cache misses for requests which are cacheable whereas “cache miss: cache unwilling to store response” in pink shows the number of cache misses for requests which are uncacheable (generally pages generated by authenticated users). By removing the uncacheable requests from the list, we get a much better indication of how well the cache works:

Table and chart showing number of requests cached vs uncached (uncacheable requests excluded)

With uncacheable requests excluded, the pie chart and table now show that 81.59% of requests for dynamic pages were served from cache over the past 5 days; this means that only 18.41% of requests from logged out users were generated while the user was waiting which is a significant improvement compared to every user having to wait the 400ms+ delay for WordPress to generate the page.

Leave a Reply

Your email address will not be published. Required fields are marked *