5 minute read

Using Varnish with WordPress Some thoughts on W3 Total Cache and Tools to test Performance

I love WordPress.  I find it easy to use for developers and especially simple to use for content creators, marketers, and site admins.  Except for web apps WordPress allows for the rapid generation of sites with a backend admin interface, simple data-structures, metadata, and taxonomies.

There is one big knock on WordPress though and that is speed.  If you deal in performance marketing, SERPs, and just overall web experience you probably know that WordPress can be slow to load and send back that first byte.  Even with a basic WordPress install you can see a time-to-first-byte(TTFB) in the 300ms range.  Luckily, there are methodologies to mitigate this TTFB.  My favorite by far is Varnish.  Luckily, using Varnish with WordPress is simple.

Tools

First, let me spell out a few tools that can help in looking at the performance of your website.  These tools not only provide a graphical visualization of the loading of your site but include suggestions on how to speed things up.  These all do pretty much the same thing.  Try them all and see which one you like the best.

  1. Pingdom Tools - Probably my favorite.  Just make sure that you select the same city every time you test.
  2. Google PageSpeed
  3. GTMetrix

W3 Total Cache

If you look around the web on speeding up performance you will find a lot of information and recommendations to use W3 Total Cache.  There are certain things that W3 Total Cache is great at but Page Caching isn't really one of them.  First, it is very difficult to set the actual time that a page will stay cached (especially if you want different lengths), warming doesn't work all that well, and the performance isn't top notch.  With that said there are some things I still use W3 Total Cache for.

  1.  Minification - I will go into this in a later post but W3 Total Cache has an excellent JS and CSS minifier.  It automatically appends a random string to the files also that allows a CDN to function simply without the need to constantly purge.
  2. CDN - I have used W3 Total Cache to set up MaxCDN and CloudFront CDNs.  It is simple, straightforward and gets the job done.
  3. Browser Caching Headers - The options for setting the browser caching headers is great and simple to use.

 Varnish Basics with WordPress

The first step to using Varnish with WordPress is to install it on your server and get it running on Port 80 instead of Apache.  DigitalOcean has a great tutorial on how to do this.  You can find it here: Installing Varnish on Ubuntu 12.04.

If you follow that guide you will have Varnish running on Port 80 and Apache running on Port 8080.  This means that every request made to your server will flow through Varnish.  WordPress has some special concerns that need to be addressed to get your site working properly.

The main file that you need to worry about is /etc/varnish/default.vcl.

This VCL file controls how Varnish works and can be extensively customized to do all sorts of things.  We will cover the basics that every Varnish with WordPress site will need.

  1. Allow Purging of pages from the command line
  2. Clean up the various Accept-Encoding headers that can be sent
  3. Allow uncached versions of the wp-admin URLs
  4. Allow uncached versions of the &preview=true URLs
  5. Allow uncached versions of the &nocache URLs
  6. Return a 'HIT' or 'PASS' in the return header for debugging
  7. OPTIONAL: If using a CDN do not cache static resources so that the CDN can serve them
  8. OPTIONAL: Allow logged in users to see uncached pages

This sounds like a lot but it really isn't.  Varnish has its own language that makes all of this very simple and you will get the hang of it quickly.

The Varnish website has a good brief tutorial that explains most of the configuration language so I encourage you to take a quick look.  It will make the following sections much simpler and only takes a minute or two to read.  Here is a link to the Varnish Configuration Language Tutorial.

The Varnish default.vcl File

OK, now we are ready for the meat of this post.  Below is a Gist of the exact default.vcl file I am using on this site.  I have left comments within the file to explain what is happening.  Hopefully, this will be enough to get you started with your Varnish configuration and give you insight into what is happening at each step.

/* SET THE HOST AND PORT OF WORDPRESS
 * *********************************************************/
backend default {
  .host = "127.0.0.1";
  .port = "8080";
}

# SET THE ALLOWED IP OF PURGE REQUESTS
# ##########################################################
acl purge {
  "localhost";
  "162.243.20.190";
}

# THE RECV FUNCTION
# ##########################################################
sub vcl_recv {
  # For Testing: If you want to test with Varnish passing (not caching) uncomment
  # return( pass );

  # FORWARD THE IP OF THE REQUEST
  if (req.restarts == 0) {
    if (req.http.x-forwarded-for) {
      set req.http.X-Forwarded-For =
      req.http.X-Forwarded-For + ", " + client.ip;
    } else {
      set req.http.X-Forwarded-For = client.ip;
    }
  }

  # CLEAN UP THE ENCODING HEADER.  
  # SET TO GZIP, DEFLATE, OR REMOVE ENTIRELY.  WITH VARY ACCEPT-ENCODING
  # VARNISH WILL CREATE SEPARATE CACHES FOR EACH
  # DO NOT ACCEPT-ENCODING IMAGES, ZIPPED FILES, AUDIO, ETC.
  # ##########################################################
  if (req.http.Accept-Encoding) {
    if (req.url ~ "\.(jpg|png|gif|gz|tgz|bz2|tbz|mp3|ogg)$") {
      # No point in compressing these
      remove req.http.Accept-Encoding;
    } elsif (req.http.Accept-Encoding ~ "gzip") {
      set req.http.Accept-Encoding = "gzip";
    } elsif (req.http.Accept-Encoding ~ "deflate") {
      set req.http.Accept-Encoding = "deflate";
    } else {
      # unknown algorithm
      remove req.http.Accept-Encoding;
    }
  }

  # IF THIS IS A PURGE REQUEST, THEN CHECK THE IPS SET ABOVE
  # BLOCK IF NOT ONE OF THOSE IPS
  # ##########################################################
  if (req.request == "PURGE") {
    if ( !client.ip ~ purge ) {
      error 405 "Not allowed.";
    }
    return (lookup);
  }

  # PIPE ALL NON-STANDARD REQUESTS
  # ##########################################################
  if (req.request != "GET" &&
    req.request != "HEAD" &&
    req.request != "PUT" && 
    req.request != "POST" &&
    req.request != "TRACE" &&
    req.request != "OPTIONS" &&
    req.request != "DELETE") {
      return (pipe);
  }
   
  # ONLY CACHE GET AND HEAD REQUESTS
  # ##########################################################
  if (req.request != "GET" && req.request != "HEAD") {
    return (pass);
  }
  
  # OPTIONAL: DO NOT CACHE LOGGED IN USERS (THIS OCCURS IN FETCH TO, EITHER
  # COMMENT OR UNCOMMENT BOTH
  # ##########################################################
  if ( req.http.cookie ~ "wordpress_logged_in" ) {
    return( pass );
  }
  
  # IF THE REQUEST IS NOT FOR A PREVIEW, WP-ADMIN OR WP-LOGIN
  # THEN UNSET THE COOKIES
  # ##########################################################
  if (
    !(req.url ~ "wp-(login|admin)") 
    && !(req.url ~ "&preview=true" ) 
  ){
    unset req.http.cookie;
  }

  # IF BASIC AUTH IS ON THEN DO NOT CACHE
  # ##########################################################
  if (req.http.Authorization || req.http.Cookie) {
    return (pass);
  }

  # LIST URLS NOT TO BE CACHED
  # USUALLY THIS INCLUDES THE PREVIEW QUERY STRING, A NOCACHE
  # QUERY STRING, AND FILES THAT WILL BE SERVED BY THE CDN
  # SUCH AS IMAGES, CSS, AND JS.  IF YOU ARE NOT USING A CDN 
  # YOU CAN CACHE YOUR STATIC RESOURCES TOO
  # ##########################################################
  if ( 
    req.url ~ "preview" 
    || req.url ~ "nocache"
    || req.url ~ "\.css$"
    || req.url ~ "\.js$"
    || req.url ~ "\.jpg$"
    || req.url ~ "\.jpeg$"
    || req.url ~ "\.gif$"
    || req.url ~ "\.png$" 
  ) {
    return (pass);
  }
  
  # IF YOU GET HERE THEN THIS REQUEST SHOULD BE CACHED
  # ##########################################################
  return (lookup);
}

# HIT FUNCTION
# ##########################################################
sub vcl_hit {
  # IF THIS IS A PURGE REQUEST THEN DO THE PURGE
  # ##########################################################
  if (req.request == "PURGE") {
    purge;
    error 200 "Purged.";
  }
  return (deliver);
}

# MISS FUNCTION
# ##########################################################
sub vcl_miss {
  if (req.request == "PURGE") {
    purge;
    error 200 "Purged.";
  }
  return (fetch);
}

# FETCH FUNCTION
# ##########################################################
sub vcl_fetch {
  # I SET THE VARY TO ACCEPT-ENCODING, THIS OVERRIDES W3TC 
  # TENDANCY TO SET VARY USER-AGENT.  YOU MAY OR MAY NOT WANT
  # TO DO THIS
  # ##########################################################
  set beresp.http.Vary = "Accept-Encoding";

  # IF NOT WP-ADMIN THEN UNSET COOKIES AND SET THE AMOUNT OF 
  # TIME THIS PAGE WILL STAY CACHED (TTL)
  # ##########################################################
  if (!(req.url ~ "wp-(login|admin)") && !req.http.cookie ~ "wordpress_logged_in" ) {
    unset beresp.http.set-cookie;
    set beresp.ttl = 96h;
  }

  if (beresp.ttl <= 0s ||
    beresp.http.Set-Cookie ||
    beresp.http.Vary == "*") {
      set beresp.ttl = 120 s;
      return (hit_for_pass);
  }

  return (deliver);
}

# DELIVER FUNCTION
# ##########################################################
sub vcl_deliver {
  # IF THIS PAGE IS ALREADY CACHED THEN RETURN A 'HIT' TEXT 
  # IN THE HEADER (GREAT FOR DEBUGGING)
  # ##########################################################
  if (obj.hits > 0) {
    set resp.http.X-Cache = "HIT";
  # IF THIS IS A MISS RETURN THAT IN THE HEADER
  # ##########################################################
  } else {
    set resp.http.X-Cache = "MISS";
  }
}

Conclusion

As you can see, it is not that difficult to setup Varnish with WordPress.  The performance increase is dramatic when compared to something like W3 Total Cache.  If you are concerned with page load times at all you should give this a try.

If you have any trouble, just leave a comment below.


Leave a Reply

Your email address will not be published.


    • @Kale: This was made for Varnish 3. Varnish 4 VCL’s are slightly different but you probably can work through it and get it going. I will try to post an update when I have a chance.

  1. Hi Ryan,

    Thanks for tutorial and helpful links. I wanted to run my wordpress site on aws t2.nano instance which has 512MB RAM for which I was looking for a way to keep wordpress alive given that much RAM.

    My site is alive again.

    Regards.

  2. Hi,

    Just wanted to leave a comment after finding it frustrating getting varnish to cache my pages. Came across this after several hours and replaced my config file with your own, and wollah! It even works on my non-Wordpress vhosts

    Thanks so much for this

  3. Hi Andrew,

    Excellent share! How if i set the server as image CDN of my wordpress elsewhere, and install varnish on both sides, will that increase the page load time?

    Or is it, when you already use CDN like maxCDN or cloudfront, you dont necesarily to activate varnish to improve loading time?

    Thanks for the advice

  4. Thanks for this, I was having issues getting my menu to save when I updated it, now it’s fixed. I will be referencing you in my post 🙂