The perfect Varnish configuration for WordPress, Joomla, Drupal & other (common) CMS based websites

default.vcl_PREFACE.md

The perfect Varnish configuration for WordPress, Joomla, Drupal & other (common) CMS based websites

Updated on December 15th, 2021

IMPORTANT: Read this before implementing one of the configuration files below (for either Varnish 3.x or 4.x+).

USE: Replace the contents of the main Varnish configuration file located in /etc/varnish/default.vcl (root server access required - obviously) with the contents of the configuration you'll use (depending on your Varnish version) from the 2 examples provided below.

IMPORTANT: The following setup assumes a 180 sec (3 minute) cache time for cacheable content that does not have the correct cache-control HTTP headers. You can safely increase this to 300 sec (or more) for less busier sites or drop it to 60 sec or even 30 sec for high traffic sites. It obviously depends on your use case.

This configuration requires an HTTP Header and a user cookie to identify if a user is logged in a site, in order to bypass caching overall (see how it's done for Joomla & WordPress). If your CMS provides a way to add these two requirements, then you can use this configuration to speed up your site or entire server. You can even exclude the domains you don't want to cache if you're looking to use it in a multi-site setup.

WORDPRESS/WOOCOMMERCE & VARNISH

WordPress and WooCommerce are fully supported by these configurations. Once a user logs into WordPress, Varnish will shut down its caching for that user.

JOOMLA & VARNISH

Since Joomla v3.6, all you need to do to have Joomla play nicely with Varnish is add your exclusion points (URLs). Just have a look at the 2 blocks starting with "Exclude the following paths..." below in the Varnish configurations.

Or you can install this handy plugin from JoomlaWorks https://github.com/joomlaworks/url-normalizer which covers all Joomla versions from 1.5 to 3.x and extends/improves the built-in changes introduced in Joomla v3.6 (as stated above).

If you're using Joomla before version 3.6 and you don't wish to use the plugin above, you need to do the following: This Varnish configuration makes use of a custom HTTP header plus a user cookie to determine whether some user is logged in or not inside Joomla. To insert the HTTP header, simply append the following code block in your template's "index.php" file, right after the line:

defined('_JEXEC') or die;

...and make sure you set the $cookieDomain value:

// Make Joomla Varnish-friendly [START]
$cookieDomain = 'domain.tld'; // Replace "domain.tld" with your "naked" domain

$getUserState = JFactory::getUser();

if ($getUserState->guest) {
    JResponse::allowCache(true);
    JResponse::setHeader('X-Logged-In', 'False', true);
    if($_COOKIE["userID"]){
        setcookie("userID", "", time() - 3600, '/', $cookieDomain, 0);
    }
} else {
    JResponse::allowCache(true);
    JResponse::setHeader('X-Logged-In', 'True', true);
    if(!isset($_COOKIE["userID"])){
        setcookie("userID", $getUserState->id, 0, '/', $cookieDomain, 0);
    }
}
// Make Joomla Varnish-friendly [FINISH]

IMPORTANT: If you use K2 (getk2.org) in your Joomla site, all the above will be automatically enabled for your K2 content when caching is also enabled in Joomla.

HOW TO HANDLE FRONTEND LOGINS (e.g. for use with member areas, forums etc.)

It is important for you to understand that since Joomla (in a very amateur way) uses session cookies for any user (even guests) supposedly for additional security (debatable), Varnish cannot work with Joomla out-of-the-box.

If you installed Varnish without any modification to its configuration besides the cache time, it could not properly cache Joomla content because of the session cookies Joomla uses for both guest and logged in visitors. To bypass Joomla's behaviour, we must additionally set Varnish to strip any cookies set by Joomla, except for a specific one (userID). For even better control, we also set a custom HTTP header (X-Logged-In), which we have Varnish check on all requests (see the code previously mentioned).

However, if we want Varnish to allow frontend logins in Joomla, without breaking Joomla (because we strip its session cookies), we must explicitly tell Varnish which entry pages (=login pages) not to cache. Such a page could be for example the default Joomla login form (e.g. with an alias "login"). In the 2 Varnish exclusion lists defined in the configurations below, we would add "^/login" to make sure Varnish completely switches off when a user visits this page. In that case, Joomla's session cookie gets set and the form can be submitted normally, passing all Joomla security checks (aka a token exists as a hidden form input). Same goes for any page in Joomla that requires user input: a contact form, a newsletter signup form, a forum, comments and so on. So the solution to keep in mind is simple:

• If the action requires the user to login first (e.g. a forum), we must create a specific/unique page for users to login first. Once they log in, Varnish switches off completely and then a user can post in the forum or write comments or use a contact form as if Varnish did not exist. If the user continues to browse the site while logged in, Varnish will be completely off ONLY for this user. If the user logs out, Varnish will kick back in.

• If the action does not require a user to be logged in first, e.g. a contact form, we simply exclude the contact form's URL from Varnish, in which case -again- Varnish will switch off completely and the user will be able to submit the form passing the Joomla security checks. If the user browses anywhere else in the site, Varnish will kick back in.

for_Varnish_3.x_default.vcl

#########################################################################
###               The perfect Varnish 3.x configuration               ###
### for WordPress, Joomla, Drupal & other (common) CMS based websites ###
#########################################################################

######################
#
# UPDATED on December 15th, 2021
#
# Configuration Notes:
# 1. Default dynamic content caching respects your backend's cache-control HTTP header.
#    If however you need to enforce a different cache-control TTL,
#    do a search for "180" and replace with the new value in seconds.
#    Stale cache is served for up to 24 hours.
# 2. Make sure you update the "backend default { ... }" section with the correct IP and port
#
######################

# Varnish Reference:
# See the VCL chapters in the User-Guide at https://varnish-cache.org/docs/

# Default backend definition. Set this to point to your content server.
backend default {
    .host = "127.0.0.1"; # UPDATE this only if the web server is not on the same machine
    .port = "8080";      # UPDATE 8080 with your web server's (internal) port
}

sub vcl_recv {

    /*
    # === The following are disabled by default - enable if you understand what you're doing ===
    # Blocks
    if (req.http.user-agent ~ "^$" && req.http.referer ~ "^$") {
        return (synth(204, "No content"));
    }
    if (req.http.user-agent ~ "(ahrefs|domaincrawler|dotbot|mj12bot|semrush)") {
        return (synth(204, "Bot blocked"));
    }

    # List domains/subdomains to exclude from caching
    if (req.http.host ~ "(domain1.tld|sub.domain2.tld)") {
        return (pass);
    }
    */

    # LetsEncrypt Certbot passthrough
    if (req.url ~ "^/\.well-known/acme-challenge/") {
        return (pass);
    }

    # Forward client's IP to the backend
    if (req.restarts == 0) {
        if (req.http.X-Real-IP) {
            set req.http.X-Forwarded-For = req.http.X-Real-IP;
        } else if (req.http.X-Forwarded-For) {
            set req.http.X-Forwarded-For = req.http.X-Forwarded-For + ", " + client.ip;
        } else {
            set req.http.X-Forwarded-For = client.ip;
        }
    }

    # httpoxy
    unset req.http.proxy;

    # Non-RFC2616 or CONNECT which is weird.
    if (
        req.request != "GET" &&
        req.request != "HEAD" &&
        req.request != "PUT" &&
        req.request != "POST" &&
        req.request != "TRACE" &&
        req.request != "OPTIONS" &&
        req.request != "DELETE"
    ) {
        return (pipe);
    }

    # We only deal with GET and HEAD by default
    if (req.request != "GET" && req.request != "HEAD") {
        return (pass);
    }

    # === URL manipulation ===
    # Remove tracking query string parameters associated with analytics/social services, useless for our backend
    if (req.url ~ "(\?|&)(_bta_[a-z]+|cof|cx|fbclid|gclid|ie|mc_[a-z]+|origin|siteurl|utm_[a-z]+|zanpid)=") {
        set req.url = regsuball(req.url, "(_bta_[a-z]+|cof|cx|fbclid|gclid|ie|mc_[a-z]+|origin|siteurl|utm_[a-z]+|zanpid)=[-_A-z0-9+()%.]+&?", "");
        set req.url = regsub(req.url, "[?|&]+$", "");
    }

    # Strip a trailing ? if it exists
    if (req.url ~ "\?$") {
        set req.url = regsub(req.url, "\?$", "");
    }

    # Strip hash, server doesn't need it.
    if (req.url ~ "\#") {
        set req.url = regsub(req.url, "\#.*$", "");
    }

    # === Generic cookie manipulation ===
    # Remove common cookies associated with analytics/social services (inc. WordPress test cookies)
    set req.http.Cookie = regsuball(req.http.Cookie, "(has_js|__utm.|_ga|_gat|utmctr|utmcmd.|utmccn.|__gads|__qc.|__atuv.|wp-settings-1|wp-settings-time-1|wordpress_test_cookie)=[^;]+(; )?", "");

    # Remove a ";" prefix in the cookie if present
    set req.http.Cookie = regsuball(req.http.Cookie, "^;\s*", "");

    # Remove blank cookies
    if (req.http.cookie ~ "^\s*$") {
        unset req.http.cookie;
    }

    # Check for the custom "X-Logged-In" header (used by K2 and other apps) to identify
    # if the visitor is a guest, then unset any cookie (including session cookies) provided
    # it's not a POST request.
    if (req.http.X-Logged-In == "False" && req.request != "POST") {
        unset req.http.Cookie;
    }

    # === DO NOT CACHE ===
    # Don't cache HTTP authorization/authentication pages and pages with certain headers or cookies
    if (
        req.http.Authorization ||
        req.http.Authenticate ||
        req.http.X-Logged-In == "True" ||
        req.http.Cookie ~ "userID" ||
        req.http.Cookie ~ "joomla_[a-zA-Z0-9_]+" ||
        req.http.Cookie ~ "(wordpress_[a-zA-Z0-9_]+|wp-postpass|comment_author_[a-zA-Z0-9_]+|woocommerce_cart_hash|woocommerce_items_in_cart|wp_woocommerce_session_[a-zA-Z0-9]+)"
    ) {
        #set req.http.Cache-Control = "private, max-age=0, no-cache, no-store";
        #set req.http.Expires = "Mon, 01 Jan 2001 00:00:00 GMT";
        #set req.http.Pragma = "no-cache";
        return (pass);
    }

    # Exclude the following paths (e.g. backend admins, user pages or ad URLs that require tracking)
    # In Joomla specifically, you are advised to create specific entry points (URLs) for users to
    # interact with the site (either common user logins or even commenting), e.g. make a menu item
    # to point to a user login page (e.g. /login), including all related functionality such as
    # password reset, email reminder and so on.
    if (
        req.url ~ "^/addons" ||
        req.url ~ "^/administrator" ||
        req.url ~ "^/cart" ||
        req.url ~ "^/checkout" ||
        req.url ~ "^/component/banners" ||
        req.url ~ "^/component/socialconnect" ||
        req.url ~ "^/component/users" ||
        req.url ~ "^/connect" ||
        req.url ~ "^/contact" ||
        req.url ~ "^/login" ||
        req.url ~ "^/logout" ||
        req.url ~ "^/lost-password" ||
        req.url ~ "^/my-account" ||
        req.url ~ "^/register" ||
        req.url ~ "^/signin" ||
        req.url ~ "^/signup" ||
        req.url ~ "^/wc-api" ||
        req.url ~ "^/wp-admin" ||
        req.url ~ "^/wp-login.php" ||
        req.url ~ "^\?add-to-cart=" ||
        req.url ~ "^\?wc-api="
    ) {
        #set req.http.Cache-Control = "private, max-age=0, no-cache, no-store";
        #set req.http.Expires = "Mon, 01 Jan 2001 00:00:00 GMT";
        #set req.http.Pragma = "no-cache";
        return (pass);
    }

    # Don't cache ajax requests
    if (req.http.X-Requested-With == "XMLHttpRequest" || req.url ~ "nocache") {
        #set req.http.Cache-Control = "private, max-age=0, no-cache, no-store";
        #set req.http.Expires = "Mon, 01 Jan 2001 00:00:00 GMT";
        #set req.http.Pragma = "no-cache";
        return (pass);
    }

    # === STATIC FILES ===
    # Properly handle different encoding types
    if (req.http.Accept-Encoding) {
        if (req.url ~ "\.(jpg|jpeg|png|gif|gz|tgz|bz2|tbz|mp3|ogg|swf)$") {
            # No point in compressing these
            unset req.http.Accept-Encoding;
        } elseif (req.http.Accept-Encoding ~ "gzip") {
            set req.http.Accept-Encoding = "gzip";
        } elseif (req.http.Accept-Encoding ~ "deflate") {
            set req.http.Accept-Encoding = "deflate";
        } else {
            # unknown algorithm (aka crappy browser)
            unset req.http.Accept-Encoding;
        }
    }

    # Remove all cookies for static files & deliver directly
    if (req.url ~ "^[^?]*\.(7z|avi|bmp|bz2|css|csv|doc|docx|eot|flac|flv|gif|gz|ico|jpeg|jpg|js|less|mka|mkv|mov|mp3|mp4|mpeg|mpg|odt|ogg|ogm|opus|otf|pdf|png|ppt|pptx|rar|rtf|svg|svgz|swf|tar|tbz|tgz|ttf|txt|txz|wav|webm|webp|woff|woff2|xls|xlsx|xml|xz|zip)(\?.*)?$") {
        unset req.http.Cookie;
        return (lookup);
    }

    return (lookup);

}

sub vcl_fetch {

    /*
    # === The following are disabled by default - enable if you understand what you're doing ===
    # List domains/subdomains to exclude from caching
    if (bereq.http.host ~ "(domain1.tld|sub.domain2.tld)") {
        set beresp.uncacheable = true;
        return (hit_for_pass);
    }
    */

    # Don't cache 50x responses
    if (
        beresp.status == 500 ||
        beresp.status == 502 ||
        beresp.status == 503 ||
        beresp.status == 504
    ) {
        return (hit_for_pass);
    }

    # === DO NOT CACHE ===
    # Exclude the following paths (e.g. backend admins, user pages or ad URLs that require tracking)
    # In Joomla specifically, you are advised to create specific entry points (URLs) for users to
    # interact with the site (either common user logins or even commenting), e.g. make a menu item
    # to point to a user login page (e.g. /login), including all related functionality such as
    # password reset, email reminder and so on.
    if (
        bereq.url ~ "^/addons" ||
        bereq.url ~ "^/administrator" ||
        bereq.url ~ "^/cart" ||
        bereq.url ~ "^/checkout" ||
        bereq.url ~ "^/component/banners" ||
        bereq.url ~ "^/component/socialconnect" ||
        bereq.url ~ "^/component/users" ||
        bereq.url ~ "^/connect" ||
        bereq.url ~ "^/contact" ||
        bereq.url ~ "^/login" ||
        bereq.url ~ "^/logout" ||
        bereq.url ~ "^/lost-password" ||
        bereq.url ~ "^/my-account" ||
        bereq.url ~ "^/register" ||
        bereq.url ~ "^/signin" ||
        bereq.url ~ "^/signup" ||
        bereq.url ~ "^/wc-api" ||
        bereq.url ~ "^/wp-admin" ||
        bereq.url ~ "^/wp-login.php" ||
        bereq.url ~ "^\?add-to-cart=" ||
        bereq.url ~ "^\?wc-api="
    ) {
        #set beresp.http.Cache-Control = "private, max-age=0, no-cache, no-store";
        #set beresp.http.Expires = "Mon, 01 Jan 2001 00:00:00 GMT";
        #set beresp.http.Pragma = "no-cache";
        return (hit_for_pass);
    }

    # Don't cache HTTP authorization/authentication pages and pages with certain headers or cookies
    if (
        bereq.http.Authorization ||
        bereq.http.Authenticate ||
        bereq.http.X-Logged-In == "True" ||
        bereq.http.Cookie ~ "userID" ||
        bereq.http.Cookie ~ "joomla_[a-zA-Z0-9_]+" ||
        bereq.http.Cookie ~ "(wordpress_[a-zA-Z0-9_]+|wp-postpass|comment_author_[a-zA-Z0-9_]+|woocommerce_cart_hash|woocommerce_items_in_cart|wp_woocommerce_session_[a-zA-Z0-9]+)"
    ) {
        #set beresp.http.Cache-Control = "private, max-age=0, no-cache, no-store";
        #set beresp.http.Expires = "Mon, 01 Jan 2001 00:00:00 GMT";
        #set beresp.http.Pragma = "no-cache";
        return (hit_for_pass);
    }

    # Don't cache ajax requests
    if (beresp.http.X-Requested-With == "XMLHttpRequest" || bereq.url ~ "nocache") {
        #set beresp.http.Cache-Control = "private, max-age=0, no-cache, no-store";
        #set beresp.http.Expires = "Mon, 01 Jan 2001 00:00:00 GMT";
        #set beresp.http.Pragma = "no-cache";
        return (hit_for_pass);
    }

    # Don't cache backend response to posted requests
    if (bereq.request == "POST") {
        return (hit_for_pass);
    }

    # Ok, we're cool & ready to cache things
    # so let's clean up some headers and cookies
    # to maximize caching.

    # Check for the custom "X-Logged-In" header to identify if the visitor is a guest,
    # then unset any cookie (including session cookies) provided it's not a POST request.
    if (beresp.http.X-Logged-In == "False" && bereq.request != "POST") {
        unset beresp.http.Set-Cookie;
    }

    # Unset the "pragma" header (suggested)
    unset beresp.http.Pragma;

    # Unset the "vary" header (suggested)
    unset beresp.http.Vary;

    # Unset the "etag" header (optional)
    #unset beresp.http.etag;

    # Allow stale content, in case the backend goes down
    set beresp.grace = 24h;

    # Enforce your own cache TTL (optional)
    #set beresp.ttl = 180s;

    # Modify "expires" header (optional)
    #set beresp.http.Expires = "" + (now + beresp.ttl);

    # If your backend server does not set the right caching headers for static assets,
    # you can set them below (uncomment first and change 604800 - which 1 week - to whatever you
    # want (in seconds)
    #if (bereq.url ~ "\.(ico|jpg|jpeg|gif|png|bmp|webp|tiff|svg|svgz|pdf|mp3|flac|ogg|mid|midi|wav|mp4|webm|mkv|ogv|wmv|eot|otf|woff|ttf|rss|atom|zip|7z|tgz|gz|rar|bz2|tar|exe|doc|docx|xls|xlsx|ppt|pptx|rtf|odt|ods|odp)(\?[a-zA-Z0-9=]+)$") {
    #    set beresp.http.Cache-Control = "public, max-age=604800";
    #}

    if (bereq.url ~ "^[^?]*\.(7z|avi|bmp|bz2|css|csv|doc|docx|eot|flac|flv|gif|gz|ico|jpeg|jpg|js|less|mka|mkv|mov|mp3|mp4|mpeg|mpg|odt|ogg|ogm|opus|otf|pdf|png|ppt|pptx|rar|rtf|svg|svgz|swf|tar|tbz|tgz|ttf|txt|txz|wav|webm|webp|woff|woff2|xls|xlsx|xml|xz|zip)(\?.*)?$") {
        unset beresp.http.set-cookie;
        set beresp.do_stream = true;
    }

    # We have content to cache, but it's got no-cache or other Cache-Control values sent
    # So let's reset it to our main caching time (180s as used in this example configuration)
    # The additional parameters specified (stale-while-revalidate & stale-if-error) are used
    # by modern browsers to better control caching. Set these to twice & four times your main
    # cache time respectively.
    # This final setting will normalize cache-control headers for CMSs like Joomla
    # which set max-age=0 even when the CMS' cache is enabled.
    if (beresp.http.Cache-Control !~ "max-age" || beresp.http.Cache-Control ~ "max-age=0") {
        set beresp.http.Cache-Control = "public, max-age=180, stale-while-revalidate=360, stale-if-error=43200";
    }

    # Optionally set a larger TTL for pages with less than 180s of cache TTL
    #if (beresp.ttl < 180s) {
    #    set beresp.http.Cache-Control = "public, max-age=180, stale-while-revalidate=360, stale-if-error=43200";
    #}

    return (deliver);

}

sub vcl_deliver {

    /*
    # === The following are disabled by default - enable if you understand what you're doing ===
    # Send a special header for excluded domains/subdomains only
    # The if statement can be identical to the ones in the vcl_recv() and vcl_fetch() functions above
    if (req.http.host ~ "(domain1.tld|sub.domain2.tld)") {
        set resp.http.X-Domain-Status = "EXCLUDED";
    }

    # Enforce redirect to HTTPS for specified domains/subdomains only
    if (
        req.http.host ~ "(domain3.tld|sub.domain4.tld)" &&
        req.http.X-Forwarded-Proto !~ "(?i)https"
    ) {
        set resp.http.Location = "https://" + req.http.host + req.url;
        set resp.status = 301;
    }
    */

    # Send special headers that indicate the cache status of each web page
    if (obj.hits > 0) {
        set resp.http.X-Cache = "HIT";
        set resp.http.X-Cache-Hits = obj.hits;
    } else {
        set resp.http.X-Cache = "MISS";
    }

    return (deliver);

}

for_Varnish_4.x_or_newer_default.vcl

#########################################################################
###               The perfect Varnish 4.x+ configuration              ###
### for WordPress, Joomla, Drupal & other (common) CMS based websites ###
#########################################################################

######################
#
# UPDATED on December 15th, 2021
#
# Configuration Notes:
# 1. Default dynamic content caching respects your backend's cache-control HTTP header.
#    If however you need to enforce a different cache-control TTL,
#    do a search for "180" and replace with the new value in seconds.
#    Stale cache is served for up to 24 hours.
# 2. Make sure you update the "backend default { ... }" section with the correct IP and port
#
######################

# Varnish Reference:
# See the VCL chapters in the User-Guide at https://varnish-cache.org/docs/

# Marker to tell the VCL compiler that this VCL has been adapted to the new 4.1 format
vcl 4.1;

# Imports
import std;

# Default backend definition. Set this to point to your content server.
backend default {
    .host = "127.0.0.1"; # UPDATE this only if the web server is not on the same machine
    .port = "8080";      # UPDATE 8080 with your web server's (internal) port
}

sub vcl_recv {

    /*
    # === The following are disabled by default - enable if you understand what you're doing ===
    # Blocks
    if (req.http.user-agent ~ "^$" && req.http.referer ~ "^$") {
        return (synth(204, "No content"));
    }
    if (req.http.user-agent ~ "(ahrefs|domaincrawler|dotbot|mj12bot|semrush)") {
        return (synth(204, "Bot blocked"));
    }

    # List domains/subdomains to exclude from caching
    if (req.http.host ~ "(domain1.tld|sub.domain2.tld)") {
        return (pass);
    }
    */

    # LetsEncrypt Certbot passthrough
    if (req.url ~ "^/\.well-known/acme-challenge/") {
        return (pass);
    }

    # Forward client's IP to the backend
    if (req.restarts == 0) {
        if (req.http.X-Real-IP) {
            set req.http.X-Forwarded-For = req.http.X-Real-IP;
        } else if (req.http.X-Forwarded-For) {
            set req.http.X-Forwarded-For = req.http.X-Forwarded-For + ", " + client.ip;
        } else {
            set req.http.X-Forwarded-For = client.ip;
        }
    }

    # httpoxy
    unset req.http.proxy;

    # Normalize the query arguments (but exclude for WordPress' backend)
    if (req.url !~ "wp-admin") {
        set req.url = std.querysort(req.url);
    }

    # Non-RFC2616 or CONNECT which is weird.
    if (
        req.method != "GET" &&
        req.method != "HEAD" &&
        req.method != "PUT" &&
        req.method != "POST" &&
        req.method != "TRACE" &&
        req.method != "OPTIONS" &&
        req.method != "DELETE"
    ) {
        return (pipe);
    }

    # We only deal with GET and HEAD by default
    if (req.method != "GET" && req.method != "HEAD") {
        return (pass);
    }

    # === URL manipulation ===
    # Remove tracking query string parameters associated with analytics/social services, useless for our backend
    if (req.url ~ "(\?|&)(_bta_[a-z]+|cof|cx|fbclid|gclid|ie|mc_[a-z]+|origin|siteurl|utm_[a-z]+|zanpid)=") {
        set req.url = regsuball(req.url, "(_bta_[a-z]+|cof|cx|fbclid|gclid|ie|mc_[a-z]+|origin|siteurl|utm_[a-z]+|zanpid)=[-_A-z0-9+()%.]+&?", "");
        set req.url = regsub(req.url, "[?|&]+$", "");
    }

    # Strip a trailing ? if it exists
    if (req.url ~ "\?$") {
        set req.url = regsub(req.url, "\?$", "");
    }

    # Strip hash, server doesn't need it.
    if (req.url ~ "\#") {
        set req.url = regsub(req.url, "\#.*$", "");
    }

    # === Generic cookie manipulation ===
    # Collapse multiple cookie headers into one
    std.collect(req.http.Cookie);

    # Remove common cookies associated with analytics/social services (inc. WordPress test cookies)
    set req.http.Cookie = regsuball(req.http.Cookie, "(has_js|__utm.|_ga|_gat|utmctr|utmcmd.|utmccn.|__gads|__qc.|__atuv.|wp-settings-1|wp-settings-time-1|wordpress_test_cookie)=[^;]+(; )?", "");

    # Remove a ";" prefix in the cookie if present
    set req.http.Cookie = regsuball(req.http.Cookie, "^;\s*", "");

    # Remove blank cookies
    if (req.http.cookie ~ "^\s*$") {
        unset req.http.cookie;
    }

    # Check for the custom "X-Logged-In" header (used by K2 and other apps) to identify
    # if the visitor is a guest, then unset any cookie (including session cookies) provided
    # it's not a POST request.
    if (req.http.X-Logged-In == "False" && req.method != "POST") {
        unset req.http.Cookie;
    }

    # === DO NOT CACHE ===
    # Don't cache HTTP authorization/authentication pages and pages with certain headers or cookies
    if (
        req.http.Authorization ||
        req.http.Authenticate ||
        req.http.X-Logged-In == "True" ||
        req.http.Cookie ~ "userID" ||
        req.http.Cookie ~ "joomla_[a-zA-Z0-9_]+" ||
        req.http.Cookie ~ "(wordpress_[a-zA-Z0-9_]+|wp-postpass|comment_author_[a-zA-Z0-9_]+|woocommerce_cart_hash|woocommerce_items_in_cart|wp_woocommerce_session_[a-zA-Z0-9]+)"
    ) {
        #set req.http.Cache-Control = "private, max-age=0, no-cache, no-store";
        #set req.http.Expires = "Mon, 01 Jan 2001 00:00:00 GMT";
        #set req.http.Pragma = "no-cache";
        return (pass);
    }

    # Exclude the following paths (e.g. backend admins, user pages or ad URLs that require tracking)
    # In Joomla specifically, you are advised to create specific entry points (URLs) for users to
    # interact with the site (either common user logins or even commenting), e.g. make a menu item
    # to point to a user login page (e.g. /login), including all related functionality such as
    # password reset, email reminder and so on.
    if (
        req.url ~ "^/addons" ||
        req.url ~ "^/administrator" ||
        req.url ~ "^/cart" ||
        req.url ~ "^/checkout" ||
        req.url ~ "^/component/banners" ||
        req.url ~ "^/component/socialconnect" ||
        req.url ~ "^/component/users" ||
        req.url ~ "^/connect" ||
        req.url ~ "^/contact" ||
        req.url ~ "^/login" ||
        req.url ~ "^/logout" ||
        req.url ~ "^/lost-password" ||
        req.url ~ "^/my-account" ||
        req.url ~ "^/register" ||
        req.url ~ "^/signin" ||
        req.url ~ "^/signup" ||
        req.url ~ "^/wc-api" ||
        req.url ~ "^/wp-admin" ||
        req.url ~ "^/wp-login.php" ||
        req.url ~ "^\?add-to-cart=" ||
        req.url ~ "^\?wc-api="
    ) {
        #set req.http.Cache-Control = "private, max-age=0, no-cache, no-store";
        #set req.http.Expires = "Mon, 01 Jan 2001 00:00:00 GMT";
        #set req.http.Pragma = "no-cache";
        return (pass);
    }

    # Don't cache ajax requests
    if (req.http.X-Requested-With == "XMLHttpRequest" || req.url ~ "nocache") {
        #set req.http.Cache-Control = "private, max-age=0, no-cache, no-store";
        #set req.http.Expires = "Mon, 01 Jan 2001 00:00:00 GMT";
        #set req.http.Pragma = "no-cache";
        return (pass);
    }

    # === STATIC FILES ===
    # Properly handle different encoding types
    if (req.http.Accept-Encoding) {
        if (req.url ~ "\.(jpg|jpeg|png|gif|gz|tgz|bz2|tbz|mp3|ogg|swf)$") {
            # No point in compressing these
            unset req.http.Accept-Encoding;
        } elseif (req.http.Accept-Encoding ~ "gzip") {
            set req.http.Accept-Encoding = "gzip";
        } elseif (req.http.Accept-Encoding ~ "deflate") {
            set req.http.Accept-Encoding = "deflate";
        } else {
            # unknown algorithm (aka crappy browser)
            unset req.http.Accept-Encoding;
        }
    }

    # Remove all cookies for static files & deliver directly
    if (req.url ~ "^[^?]*\.(7z|avi|bmp|bz2|css|csv|doc|docx|eot|flac|flv|gif|gz|ico|jpeg|jpg|js|less|mka|mkv|mov|mp3|mp4|mpeg|mpg|odt|ogg|ogm|opus|otf|pdf|png|ppt|pptx|rar|rtf|svg|svgz|swf|tar|tbz|tgz|ttf|txt|txz|wav|webm|webp|woff|woff2|xls|xlsx|xml|xz|zip)(\?.*)?$") {
        unset req.http.Cookie;
        return (hash);
    }

    return (hash);

}

sub vcl_backend_response {

    /*
    # === The following are disabled by default - enable if you understand what you're doing ===
    # List domains/subdomains to exclude from caching
    if (bereq.http.host ~ "(domain1.tld|sub.domain2.tld)") {
        set beresp.uncacheable = true;
        return (deliver);
    }
    */

    # Don't cache 50x responses
    if (
        beresp.status == 500 ||
        beresp.status == 502 ||
        beresp.status == 503 ||
        beresp.status == 504
    ) {
        return (abandon);
    }

    # === DO NOT CACHE ===
    # Exclude the following paths (e.g. backend admins, user pages or ad URLs that require tracking)
    # In Joomla specifically, you are advised to create specific entry points (URLs) for users to
    # interact with the site (either common user logins or even commenting), e.g. make a menu item
    # to point to a user login page (e.g. /login), including all related functionality such as
    # password reset, email reminder and so on.
    if (
        bereq.url ~ "^/addons" ||
        bereq.url ~ "^/administrator" ||
        bereq.url ~ "^/cart" ||
        bereq.url ~ "^/checkout" ||
        bereq.url ~ "^/component/banners" ||
        bereq.url ~ "^/component/socialconnect" ||
        bereq.url ~ "^/component/users" ||
        bereq.url ~ "^/connect" ||
        bereq.url ~ "^/contact" ||
        bereq.url ~ "^/login" ||
        bereq.url ~ "^/logout" ||
        bereq.url ~ "^/lost-password" ||
        bereq.url ~ "^/my-account" ||
        bereq.url ~ "^/register" ||
        bereq.url ~ "^/signin" ||
        bereq.url ~ "^/signup" ||
        bereq.url ~ "^/wc-api" ||
        bereq.url ~ "^/wp-admin" ||
        bereq.url ~ "^/wp-login.php" ||
        bereq.url ~ "^\?add-to-cart=" ||
        bereq.url ~ "^\?wc-api="
    ) {
        #set beresp.http.Cache-Control = "private, max-age=0, no-cache, no-store";
        #set beresp.http.Expires = "Mon, 01 Jan 2001 00:00:00 GMT";
        #set beresp.http.Pragma = "no-cache";
        set beresp.uncacheable = true;
        return (deliver);
    }

    # Don't cache HTTP authorization/authentication pages and pages with certain headers or cookies
    if (
        bereq.http.Authorization ||
        bereq.http.Authenticate ||
        bereq.http.X-Logged-In == "True" ||
        bereq.http.Cookie ~ "userID" ||
        bereq.http.Cookie ~ "joomla_[a-zA-Z0-9_]+" ||
        bereq.http.Cookie ~ "(wordpress_[a-zA-Z0-9_]+|wp-postpass|comment_author_[a-zA-Z0-9_]+|woocommerce_cart_hash|woocommerce_items_in_cart|wp_woocommerce_session_[a-zA-Z0-9]+)"
    ) {
        #set beresp.http.Cache-Control = "private, max-age=0, no-cache, no-store";
        #set beresp.http.Expires = "Mon, 01 Jan 2001 00:00:00 GMT";
        #set beresp.http.Pragma = "no-cache";
        set beresp.uncacheable = true;
        return (deliver);
    }

    # Don't cache ajax requests
    if (beresp.http.X-Requested-With == "XMLHttpRequest" || bereq.url ~ "nocache") {
        #set beresp.http.Cache-Control = "private, max-age=0, no-cache, no-store";
        #set beresp.http.Expires = "Mon, 01 Jan 2001 00:00:00 GMT";
        #set beresp.http.Pragma = "no-cache";
        set beresp.uncacheable = true;
        return (deliver);
    }

    # Don't cache backend response to posted requests
    if (bereq.method == "POST") {
        set beresp.uncacheable = true;
        return (deliver);
    }

    # Ok, we're cool & ready to cache things
    # so let's clean up some headers and cookies
    # to maximize caching.

    # Check for the custom "X-Logged-In" header to identify if the visitor is a guest,
    # then unset any cookie (including session cookies) provided it's not a POST request.
    if (beresp.http.X-Logged-In == "False" && bereq.method != "POST") {
        unset beresp.http.Set-Cookie;
    }

    # Unset the "pragma" header (suggested)
    unset beresp.http.Pragma;

    # Unset the "vary" header (suggested)
    unset beresp.http.Vary;

    # Unset the "etag" header (optional)
    #unset beresp.http.etag;

    # Allow stale content, in case the backend goes down
    set beresp.grace = 24h;

    # Enforce your own cache TTL (optional)
    #set beresp.ttl = 180s;

    # Modify "expires" header (optional)
    #set beresp.http.Expires = "" + (now + beresp.ttl);

    # If your backend server does not set the right caching headers for static assets,
    # you can set them below (uncomment first and change 604800 - which 1 week - to whatever you
    # want (in seconds)
    #if (bereq.url ~ "\.(ico|jpg|jpeg|gif|png|bmp|webp|tiff|svg|svgz|pdf|mp3|flac|ogg|mid|midi|wav|mp4|webm|mkv|ogv|wmv|eot|otf|woff|ttf|rss|atom|zip|7z|tgz|gz|rar|bz2|tar|exe|doc|docx|xls|xlsx|ppt|pptx|rtf|odt|ods|odp)(\?[a-zA-Z0-9=]+)$") {
    #    set beresp.http.Cache-Control = "public, max-age=604800";
    #}

    if (bereq.url ~ "^[^?]*\.(7z|avi|bmp|bz2|css|csv|doc|docx|eot|flac|flv|gif|gz|ico|jpeg|jpg|js|less|mka|mkv|mov|mp3|mp4|mpeg|mpg|odt|ogg|ogm|opus|otf|pdf|png|ppt|pptx|rar|rtf|svg|svgz|swf|tar|tbz|tgz|ttf|txt|txz|wav|webm|webp|woff|woff2|xls|xlsx|xml|xz|zip)(\?.*)?$") {
        unset beresp.http.set-cookie;
        set beresp.do_stream = true;
    }

    # We have content to cache, but it's got no-cache or other Cache-Control values sent
    # So let's reset it to our main caching time (180s as used in this example configuration)
    # The additional parameters specified (stale-while-revalidate & stale-if-error) are used
    # by modern browsers to better control caching. Set these to twice & four times your main
    # cache time respectively.
    # This final setting will normalize cache-control headers for CMSs like Joomla
    # which set max-age=0 even when the CMS' cache is enabled.
    if (beresp.http.Cache-Control !~ "max-age" || beresp.http.Cache-Control ~ "max-age=0") {
        set beresp.http.Cache-Control = "public, max-age=180, stale-while-revalidate=360, stale-if-error=43200";
    }

    # Optionally set a larger TTL for pages with less than 180s of cache TTL
    #if (beresp.ttl < 180s) {
    #    set beresp.http.Cache-Control = "public, max-age=180, stale-while-revalidate=360, stale-if-error=43200";
    #}

    return (deliver);

}

sub vcl_deliver {

    /*
    # === The following are disabled by default - enable if you understand what you're doing ===
    # Send a special header for excluded domains/subdomains only
    # The if statement can be identical to the ones in the vcl_recv() and vcl_backend_response() functions above
    if (req.http.host ~ "(domain1.tld|sub.domain2.tld)") {
        set resp.http.X-Domain-Status = "EXCLUDED";
    }

    # Enforce redirect to HTTPS for specified domains/subdomains only
    if (
        req.http.host ~ "(domain3.tld|sub.domain4.tld)" &&
        req.http.X-Forwarded-Proto !~ "(?i)https"
    ) {
        set resp.http.Location = "https://" + req.http.host + req.url;
        set resp.status = 301;
    }
    */

    # Send special headers that indicate the cache status of each web page
    if (obj.hits > 0) {
        set resp.http.X-Cache = "HIT";
        set resp.http.X-Cache-Hits = obj.hits;
    } else {
        set resp.http.X-Cache = "MISS";
    }

    return (deliver);

}

@fevangelou thank you very much for the information provided. It helped me to know where to stop looking and focus on where the problem could be.
It was a configuration problem for apache and wordpress, it has already been solved.
Thanks again!

Hello @fevangelou I don't know if it's ok to ask via github, but need help implementing VCL for my multisite woocommerce store with one domain and two different sites (for two languages Dutch and French) with standard vcl template from my hosting it does not work. Could you help me to implement such VCL and adjust it for my website. I don't know if it's ok to ask it here on github. Can you write a private message here?
Greetings from Belgium

@Amelia-Mar You can email me on engintron [at] gmail [dot] com for any custom work/requests.

Hello Fevangelou, I wrote you an email, but didn’t got an answer yet :) With kind regards,Amelia Verzonden via Yahoo Mail voor iPhone Op dinsdag, april 13, 2021, 3:47 PM schreef Fotis Evangelou ***@***.***>: @fevangelou commented on this gist. @Amelia-Mar You can email me on engintron [at] gmail [dot] com for any custom work/requests. — You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub, or unsubscribe.

@fevangelou This is exactly what I was looking for, very cool! I am pretty much new to Varnish. At the risk of making a fool out of myself: would this work in conjunction with a purge plugin within Wordpress? I know the TTL is 180 seconds. I will like to up this considerably and purge new and edited content only via some plugin. Would this work? I tried Proxy Cache Purge plugin but somehow it didn't work

Another thing that is tricky to grasp for me. I installed it and it seemed to work perfectly well straight away. However I am used to seeing MISS the first time a page is being served for the simple reason it is not in Varnish cache yet. However on my test site all (not logged in) pages show HIT, even if they have never been requested yet. Am I missing something? Thanks for your patience ;)

@Japhys You can use a WP plugin for purging Varnish's cache but it would require additional code in the configs above so Varnish can understand a PURGE request sent from such a WP plugin. In any case, as this is plugin-specific, some documentation must exist on how to make Varnish respond to PURGE requests on the WP plugin dev's site.

As for the 2nd issue you mention, if your test site is publicly accessible and has Google Analytics on, it's crawled by Google's bots (and perhaps others) that force your content to be cached.

@Japhys You can use a WP plugin for purging Varnish's cache but it would require additional code in the configs above so Varnish can understand a PURGE request sent from such a WP plugin. In any case, as this is plugin-specific, some documentation must exist on how to make Varnish respond to PURGE requests on the WP plugin dev's site.

As for the 2nd issue you mention, if your test site is publicly accessible and has Google Analytics on, it's crawled by Google's bots (and perhaps others) that force your content to be cached.

Thanks @fevangelou I am starting to get a better understanding of how this all works!

As for the HIT/MISS: it is a closed site but who knows, maybe I closed it after the culprit indexed it :) I'll do some tests later.

Hi, on varnish 4 I have similar problem as in https://stackoverflow.com/questions/60918936/varnish-4-always-returns-x-cache-pass-uncacheable . Has anyone had met such case ?
Regards

@Piterek79 Well, it depends on your config and whether the current URL falls into one of the patterns that disable caching...

Configurations updated on December 15th, 2021. They include improved URL & cookie filtering as well as refined comments. Both configurations for Varnish 3.x and 4.x+ are now much more inline too, with the exception of newer features in 4.x+ that cannot be used in 3.x (e.g. the cookie collapsing option).

Enjoy ;)

Hello,
Why not varnish 6?

4+ covers 6. Duh...

Sorry my mistake. Thanks for you work!

I'd expect nothing changed, but did you test it with Joomla 4 too? Thanks.

@MyWay Yes.

can this configuration be applied to moodle cms? please make config varnish for moodle, i want to bypass session id and login page with url https://example.com/login/index.php

@fevangelou Great work Fotis, thanks!

Would it be possible to add some example VCL for purging? I’m writing the purging logic myself (not using a plugin) but I’m not sure how to set things up in Varnish.

I’d like to be able to purge by host and url with wildcards. Eg:

domain1.com/shop/*
domain2.com/blog/*

Awesome!
Tnx

I'm having issue. The cart is cached for all visitors.