HTML caching with Cloudflare made an online shop 28 percent faster

The result

On March 20, 2019 we enabled HTML caching in Cloudflare for a German online shop. The page timings report in Google Analytics showed the effect from that day on. The average server response time fell from around 1.5 seconds to under one second.

We compared the period from 21 March, the day after the switch, with the period before. During that time there were no other changes that would have noticeably affected load or response times. The measurement uses the load time from Google Analytics. Core Web Vitals such as LCP did not exist yet.

Standard caching and HTML caching

Cloudflare sits as a network of servers between visitors and your own web server. Files in the cache are delivered by Cloudflare directly. Your own server then does not have to answer every request itself.

By default, Cloudflare only caches static files, recognized by their file extension, such as images, CSS, JavaScript, fonts and PDFs. Cloudflare fetches HTML pages fresh from the server for every request, because their content can change per visitor and over time. According to the Cloudflare documentation, this is still the case today. HTML caching is an additional setting that you switch on specifically for certain pages and visitors.

Three hurdles in the online shop

The cart preview

Every page of the shop showed a cart preview with the number of items and the total. Cloudflare stores the version of a page that the first real visitor fetches. Without an exception, the next visitor would have seen the previous visitor's cart.

The name of logged-in customers

After login, the shop showed the customer's name in the header. This name would also have ended up in the cache and been shown to all subsequent visitors.

Prices that change every 30 minutes

Prices were updated every 30 minutes. Which prices would change in a given run was not predictable. A cached product or category page therefore must not show old prices for long.

Cookie and expiry time

We were looking for a way that needed little development effort. A clean option would have been to insert the cart and name only in the browser via JavaScript. The HTML would then have been the same for everyone and cacheable without restriction. But that would have required larger changes to the shop.

So we only served anonymous visitors with an empty cart from the cache. Cloudflare offered the setting “Bypass Cache on Cookie” for this, at the time only on the Business and Enterprise plans. As soon as a visitor put something in the cart or logged in, the shop set a cookie called bypassCache. If it was present, the request bypassed the cache and went to the server. For the prices, the “Edge Cache TTL” made sure that Cloudflare discarded a stored page after 30 minutes.

Implementation in three Page Rules

Page Rules apply from top to bottom, and only the first matching rule applies to a URL. Rule 2 therefore affects no URL from rule 1, and rule 3 only the rest.

The 30 minutes were a compromise. Cloudflare stores a page on the first request and discards it 30 minutes later, regardless of when the price update happens. In the worst case, a visitor saw an old price for at most 30 minutes. During the test phase there were no complaints about prices changing in the cart.

The cookie via Google Tag Manager

Depending on the shop system, suitable cookies already exist, or developers set them. Here we set the cookie with Google Tag Manager, because that required no development effort. Two variables of the type “DOM element” read the text of the login link and the value of the cart preview. From these we built four triggers for the states “product in cart”, “empty cart”, “logged in” and “logged out”. One tag set bypassCache as soon as a product was in the cart or the customer was logged in. A second tag deleted the cookie when the cart was empty and on logout. A solution in the shop itself is more robust. Tag Manager was the quick route to a test here.

Limits and next steps

According to the e-commerce data in Google Analytics, visitors put nothing in the cart in about 95 percent of sessions. The share of visitors who were not logged in was even higher. Most visitors therefore already benefited from the simple solution.

• Pages with little traffic were rarely in the cache, because the copy expired after 30 minutes. This affects crawlers more than visitors, because they often fetch exactly these pages.
• The planned next step was to purge only the URLs with changed prices from the cache via the Cloudflare API after each price update. The expiry time could then have been much longer.
• The cart and personal areas were later to be loaded in the browser via JavaScript, so that visitors with a cart would also get cached pages.

What has changed since then?

The case study dates from 2019. The measurement and result apply to that period, and we have not re-tested them. Some of the technology is different today.

• Page Rules are being replaced by Cache Rules. There, “Cache Everything” is called “Eligible for cache”, the Edge Cache TTL becomes the Edge TTL, and “Bypass Cache on Cookie” becomes a rule with the http.cookie field and the “Bypass cache” setting. Cache Rules are available on all plans.
• The order is reversed. Cache Rules can be stacked, and in a conflict the last matching rule wins. The exception for cart and login therefore goes at the end.
• Universal Analytics has been switched off. Today you measure the server response as TTFB and the effect on users with field data from CrUX, such as LCP. More on this in the Core Web Vitals guide.

This is what the setup would look like as Cache Rules today.

Common mistakes

• Cache Everything for the whole domain without an exception for cart, login and checkout.
• Keeping the Page Rules order when switching to Cache Rules. The exception then sits at the top and gets overridden.
• A long expiry time for pages with prices, without targeted purging from the cache.
• The cookie is set but never deleted on logout or when the cart is empty. These visitors then never get cached pages.
• Not checking after the switch whether the response shows HIT or BYPASS in the cf-cache-status header.

FAQ