CDNs and Edge Caching: A Real Cache in Front of a Slow Origin

A CDN is a cache that sits between your users and your server (the origin), close to the users. You can’t see Cloudflare’s edge from your laptop, but you can build one: a slow origin, an nginx edge cache in front of it, and curl reporting what the edge did with every request. Everything here runs in Docker, with no cloud account.

The lab: a slow origin and an edge cache

The origin takes 300 ms per request, says how long its pages may be cached, and counts how many full pages it has built, so you can see when a request really reached it.

# origin.py - a slow origin server: 300 ms per response, with Cache-Control and an ETag
import os
import time
from http.server import BaseHTTPRequestHandler, ThreadingHTTPServer

CACHE = os.environ.get("CACHE_CONTROL", "public, max-age=10, s-maxage=4")
full = 0  # full 200 responses this origin has built
seen = 0  # every request that reached the origin, 304s included


class Origin(BaseHTTPRequestHandler):
    def do_GET(self):
        global full, seen
        if self.path == "/stats":
            return self.reply(200, f"origin saw {seen} request(s), built {full} full page(s)\n", "no-store")
        seen += 1
        time.sleep(0.3)  # stands in for a slow database query or page render
        etag = '"v1"'
        if self.headers.get("If-None-Match") == etag:
            return self.reply(304, "", CACHE, etag)
        full += 1
        private = self.path.startswith("/me")
        self.reply(200, f"page {self.path}\n", "private, max-age=60" if private else CACHE,
                   etag, cookie="session=abc123" if private else None)

    def reply(self, status, body, cache, etag=None, cookie=None):
        data = body.encode()
        self.send_response(status)
        self.send_header("Cache-Control", cache)
        self.send_header("X-Origin-Hit", str(full))
        if etag:
            self.send_header("ETag", etag)
        if cookie:
            self.send_header("Set-Cookie", cookie)
        if os.environ.get("VARY"):
            self.send_header("Vary", os.environ["VARY"])
        self.send_header("Content-Length", str(len(data)))
        self.end_headers()
        self.wfile.write(data)

    def log_message(self, *args):
        pass


ThreadingHTTPServer(("", 8000), Origin).serve_forever()

The edge is plain nginx with a cache zone. add_header X-Cache-Status reports the cache’s decision on every response: MISS, HIT, EXPIRED, REVALIDATED, STALE or BYPASS.

# edge.conf - nginx as a CDN edge: cache what the origin allows, report HIT or MISS
proxy_cache_path /var/cache/edge keys_zone=edge:10m inactive=1h;

server {
    listen 80;
    location / {
        proxy_pass http://origin:8000;
        proxy_cache edge;
        proxy_cache_revalidate on;
        proxy_cache_use_stale error timeout;
        proxy_cache_bypass $http_x_refresh;
        add_header X-Cache-Status $upstream_cache_status always;
    }
}

get.sh asks the edge for each path and prints the time, the cache status and which origin build served it. +N waits N seconds, and stats asks the origin directly for its counters. probe.sh runs it in a curl container on the lab network, and start.sh gives every example a fresh, empty cache.

# get.sh - fetch each path through the edge; "+N" waits N seconds, "stats" asks the origin
for arg in "$@"; do
  case $arg in
    +*) sleep "${arg#+}"; echo "   (${arg#+}s later)"; continue ;;
    stats) curl -s http://origin:8000/stats; continue ;;
  esac
  curl -s -o /dev/null -H "${HDR:-X-Probe: 1}" \
    -w '%{time_total} %header{x-cache-status} %header{x-origin-hit}\n' "http://edge$arg" |
    while read -r t status hit; do
      printf '%-14s %.3fs  %-12s origin build #%s\n' "$arg" "$t" "$status" "$hit"
    done
done

# probe.sh - run get.sh in a curl container on the lab network
export MSYS_NO_PATHCONV=1  # Git Bash on Windows: keep /lab as a container path
docker run --rm --network cdnlab -e HDR -v "$PWD:/lab" -w /lab curlimages/curl:8.17.0 sh get.sh "$@"

# start.sh - a fresh lab: start the origin first (nginx resolves it at startup), then the edge
export MSYS_NO_PATHCONV=1
conf=${1:-edge.conf}
docker rm -f edge origin >/dev/null 2>&1
while docker ps -a --format '{{.Names}}' | grep -qxE 'edge|origin'; do sleep 0.5; done
docker network create cdnlab >/dev/null 2>&1
docker run -d --rm --name origin --network cdnlab -v "$PWD:/lab" -e VARY \
  -e CACHE_CONTROL="${2:-public, max-age=10, s-maxage=4}" python:3.12-slim python /lab/origin.py >/dev/null
docker run -d --rm --name edge --network cdnlab \
  -v "$PWD/$conf:/etc/nginx/conf.d/default.conf:ro" nginx:1.29-alpine >/dev/null
sleep 2
echo "lab up: origin + edge ($conf)"

Save all five files in one folder and run the commands below from it. The origin starts first because nginx looks up the name origin once, when it starts, and refuses to start if the name doesn’t exist yet.

1. A miss, then hits

$ sh start.sh
lab up: origin + edge (edge.conf)
$ sh probe.sh /page /page /page stats
/page          0.304s  MISS         origin build #1
/page          0.002s  HIT          origin build #1
/page          0.002s  HIT          origin build #1
origin saw 1 request(s), built 1 full page(s)

The first request missed: the edge had nothing, so it fetched the page from the origin and the user waited the full 300 ms. The next two were answered from the edge’s cache in 1 to 2 ms, still showing origin build #1, and the origin’s counter confirms it served one request in total. Across four runs the miss took 0.304 to 0.305 s and the hits 0.001 to 0.002 s. On a real CDN the gap is bigger again, because the edge is also closer to the user than the origin is.

2. s-maxage is the edge’s clock, max-age is the browser’s

The origin sends Cache-Control: public, max-age=10, s-maxage=4.

$ sh start.sh
lab up: origin + edge (edge.conf)
$ sh probe.sh /page +2 /page +4 /page
/page          0.305s  MISS         origin build #1
   (2s later)
/page          0.002s  HIT          origin build #1
   (4s later)
/page          0.305s  REVALIDATED  origin build #1

Two seconds in, the page was still fresh. Six seconds in, the edge went back to the origin, even though max-age=10 says the page is good for ten. s-maxage applies only to shared caches such as a CDN or a reverse proxy, and it overrides max-age there, so the edge used 4 seconds while a browser keeps its own copy for 10. That split is useful: browsers can hold a page longer than the edge does, and the edge picks up changes sooner. nginx counts these lifetimes in whole seconds, so expiry can land up to a second either side. The examples wait well past the boundary for that reason.

3. Revalidation: an ETag saves the body, not the wait

When a cached page expires, the edge doesn’t throw it away. It asks the origin whether it changed, sending the page’s ETag in an If-None-Match header (that’s what proxy_cache_revalidate on turns on).

$ sh start.sh
lab up: origin + edge (edge.conf)
$ sh probe.sh /page +6 /page /page stats
/page          0.305s  MISS         origin build #1
   (6s later)
/page          0.304s  REVALIDATED  origin build #1
/page          0.001s  HIT          origin build #1
origin saw 2 request(s), built 1 full page(s)

After six seconds the copy had expired, so the edge sent If-None-Match: "v1". The tag still matched, so the origin answered 304 Not Modified with no body, and the edge kept serving the page it had: still origin build #1, and the origin reports 2 requests but only 1 full page built. The next request was an ordinary hit again. Look at the time on the REVALIDATED line, though: 0.304 s in all four runs, the same as a miss. Revalidation saves bandwidth and the origin’s work of building the page. It doesn’t save the user the round trip. Example 5 shows the setting that does.

4. Query strings fragment the cache

nginx’s default cache key is the scheme, host and full request URI, query string included. Tracking tags like ?utm= make the same page look like different pages.

$ sh start.sh
lab up: origin + edge (edge.conf)
$ sh probe.sh "/page?utm=a" "/page?utm=b" "/page?utm=a" stats
/page?utm=a    0.305s  MISS         origin build #1
/page?utm=b    0.303s  MISS         origin build #2
/page?utm=a    0.001s  HIT          origin build #1
origin saw 2 request(s), built 2 full page(s)

/page?utm=a and /page?utm=b are the same page, but they’re two cache entries, and each one cost a trip to the origin. Only an exact repeat (?utm=a again) hit. With a different tag on every campaign link, the cache stops helping. This version builds the key from the path alone:

# edge-key.conf - edge.conf, but the cache key ignores the query string
proxy_cache_path /var/cache/edge keys_zone=edge:10m inactive=1h;

server {
    listen 80;
    location / {
        proxy_pass http://origin:8000;
        proxy_cache edge;
        proxy_cache_key $scheme$proxy_host$uri;
        proxy_cache_revalidate on;
        proxy_cache_use_stale error timeout;
        proxy_cache_bypass $http_x_refresh;
        add_header X-Cache-Status $upstream_cache_status always;
    }
}

$ sh start.sh edge-key.conf
lab up: origin + edge (edge-key.conf)
$ sh probe.sh "/page?utm=a" "/page?utm=b" "/page?utm=c" stats
/page?utm=a    0.304s  MISS         origin build #1
/page?utm=b    0.001s  HIT          origin build #1
/page?utm=c    0.001s  HIT          origin build #1
origin saw 1 request(s), built 1 full page(s)

Now all three URLs share one entry: one origin request for three different query strings. The catch is that this key ignores every query parameter, so /search?q=shoes and /search?q=hats would get the same cached answer. A real edge drops only the parameters that don’t change the response. On nginx that’s a map that builds a normalised key; CDNs have a “cache key” setting for it. Getting this wrong in the other direction (too few things in the key) is how one user ends up seeing a page meant for another.

5. When the origin hangs

edge.conf already has proxy_cache_use_stale error timeout: if the origin fails, serve the expired copy instead of an error. docker pause freezes the origin without closing its port, which is what a hung or overloaded server looks like from the outside.

$ sh start.sh
lab up: origin + edge (edge.conf)
$ sh probe.sh /page
/page          0.310s  MISS         origin build #1
$ docker pause origin
origin
$ sh probe.sh +5 /page
   (5s later)
/page          60.010s  STALE        origin build #1
$ docker unpause origin
origin

The page did survive the outage, but the user waited a full minute for it. 60.008 to 60.029 s across four runs. The connection to the frozen origin opened (its kernel still answers), so nginx sat in proxy_read_timeout, which defaults to 60 seconds, and only gave up and served the stale copy after that. Its error log says so: upstream timed out ... while reading response header from upstream. Lower the timeouts:

# edge-stale.conf - edge.conf, plus short upstream timeouts
proxy_cache_path /var/cache/edge keys_zone=edge:10m inactive=1h;

server {
    listen 80;
    location / {
        proxy_pass http://origin:8000;
        proxy_cache edge;
        proxy_cache_revalidate on;
        proxy_connect_timeout 1s;
        proxy_read_timeout 1s;
        proxy_cache_use_stale error timeout;
        proxy_cache_bypass $http_x_refresh;
        add_header X-Cache-Status $upstream_cache_status always;
    }
}

$ sh start.sh edge-stale.conf
lab up: origin + edge (edge-stale.conf)
$ sh probe.sh /page
/page          0.305s  MISS         origin build #1
$ docker pause origin
origin
$ sh probe.sh +5 /page
   (5s later)
/page          1.007s  STALE        origin build #1
$ docker unpause origin
origin

Now the stale page arrives after one second (1.003 to 1.016 s across four runs) instead of sixty. A killed origin is less dangerous than a hung one: its address refuses connections at once, and the edge goes stale immediately. Pick timeouts just above your slowest legitimate response, or slow pages start being served stale.

The last second can go too. proxy_cache_background_update serves the stale copy straight away and refreshes it from the origin behind the scenes. That’s what the stale-while-revalidate directive asks for.

# edge-swr.conf - edge-stale.conf, plus stale-while-revalidate: answer stale, refresh behind
proxy_cache_path /var/cache/edge keys_zone=edge:10m inactive=1h;

server {
    listen 80;
    location / {
        proxy_pass http://origin:8000;
        proxy_cache edge;
        proxy_cache_revalidate on;
        proxy_connect_timeout 1s;
        proxy_read_timeout 1s;
        proxy_cache_use_stale error timeout updating;
        proxy_cache_background_update on;
        proxy_cache_bypass $http_x_refresh;
        add_header X-Cache-Status $upstream_cache_status always;
    }
}

$ sh start.sh edge-swr.conf
lab up: origin + edge (edge-swr.conf)
$ sh probe.sh /page +5 /page +0.5 /page stats
/page          0.305s  MISS         origin build #1
   (5s later)
/page          0.008s  STALE        origin build #1
   (0.5s later)
/page          0.002s  HIT          origin build #1
origin saw 2 request(s), built 1 full page(s)

This time the origin was healthy. The expired page came back in 1 to 12 ms as STALE (four runs), nginx revalidated it in the background (the origin saw a second request but built only one page), and the next request was a plain HIT. Compare example 3, where the same expiry cost the user 0.3 s. The price is that one user gets a copy a few seconds out of date, which is fine for most pages and wrong for, say, a stock level at checkout.

6. Forcing a refresh, and why versioned URLs beat purging

Sometimes a page changes before its TTL runs out. edge.conf has proxy_cache_bypass $http_x_refresh: a request carrying an X-Refresh header skips the cache, fetches a fresh copy and stores it.

$ sh start.sh
lab up: origin + edge (edge.conf)
$ sh probe.sh /page /page
/page          0.310s  MISS         origin build #1
/page          0.002s  HIT          origin build #1
$ HDR='X-Refresh: 1' sh probe.sh /page
/page          0.306s  BYPASS       origin build #2
$ sh probe.sh /page /app.v1.js /app.v2.js /app.v2.js
/page          0.003s  HIT          origin build #2
/app.v1.js     0.304s  MISS         origin build #3
/app.v2.js     0.303s  MISS         origin build #4
/app.v2.js     0.002s  HIT          origin build #4

The BYPASS request went to the origin and got build #2, and the next plain request got build #2 from the cache. Two warnings. First, anyone on the internet can send that header, so on a real edge it has to be limited to your own network or a secret token, or it becomes a way to push all your traffic onto the origin. Second, open-source nginx has no purge API to delete one entry. Commercial CDNs do (an API call or a dashboard button per URL or tag), though a purge can take a while to reach every edge location.

The usual answer for static files is to never change them in place: put a version in the name. /app.v2.js is a new URL, so it’s a guaranteed miss followed by hits, with nothing to purge, and the old /app.v1.js can stay cached for as long as you like.

7. Personalised pages stay at the origin

The origin answers /me the way a logged-in page would: Cache-Control: private, max-age=60 and a Set-Cookie header.

$ sh start.sh
lab up: origin + edge (edge.conf)
$ sh probe.sh /me /me /me stats
/me            0.304s  MISS         origin build #1
/me            0.303s  MISS         origin build #2
/me            0.303s  MISS         origin build #3
origin saw 3 request(s), built 3 full page(s)

Three requests, three misses, three pages built. private means “a browser may keep this, a shared cache may not”, and nginx obeys it. A separate run with only private and no cookie also missed every time. nginx’s documentation lists Set-Cookie as another reason it won’t cache a response. This is the safety net behind the warning in example 4: a page with one user’s name on it must never be served to the next user, so the origin has to mark it. A CDN can still help a page like this by caching the parts that are the same for everyone (CSS, scripts, images, a shared page shell) and fetching only the personal part.

8. What the edge saves

bench.sh sends 100 requests for /page one after another and adds up the time, then asks the origin what it saw. It runs once against the origin directly and once through the edge.

# bench.sh - 100 requests for /page through one host, then what the origin saw
host=$1
for i in $(seq 100); do
  curl -s -o /dev/null -w '%{time_total}\n' "http://$host/page"
done | awk -v h="$host" '{ s += $1 } END { printf "%-12s 100 requests in %.2fs\n", h, s }'
curl -s http://origin:8000/stats

$ sh start.sh
lab up: origin + edge (edge.conf)
$ MSYS_NO_PATHCONV=1 docker run --rm --network cdnlab -v "$PWD:/lab" -w /lab curlimages/curl:8.17.0 sh bench.sh origin:8000
origin:8000  100 requests in 30.24s
origin saw 100 request(s), built 100 full page(s)
$ sh start.sh
lab up: origin + edge (edge.conf)
$ MSYS_NO_PATHCONV=1 docker run --rm --network cdnlab -v "$PWD:/lab" -w /lab curlimages/curl:8.17.0 sh bench.sh edge
edge         100 requests in 0.41s
origin saw 1 request(s), built 1 full page(s)

Through the edge, 100 requests took 0.41 to 0.43 s instead of 30.24 to 30.31 s, about 70 to 74 times faster in each of the four runs on this machine. More important for the origin: it served 1 request instead of 100. These 100 requests all fit inside one 4-second s-maxage, so a longer test would add one revalidation every 4 seconds, still far fewer than one per request. That ratio of requests the edge answers to requests it passes on (the hit ratio) is the number a CDN is judged by, and examples 4 and 7 are the two usual reasons it’s lower than expected.

How this maps to a real CDN

A CDN runs this edge in hundreds of locations (points of presence), routes each user to a nearby one by DNS or anycast, and often adds a middle tier (an “origin shield”) so that a miss in one city doesn’t send one more request to your origin from every location. It also usually terminates TLS. The rules above still apply: cache keys, Cache-Control, revalidation and stale serving are what you configure. Each CDN’s defaults differ, though (some don’t cache HTML until a cache rule says so), so check what yours caches out of the box. A CDN is for responses shared between users; caching data inside your application (cache-aside, in front of a database) is a different layer: see Cache-Aside with Redis. When an edge’s cache fills up it has to drop entries, which is cache eviction. To see the HTTP requests and headers themselves as raw text, see HTTP on the wire.

Common gotchas

• The origin must start first. nginx resolves origin once, at startup, and exits with host not found in upstream "origin" if it can’t.
• Default upstream timeouts are 60 seconds. A hung origin turns “serve stale on error” into a one-minute wait (example 5).
• The default cache key includes the query string. Tracking parameters fragment the cache; dropping all parameters can serve the wrong page (example 4).
• s-maxage wins at the edge. A long max-age with no s-maxage means the edge keeps the page as long as browsers do.
• Revalidation still costs a round trip. It saves bandwidth, not latency, unless you serve stale while refreshing (examples 3 and 5).
• A cache-bypass header is a public switch. Restrict who can send it.
• On Git Bash for Windows, Docker volume paths need MSYS_NO_PATHCONV=1, which start.sh and probe.sh set.

Practice

start.sh takes an optional config file and an optional Cache-Control value for the origin, so each task is a one-line restart.

1. Make the origin send Cache-Control: no-store and request /page three times. Predict the cache statuses and the origin build numbers first.

$ sh start.sh edge.conf "no-store"
lab up: origin + edge (edge.conf)
$ sh probe.sh /page /page /page
/page          0.305s  MISS         origin build #1
/page          0.306s  MISS         origin build #2
/page          0.303s  MISS         origin build #3

2. Raise the edge’s lifetime to 60 seconds (public, max-age=60, s-maxage=60) and request /page at 0, 5 and 10 seconds. How many requests reach the origin?

$ sh start.sh edge.conf "public, max-age=60, s-maxage=60"
lab up: origin + edge (edge.conf)
$ sh probe.sh /page +5 /page +5 /page stats
/page          0.308s  MISS         origin build #1
   (5s later)
/page          0.008s  HIT          origin build #1
   (5s later)
/page          0.003s  HIT          origin build #1
origin saw 1 request(s), built 1 full page(s)

3. Serve static files the way example 6 suggests: a year-long, immutable lifetime, with a version in the URL. Show that /app.v1.js is still a hit after 5 seconds, and that /app.v2.js needs no purge.

$ sh start.sh edge.conf "public, max-age=31536000, immutable"
lab up: origin + edge (edge.conf)
$ sh probe.sh /app.v1.js +5 /app.v1.js /app.v2.js /app.v2.js stats
/app.v1.js     0.304s  MISS         origin build #1
   (5s later)
/app.v1.js     0.002s  HIT          origin build #1
/app.v2.js     0.304s  MISS         origin build #2
/app.v2.js     0.002s  HIT          origin build #2
origin saw 2 request(s), built 2 full page(s)

4. origin.py adds a Vary header when you set VARY. Start the lab with VARY=Accept-Encoding, then request /page once with Accept-Encoding: gzip, once without, and both again.

$ VARY=Accept-Encoding sh start.sh
lab up: origin + edge (edge.conf)
$ HDR='Accept-Encoding: gzip' sh probe.sh /page
/page          0.305s  MISS         origin build #1
$ sh probe.sh /page
/page          0.307s  MISS         origin build #2
$ HDR='Accept-Encoding: gzip' sh probe.sh /page
/page          0.003s  HIT          origin build #1
$ sh probe.sh /page stats
/page          0.005s  HIT          origin build #2
origin saw 2 request(s), built 2 full page(s)