Caching Strategies & Cache Invalidation

Most requests a backend serves are wastefully repetitive. The same product page gets rendered a thousand times a minute from data that changes once an hour. The same permission check runs on every request against a row that hasn't changed in days. A cache exploits this skew directly: it trades a small amount of memory for skipping an expensive recomputation or a round trip to the database, on data that is read far more often than it's written.

Done well, caching is the single highest-leverage change you can make to a slow system — it's often the difference between a database that's pinned at 100% CPU and one that's barely breathing. Done carelessly, it's the reason users see stale prices, stale permissions, or a profile picture that never updates. The mechanics of reading from a cache are simple; the discipline is almost entirely in deciding when to write to it and when to throw it away.

Cache-aside: the default for a reason

The pattern you reach for first, and the one that covers the large majority of real caching needs, is cache-aside (also called lazy loading). The application, not the cache, owns the logic: check the cache first; on a hit, return immediately; on a miss, query the source of truth, then write the result into the cache before returning it. The next request for the same key is a cache hit.

The appeal of cache-aside is that it's demand-driven — it only ever caches data that was actually requested, so you never waste memory pre-warming keys nobody asks for. It's also resilient to a cold or flushed cache: worst case, every request temporarily behaves like there was no cache at all, and the cache rebuilds itself organically as traffic comes in.

def get_user_profile(user_id: str) -> dict:
    cache_key = f"user:{user_id}"

    # 1. Check the cache first
    cached = cache.get(cache_key)
    if cached is not None:
        return cached

    # 2. Miss — fall back to the source of truth
    profile = db.query_one(
        "SELECT * FROM users WHERE id = %s", [user_id]
    )

    # 3. Populate the cache for next time, with a TTL
    # and a little jitter so peers don't all expire together
    ttl = 300 + random.randint(0, 30)
    cache.set(cache_key, profile, ttl_seconds=ttl)

    return profile

Notice the shape: the cache is consulted first, the database is only touched on a miss, and the write-back happens before the function returns. Almost every caching library and every "add Redis in front of this query" retrofit follows exactly this shape.

Write-through vs. write-behind

Cache-aside says nothing about what happens when the underlying data changes — that's a separate decision, and it comes down to two patterns with a real durability/latency trade-off between them.

Write-through writes to the cache and the store together, synchronously, as part of the same operation. The request doesn't complete until both are durable. This keeps the cache and the source of truth in lockstep — a read right after a write is guaranteed to see the new value — at the cost of every write paying the latency of two systems instead of one.

Write-behind (or write-back) writes to the cache immediately and returns, then persists to the store asynchronously — on a delay, in a batch, or via a background queue. Writes feel instant and the store can absorb bursts more gracefully, but there's a window where the "true" record only exists in the cache. If the cache node dies before the flush happens, that write is gone.

TTL: the simplest invalidation strategy that works

The hardest question in caching isn't how to populate a cache — it's how to know when a cached value is no longer true. The simplest possible answer is to stop asking: give every entry a time-to-live (TTL) and let it expire automatically, regardless of whether the underlying data actually changed.

TTL's entire trade-off lives on one dial. A short TTL means the cache is almost always fresh, but it also means more frequent misses and more load pushed back onto the database — you're barely caching at all. A long TTL means far fewer misses and much better database offload, but it widens the window during which a client can be served data that's quietly wrong. There's no TTL that's correct in the abstract; the right value depends entirely on how stale your callers can tolerate the data being, which is a product question as much as an engineering one.

The thundering herd problem

TTL expiry creates a second, sneakier problem once a key gets popular enough. Picture a single cache entry — say, the homepage's "featured products" list — being read thousands of times a second. The moment its TTL lapses, every one of those thousands of concurrent requests misses at the same instant, and all of them turn around and hit the database simultaneously to recompute the exact same value. This is the thundering herd (or cache stampede) problem, and it can take down a database that was otherwise handling load comfortably — the cache didn't fail, it just uncovered how much load it was actually absorbing.

Three mitigations are standard, and they compose well together:

• Locking / single-flight — when a key misses, the first request acquires a short-lived lock and is the only one allowed to query the database and repopulate the cache; every other concurrent request for that key waits on the lock (or briefly serves a slightly-stale fallback) instead of stampeding the database itself.
• Jittered TTLs — instead of setting every related key to expire in exactly 300 seconds, add a small random offset (300 ± 30 seconds, say) so keys that were populated together don't all expire in the same instant.
• Stale-while-revalidate — keep serving the expired value to callers while one background request refreshes it, rather than making every caller wait on (or trigger) a fresh recompute. Callers get a fast response and a value that's at most one TTL window old.

Pick the simplest thing that meets your staleness tolerance

There's an old joke that there are only two hard problems in computer science: cache invalidation, naming things, and off-by-one errors. It's a joke because it's true — invalidation is hard precisely because "is this value still correct" is a question about the rest of your system, not about the cache. Event- driven invalidation (publish a message when a row changes, evict the corresponding key) can get you closer to perfect freshness, but it adds real complexity: every writer needs to know what to invalidate, and a missed invalidation is a silent, hard-to-detect bug.

Before reaching for that complexity, ask what staleness your callers can actually tolerate. If "wrong for up to five minutes" is fine, a jittered TTL with cache-aside is simpler to reason about, simpler to debug, and fails safe — worst case, it just expires. Start there, measure whether it's good enough, and only build explicit invalidation when the data genuinely can't tolerate the delay.