Designing a Caching System

This guide walks through designing a robust caching layer for a backend that serves user profiles. The user journey is straightforward: a client requests a profile (for example, Alan1). The backend first consults the cache; on a hit it returns cached data, on a miss it reads the database, populates the cache, and returns the result. We’ll build that flow step-by-step and show where it breaks and how to harden it. Version one — naive approach When a request asks for a user profile (e.g., Alan1), the application follows the cache-aside pattern:

Check cache.
If present (cache hit) → return cached value.
If absent (cache miss) → read from DB, write result to cache, and return to client.

This is simple and fast for read-heavy workloads but has correctness and operational pitfalls.

The image depicts a flow diagram showing a "naive" caching strategy with Redis, where data is served from the cache to users, bypassing the database.

Cache staleness: a correctness problem A major correctness issue with the naive approach is staleness. If Alan1 updates their username to Alan4 in the database, the DB contains the fresh value but the cache still holds the old Alan1. Unless the application explicitly updates or evicts the cache entry during the write, clients will continue to see stale data.

The image illustrates a data flow issue where a backend app retrieves incorrect data from Redis while the correct data exists in the database, leading to a stale state.

Version two — TTL and invalidation Common mitigations:

TTL (time-to-live): add expiration to cache entries so stale values are eventually evicted. For example, set a 5-minute TTL per profile key.
Active invalidation: when the database is updated, explicitly evict (or update) the corresponding cache entry.
Ordering: for the cache-aside pattern, a safe approach is to commit the DB write and then invalidate the cache (or populate the cache after the DB transaction commits). Other variants include write-through and write-back caches, each with different trade-offs.

Example cache-aside pseudocode:

// Read path
value = cache.get(key)
if value == nil:
    value = db.read(key)
    cache.set(key, value, ttl)
return value

// Write path (safe ordering)
db.write(key, newValue)
cache.del(key)  // or cache.set(key, newValue)

Notes on alternative strategies:

Write-through: writes go to cache first and synchronously to DB; simplifies reads but increases write latency.
Write-back: writes are buffered in cache and flushed to DB later; can improve write throughput but risks data loss on failures.

Version two helps, but fixed TTLs alone (e.g., exactly five minutes) can still expose users to stale data and can create synchronized expiration spikes. Combine TTL + invalidation for better freshness. Version three — cache miss storm (cache stampede) When a very hot key receives heavy traffic, a single expiration or eviction can cause a large number of clients to miss the cache simultaneously. This “stampede” floods the database with read requests and can overwhelm it. Mitigations to prevent cache stampedes:

Single-flight / Request coalescing

Ensure that when multiple concurrent requests miss the same key, only one request queries the database and populates the cache; other requests wait and then read the cached value.
In single-process systems, in-memory synchronization (e.g., mutexes) is sufficient. In distributed systems, use short-lived per-key locks in Redis, a coordination service, or a distributed single-flight implementation.

Use single-flight/coalescing so concurrent cache misses for the same key result in a single database load.

The image is a diagram illustrating a system architecture using Redis as a cache between a backend application and a database, showing how requests are managed to prevent cache misses.

Simple single-flight pseudocode (conceptual):

if cache.get(key) == nil:
    lock = acquire_distributed_lock(key)
    if lock.acquired:
        value = db.read(key)
        cache.set(key, value, ttl)
        release(lock)
    else:
        wait_for_cache_fill_or_lock_release()
return cache.get(key)

TTL jitter

Add a small random offset to TTLs so keys do not all expire at the exact same time. This smooths expiration and reduces synchronized load spikes.

The image is a diagram showing a Redis implementation with various keys and their time-to-live (TTL) values set to 5:00. It illustrates the concept of adding random seconds to each TTL value.

Replication for extremely hot keys

When a single key’s read volume exceeds what one cache node can serve, replicate that object across multiple cache nodes (read replicas) so multiple nodes can serve reads for the same key.
Sharding distributes keys across nodes but does not split one key; replication or read-multiplexing increases aggregate read capacity for extremely popular objects.

Summary of techniques

Technique	What it does	Pros	Cons	When to use
TTL	Evicts entries after time	Simple, reduces stale window	Fixed TTLs can be too long or synchronized	Use when eventual freshness is acceptable
Active invalidation	Evict/update cache on DB writes	Keeps cache fresh immediately	Requires correct ordering and extra complexity	Use for high-freshness data
Single-flight	Coalesce concurrent cache misses	Prevents DB overload during misses	Requires cross-process coordination in distributed systems	Use for hot keys with bursty traffic
TTL jitter	Randomize expirations	Smooths load spikes	Adds complexity to TTL management	Use for many keys with similar TTLs
Replication	Copy hot keys to many cache nodes	Increases read throughput	More memory and replication overhead	Use for extremely popular objects

Final guidance and warnings

Combine techniques: TTL + invalidation + single-flight + jitter + replication as appropriate for your workload.
Measure and tune TTLs, lock durations, and replication strategies based on traffic patterns, read/write ratios, and acceptable staleness.
Test failure scenarios (network partitions, cache evictions, DB latency spikes) to validate that cascading failure is contained.

Be careful: caching introduces complexity — staleness, invalidation, race conditions, and potential for cache stampedes. Combine TTL, invalidation, single-flight, TTL jitter, and replication as appropriate to balance freshness, latency, and system resilience.

Links and references

Cache-aside pattern: https://learn.microsoft.com/en-us/azure/architecture/patterns/cache-aside
Singleflight (Go): https://pkg.go.dev/golang.org/x/sync/singleflight
Redis docs: https://redis.io/
Write-through vs write-back: https://en.wikipedia.org/wiki/Write-through_cache and https://en.wikipedia.org/wiki/Write-back_cache
TTL / Time to live: https://en.wikipedia.org/wiki/Time_to_live_(networking)
Jitter and backoff guidance: https://aws.amazon.com/blogs/architecture/exponential-backoff-and-jitter/

Docker

Kubernetes

CI/CD & DevOps

Security

Networking

Advanced Concepts

Interview Setup

System Design

Designing a Caching System

Watch Video