Caching
Explore how high-speed data storage improves performance, and the challenges of invalidation and eviction.
Cache Config
Read Key
What's happening?
When you click a key (A–H), a read request flows from the client down to the cache layer. A HIT returns instantly from Redis in ~10ms. A MISS falls through to PostgreSQL (~150ms) and populates the cache. TTL evicts entries automatically; the LRU policy evicts the least-recently used slot when the cache is full.
Why it matters?
Caching leverages temporal locality — data accessed once is likely needed again. A 90%+ hit rate can cut database load by 10×. The tradeoff is consistency: stale reads serve outdated data after DB writes. Cache invalidation (write-through or TTL expiry) is famously one of the hardest problems in distributed systems.
Why this exists
Caching trades freshness complexity for latency and throughput gains. By serving repeated reads from faster storage, it removes pressure from databases and external services. The hard part is not reading from cache. The hard part is invalidation and consistency.
Hits save expensive work
A good cache removes repeated database queries, remote calls, or expensive computation from the hot path.
Misses define the fallback path
A cache is never the system of record. You still need a correct miss path, fill strategy, and expiration model.
Freshness is the tax
The reason caching is hard is invalidation. Every performance win comes with a consistency question.
Key takeaways
- Cache hit rate is one of the most powerful leverage points in system design.
- Caches reduce latency and backend QPS, but add consistency complexity.
- The miss path must remain correct even if the cache is empty or down.
- Eviction, TTLs, and invalidation are design choices, not implementation details.