Caching
Store expensive work closer. Compare a direct database path with a cached path — and watch hit rate, latency, database load, and stale-data risk move as you change traffic.
Step 01 · Concept snapshot
Store expensive work closer
See how a cache changes latency, database load, and system behavior.
Without cache
“Repeat expensive work.”
Every request performs the work again — the database handles everything.
With cache
“Reuse previous work.”
Repeated results are reused — only misses reach the database.
Pick a scenario
What changed
Catalog reads repeat constantly and rarely change.
Why it matters
Strong cache candidate — high hit rate, low staleness.
Step 03 · Main interaction
Same traffic, three views
Switch between a direct DB path, a cached path, and a side-by-side compare.
Step 02 · Simulate repeated requests
Stress the system
Watch cache hits, misses, and database pressure react to every change.
Cache size
Cache health
Live architecture
Reuse previous work — repeated reads are served from cache.
88% of reads return from cache.
Repeated reads are being served.
12% of reads reach the DB.
Long TTL — data can drift.
Steps 05 / 08 / 09 · Cause and effect
What just happened?
What changed
Repeated results are served straight from cache.
Why it matters
Reads get fast and the database is shielded from repeated work.
Where's the bottleneck
None — the cache is absorbing repeated reads.
Long TTL improves performance but increases stale-data risk.
Step 06 · TTL & stale data
How long should data stay cached?
Update the database and watch the cache lag behind until its TTL expires.
$25
source of truth
$25
fresh
What changed
Cache and database agree.
Why it matters
Caching introduces freshness problems — readers can see old data until the TTL expires.
Long TTL improves performance but increases stale-data risk.
Step 07 · Eviction
What happens when the cache is full?
Limited memory forces a choice about what to drop.
Eviction strategy
LRU: Removes the least recently used item. Limited memory forces tradeoffs about what to keep.
Step 10 · Challenges
Match the fix
Adjust the controls above. A card turns green when your configuration solves it.
Users repeatedly load same products
Cache
Data changes constantly
Be careful — short TTL
Database overloaded
Raise cache effectiveness
Cache full
Eviction required
Cache server dies
Fallback to DB
Summary
- Without a cache, every request repeats expensive work and the database absorbs full load.
- With a cache, repeated reads are reused — latency drops and the database is shielded.
- Hit rate depends on repetition and cache size; misses still pay full database latency.
- Caching adds TTL, staleness, eviction, and failure handling — it must not be a single point of failure.
Why this exists
Caching trades freshness complexity for latency and throughput gains. By serving repeated reads from faster storage, it removes pressure from databases and external services. The hard part is not reading from cache. The hard part is invalidation and consistency.
Hits save expensive work
A good cache removes repeated database queries, remote calls, or expensive computation from the hot path.
Misses define the fallback path
A cache is never the system of record. You still need a correct miss path, fill strategy, and expiration model.
Freshness is the tax
The reason caching is hard is invalidation. Every performance win comes with a consistency question.
Key takeaways
- Cache hit rate is one of the most powerful leverage points in system design.
- Caches reduce latency and backend QPS, but add consistency complexity.
- The miss path must remain correct even if the cache is empty or down.
- Eviction, TTLs, and invalidation are design choices, not implementation details.