Caching is the highest-leverage performance optimization available to most applications. At 10,000 requests per second, an uncached database-backed API requires 10,000 database queries per second — a load that requires expensive infrastructure. A 95% cache hit rate reduces that to 500 queries per second, cutting database costs by 90% and reducing p99 latency from 200ms to 5ms for the cached majority.
The challenge is not adding a cache — it is designing a cache that stays consistent, invalidates correctly, and does not create the cache stampede, thundering herd, and stale read problems that plague naive implementations.
This guide covers a complete 5-layer caching architecture, Redis patterns with production code, invalidation strategy comparisons, consistency models, TTL design by content type, and stampede prevention.
---
5-Layer Caching Architecture
Each layer serves a different scope and has different tradeoffs on hit rate, consistency, and invalidation complexity.
Request Flow (fastest to slowest) Hit Rate Latency
──────────────────────────────────────────────── ───────── ───────
Layer 1: Browser Cache (Cache-Control headers) 40-70% 0ms
↓ miss
Layer 2: CDN Edge Cache (Cloudflare, Fastly) 50-80% 5-20ms
↓ miss
Layer 3: Reverse Proxy Cache (Nginx, Varnish) 30-60% 1-5ms
↓ miss
Layer 4: Application Cache (Redis, Memcached) 60-90% 0.5-2ms
↓ miss
Layer 5: Database Query Cache (pg result cache) 20-40% 1-10ms
↓ miss
Origin: Database (PostgreSQL, MySQL, MongoDB) 100% miss 10-200ms
Combined effective hit rate: with all 5 layers properly configured, 95-99% of requests never reach the database. The remaining 1-5% are cache misses that require fresh data.
---
Layer 1: Browser Cache with Cache-Control Headers
// Express.js — set cache headers per route type
const express = require('express');
const app = express();
// Static assets: cache for 1 year (content-hash in filename guarantees freshness)
app.use('/static', express.static('public', {
maxAge: '1y',
immutable: true, // Tell browser: never revalidate this URL
}));
// API responses: no browser caching (user-specific data)
app.use('/api/user', (req, res, next) => {
res.set('Cache-Control', 'private, no-store');
next();
});
// Public API data: short browser cache + CDN cache
app.get('/api/products', (req, res) => {
res.set({
'Cache-Control': 'public, max-age=60, s-maxage=300, stale-while-revalidate=600',
'Vary': 'Accept-Language', // Separate cache per language
'ETag': computeETag(products),
});
res.json(products);
});
---
Layer 2 and 3: CDN and Reverse Proxy Cache
<h1>nginx.conf — reverse proxy cache configuration</h1>
proxy_cache_path /var/cache/nginx
levels=1:2
keys_zone=api_cache:10m
max_size=1g
inactive=60m
use_temp_path=off;
server {
listen 80;
location /api/products {
proxy_cache api_cache;
proxy_cache_key "$scheme$host$request_uri";
proxy_cache_valid 200 5m; # Cache 200 responses for 5 minutes
proxy_cache_valid 404 1m; # Cache 404s for 1 minute
proxy_cache_use_stale error timeout updating; # Serve stale on upstream error
proxy_cache_lock on; # Prevent thundering herd (only 1 upstream request)
proxy_cache_lock_timeout 5s;
add_header X-Cache-Status $upstream_cache_status; # HIT/MISS/STALE for debugging
proxy_pass http://api-backend;
}
# Never cache authenticated routes
location /api/user {
proxy_cache off;
proxy_pass http://api-backend;
}
}
---
Layer 4: Redis Cache Patterns
Cache-Aside (Lazy Loading)
Cache-aside is the most common pattern. The application checks the cache first; on a miss, it loads from the database and populates the cache.
import redis.asyncio as redis
import json
import hashlib
class CacheAside:
def __init__(self, redis_client: redis.Redis, db):
self.redis = redis_client
self.db = db
async def get_product(self, product_id: int) -> dict:
cache_key = f"product:{product_id}"
# 1. Check cache
cached = await self.redis.get(cache_key)
if cached:
return json.loads(cached) # Cache HIT
# 2. Cache MISS — load from database
product = await self.db.fetch_one(
"SELECT * FROM products WHERE id = $1", product_id
)
if not product:
# Cache negative results to prevent DB hammering on 404s
await self.redis.setex(cache_key, 60, json.dumps(None))
return None
# 3. Populate cache
await self.redis.setex(
cache_key,
300, # TTL: 5 minutes
json.dumps(dict(product))
)
return dict(product)
async def invalidate_product(self, product_id: int):
"""Call this on every product update/delete."""
await self.redis.delete(f"product:{product_id}")
# Also invalidate list caches that may contain this product
async for key in self.redis.scan_iter("products:list:*"):
await self.redis.delete(key)
Write-Through Cache
Write-through keeps the cache always warm by writing to both cache and database on every update. Higher write latency; zero cold-start misses.
class WriteThrough:
async def update_product(self, product_id: int, data: dict) -> dict:
# 1. Write to database first (source of truth)
updated = await self.db.fetch_one(
"UPDATE products SET name=$1, price=$2 WHERE id=$3 RETURNING *",
data["name"], data["price"], product_id
)
# 2. Immediately update cache — no stale window
cache_key = f"product:{product_id}"
await self.redis.setex(
cache_key,
300,
json.dumps(dict(updated))
)
return dict(updated)
# Reads always hit cache (warm guaranteed)
async def get_product(self, product_id: int) -> dict:
cached = await self.redis.get(f"product:{product_id}")
return json.loads(cached) if cached else None
Write-Behind (Write-Back) Cache
Write-behind acknowledges the write immediately, queues the database update asynchronously. Highest write throughput; risk of data loss if Redis crashes before flush.
import asyncio
class WriteBehind:
WRITE_QUEUE_KEY = "write-behind:products"
FLUSH_INTERVAL_SECONDS = 5
async def update_product(self, product_id: int, data: dict):
# Write to cache immediately (synchronous — fast)
await self.redis.setex(f"product:{product_id}", 300, json.dumps(data))
# Queue DB write (asynchronous — decoupled)
await self.redis.lpush(self.WRITE_QUEUE_KEY, json.dumps({
"product_id": product_id,
"data": data,
"timestamp": time.time()
}))
async def flush_worker(self):
"""Background worker — runs every 5 seconds."""
while True:
await asyncio.sleep(self.FLUSH_INTERVAL_SECONDS)
batch = []
while True:
item = await self.redis.rpop(self.WRITE_QUEUE_KEY)
if not item: break
batch.append(json.loads(item))
if batch:
# Batch insert — much faster than individual updates
await self._batch_db_update(batch)
---
Cache Invalidation Strategies
| Strategy | Consistency | Complexity | Best For |
| TTL expiration | Eventual (stale up to TTL) | Low | Public content, tolerates seconds of staleness |
| Event-driven (pub/sub) | Strong | High | User-specific data, inventory, prices |
| Write-through | Strong | Medium | Read-heavy data updated frequently |
| Cache tagging | Strong | High | Complex related data (product + category + brand) |
| Versioned keys | Strong | Medium | Config data, feature flags |
| Polling (short TTL) | Near-real-time | Low | Dashboard metrics, aggregates |
Cache tagging with Redis Sets:
async def cache_product_with_tags(product: dict):
"""Tags allow invalidating all related caches at once."""
product_id = product["id"]
category_id = product["category_id"]
# Store the product
await redis.setex(f"product:{product_id}", 3600, json.dumps(product))
# Register this key under its category tag
await redis.sadd(f"tag:category:{category_id}", f"product:{product_id}")
await redis.expire(f"tag:category:{category_id}", 3600)
async def invalidate_category(category_id: int):
"""Delete all products in a category with one operation."""
tag_key = f"tag:category:{category_id}"
keys = await redis.smembers(tag_key)
if keys:
await redis.delete(*keys, tag_key)
---
Consistency Models
| Model | Reads See | When to Use | Example |
| Strong consistency | Always latest write | Financial data, inventory | Account balance, stock count |
| Eventual consistency | Latest write within TTL | Product listings, blog posts | 5-min stale product catalog |
| Read-your-writes | Own writes immediately | User profile updates | User sees their own changes |
| Monotonic reads | Never older than previous read | User activity feeds | Timeline never goes backward |
| Session consistency | Consistent within session | Shopping cart | Cart state coherent per user |
---
TTL Design by Content Type
| Content Type | Recommended TTL | Invalidation Trigger | Notes |
| User session | 24h (sliding) | Logout, password change | Extend on activity |
| User profile | 5 min | Profile update | Short TTL; changes are personal |
| Product catalog | 10 min | Admin update | Public data; staleness acceptable |
| Product inventory | 30 sec | Every order | Inventory accuracy is critical |
| Pricing | 1 min | Pricing rule change | Balance freshness vs DB load |
| Homepage featured | 15 min | CMS publish | Marketing content can lag slightly |
| Search results | 5 min | Product update | Re-index triggers invalidation |
| User permissions | 10 min | Role change | Security-sensitive; keep short |
| Feature flags | 30 sec | Flag update | Must propagate quickly |
| Analytics aggregates | 1 hour | Time-based refresh | Batch recalculation acceptable |
| Static assets | 1 year | Content hash change | Immutable URLs with hash suffix |
---
Cache Stampede Prevention
Cache stampede (thundering herd) occurs when a popular key expires and hundreds of concurrent requests all miss the cache simultaneously, flooding the database.
import asyncio
import random
class StampedeProtectedCache:
"""Prevents thundering herd using probabilistic early expiration."""
async def get_with_lock(self, key: str, fetch_fn, ttl: int):
"""
Mutex-based approach: only one process recomputes, others wait.
Best for: expensive computations (ML inference, complex aggregations).
"""
lock_key = f"lock:{key}"
# Try to get from cache
cached = await self.redis.get(key)
if cached:
return json.loads(cached)
# Acquire lock (SET NX EX = atomic set-if-not-exists with TTL)
lock_acquired = await self.redis.set(lock_key, "1", nx=True, ex=10)
if lock_acquired:
try:
# This process recomputes
value = await fetch_fn()
await self.redis.setex(key, ttl, json.dumps(value))
return value
finally:
await self.redis.delete(lock_key)
else:
# Other processes wait and retry
for _ in range(10):
await asyncio.sleep(0.1)
cached = await self.redis.get(key)
if cached:
return json.loads(cached)
# Fallback: go to database directly after timeout
return await fetch_fn()
async def get_with_jitter(self, key: str, fetch_fn, ttl: int):
"""
Probabilistic early expiration: refresh slightly before TTL expires.
Best for: high-traffic keys where lock overhead is unacceptable.
"""
data = await self.redis.get(key)
if data:
cached = json.loads(data)
remaining_ttl = await self.redis.ttl(key)
# Early refresh probability increases as TTL decreases
# At 10% of TTL remaining: 50% chance of refresh
early_refresh_probability = max(0, 1 - (remaining_ttl / (ttl * 0.1)))
if random.random() < early_refresh_probability:
asyncio.create_task(self._background_refresh(key, fetch_fn, ttl))
return cached
# Cache miss: normal fetch
value = await fetch_fn()
await self.redis.setex(key, ttl, json.dumps(value))
return value
async def _background_refresh(self, key: str, fetch_fn, ttl: int):
"""Refresh cache in background — caller gets stale data, not a delay."""
try:
value = await fetch_fn()
await self.redis.setex(key, ttl, json.dumps(value))
except Exception:
pass # Stale data continues serving; no crash
---
Monitoring Cache Hit Rates
<h1>Prometheus metrics for cache monitoring</h1>
from prometheus_client import Counter, Histogram, Gauge
cache_hits = Counter('cache_hits_total', 'Cache hits', ['layer', 'key_type'])
cache_misses = Counter('cache_misses_total', 'Cache misses', ['layer', 'key_type'])
cache_hit_rate = Gauge('cache_hit_rate', 'Rolling cache hit rate', ['layer'])
async def tracked_get(self, key: str, key_type: str = "generic") -> any:
value = await self.redis.get(key)
if value:
cache_hits.labels(layer="redis", key_type=key_type).inc()
else:
cache_misses.labels(layer="redis", key_type=key_type).inc()
return value
<h1>Alert when hit rate drops below 80%</h1>
<h1>(signals TTL too short, cold cache, or traffic pattern change)</h1>
Key cache metrics to alert on:
- Hit rate < 80% for any key type: TTL too short or cache not warmed
- Redis memory > 80% of maxmemory: risk of eviction; scale Redis or reduce TTLs
- Eviction rate > 0: keys being evicted before expiry; increase memory or reduce stored data size
- Lock wait time > 500ms: stampede protection too slow; switch to jitter approach
- Replication lag > 100ms: read replicas serving stale data
Arxentra's platform surfaces a cache hit ratio panel under platform features for managed Postgres, while Redis memory limits are set at provisioning time and sized to your workload. See the services documentation for Redis provisioning options including cluster mode for horizontal cache scaling. View pricing for managed Redis tiers optimized for different throughput requirements.
---
Caching Implementation Checklist
- Define cache keys with consistent naming convention: `{entity}:{id}:{variant}`
- Set TTLs by content type using the table above as baseline
- Implement cache-aside as default; switch to write-through only where consistency requires it
- Add stampede protection for any key with >100 concurrent readers
- Cache negative results (404s) with short TTL (60s) to prevent DB hammering
- Monitor hit rate per key type with Prometheus; alert below 80%
- Use Redis OBJECT ENCODING to verify data types — avoid storing JSON when a hash is more efficient
- Test cache invalidation paths in integration tests — the correctness bugs live here
- Verify maxmemory-policy is set to allkeys-lru or volatile-lru (never noeviction in production)
- Document TTL decisions in code comments — future you will forget why 5 minutes was chosen
A well-designed caching strategy is the difference between a database that handles 500 queries per second comfortably and one that needs 10x the compute to handle the same user load. Start with Layer 4 (Redis cache-aside), measure your hit rate, then add CDN caching for public content. Each layer you add multiplies your effective capacity without adding database cost.