Deep Dive

Distributed Cache (Redis Deep Dive)

Redis vs Memcached

Feature	Redis	Memcached
Data Structures	Strings, Hash, List, Set, Sorted Set, Bitmap, HLL, Streams	Simple key-value (strings only)
Persistence	RDB snapshots + AOF log (configurable)	None — memory only
Replication	Primary-Replica + Sentinel/Cluster	Third-party only
Pub/Sub	Yes (built-in)	No
Transactions	MULTI/EXEC (optimistic locking)	No
Lua Scripting	Yes (atomic scripts)	No
Memory efficiency	Slightly higher overhead	More memory efficient for pure caching
Threading	Single-threaded (I/O threaded in v6+)	Multi-threaded
Max memory	Configurable + eviction policies	Configurable
Use when	Feature-rich caching, Pub/Sub, sessions, leaderboards	Pure high-throughput string caching only

Rule of thumb: Use Redis unless you specifically need multi-threaded string caching at extremely high throughput and have no need for persistence, pub/sub, or complex data structures.

Eviction Policies (Memory Full)

Policy	Evicts	Best For
noeviction	Returns error on new writes	When you cannot tolerate data loss (sessions, auth tokens)
allkeys-lru ✓	Least recently used from ALL keys	General caching — most common choice
volatile-lru	LRU from keys WITH TTL set	Mix of persistent and cached data
allkeys-lfu	Least frequently used from ALL keys	When access patterns are predictable and stable
volatile-lfu	LFU from keys WITH TTL	Frequency-based eviction for TTL'd items
allkeys-random	Random key from all	When key access is truly random
volatile-ttl	Shortest TTL first	When you want soonest-expiring items evicted

Cache Invalidation Strategies

"There are only two hard things in Computer Science: cache invalidation and naming things." — Phil Karlton

TTL-Based Expiry

Set TTL on every cache entry. Simple, works everywhere.

✓ Zero code complexity

✗ Stale data during TTL window. Thundering herd on expiry.

→ Non-critical data, config, product catalogs.

Event-Driven Invalidation

DB write → publish event to Kafka → cache consumer deletes/updates key.

✓ Near-real-time consistency

✗ Complexity. Event ordering issues. Consumer lag.

→ Critical data: user profiles, prices, inventory.

Write-Through (update on write)

When DB is updated, update cache simultaneously in the same transaction.

✓ Always consistent

✗ Write latency = DB + cache latency. Wasted cache if data never read.

→ Frequently read, frequently written data.

Cache-Busting (versioned keys)

Change cache key when data changes: user:123:v1 → user:123:v2

✓ No invalidation needed

✗ Old keys accumulate unless manually cleaned up.

→ Static assets, deployment artifacts.

The Thundering Herd Problem

Scenario: A popular cache key expires. 10,000 requests simultaneously hit the DB to repopulate it. DB is overwhelmed. This is called thundering herd or cache stampede.

Mutex Lock (Cache Lock)

First request acquires a Redis lock and queries DB. Other requests wait. On success, all use the cached value.

# Pseudocode
if cache.get(key): return cache.get(key)
lock = redis.setnx("lock:" + key, 1, ex=5)  # 5 sec timeout
if lock:
    value = db.query(...)
    cache.set(key, value, ex=300)
    redis.delete("lock:" + key)
else:
    time.sleep(0.05)  # brief wait
    return cache.get(key) or db.query(...)

Probabilistic Early Expiry (PER)

Before TTL expires, randomly decide to refresh. Earlier refresh probability increases as TTL approaches 0. Prevents simultaneous expiry.

import math, random, time

def get_with_per(cache, key, ttl, beta=1.0):
    value, exp_time = cache.get_with_expiry(key)
    # PER: refresh early with increasing probability
    if -beta * math.log(random.random()) >= exp_time - time.time():
        value = db.query(...)
        cache.set(key, value, ttl + random.uniform(0, ttl * 0.1))
    return value

TTL Jitter

Add random offset to TTL so keys don't expire at the same time.

import random

BASE_TTL = 3600  # 1 hour
# Add ±10% jitter
ttl = BASE_TTL + random.randint(-360, 360)
cache.set(key, value, ex=ttl)

Redis at Scale

Redis Sentinel

Automatic failover. Monitors primary and replicas. Promotes replica to primary on failure. Used for high availability with a single shard.

→ Single shard HA. Up to ~100GB dataset.

Redis Cluster

Automatic sharding across 16,384 hash slots. Horizontal scaling. Each primary handles a subset of key space with replicas for HA.

→ Dataset > 100GB. Need to scale beyond single node.

Consistent Hashing

Map cache keys to nodes using a virtual ring. Adding/removing nodes only migrates ~1/N of keys instead of all keys.

→ Need to minimize cache misses during node addition/removal.

Read Replicas

Route read-heavy operations to replicas. Primary handles only writes. Replication is asynchronous — slight staleness risk.

→ Read:write ratio > 5:1. Replica lag acceptable (< 100ms usually).

Cache Performance Ratios

Target Cache Hit Rate

> 95%

If below 80%, either cache is too small or TTLs too short

Cache vs DB Speed

100–1000×

RAM at 100ns vs SSD at 100µs = 1000× faster reads

80/20 Rule

Cache 20%

20% of content = 80% of all reads. Cache those keys.

Redis Throughput

~1M ops/sec

Per node for simple GET/SET operations

TTL Sweet Spot

5 min–1 hour

Balance: freshness vs DB load. Start at 5min, tune up.

Cache Size Rule

10% of DB

Caching 10% of dataset often yields >90% hit rate

← Rate Limiter Notification System →