Must Read First

System Design Fundamentals

// Latency Numbers Every Engineer Must Know

These are rough orders of magnitude. Memorize the ratios, not exact numbers.

Operation	Latency	Ratio vs RAM	Implication
L1 cache reference	0.5 ns	1×	Near instantaneous
L2 cache reference	7 ns	14×	Very fast
RAM read	100 ns	1×	Baseline
SSD random read	150 µs	1,500×	1,000× slower than RAM
Network same datacenter	500 µs	5,000×	Network is expensive
HDD seek	10 ms	100,000×	Avoid disk seeks
Network cross-region (US→EU)	150 ms	1,500,000×	Huge — use CDN
Packet US→EU→US roundtrip	300 ms	—	Perceptible to users

Key Ratios: Cache is 100–1000× faster than DB. In-memory is 100× faster than SSD. SSD is 100× faster than HDD. Network within DC ≈ SSD speed. Cross-region ≈ HDD speed.

// Availability SLAs

Availability	Downtime/Year	Downtime/Month	Nines
90%	36.5 days	72 hours	One nine
99%	3.65 days	7.2 hours	Two nines
99.9%	8.7 hours	43.8 min	Three nines (most SaaS)
99.99%	52.6 min	4.4 min	Four nines (enterprise)
99.999%	5.26 min	26 sec	Five nines (telco, payments)

Every additional "9" requires roughly 10× more effort to achieve. Going 99.9% → 99.99% requires redundant components, automated failover, chaos engineering, and careful deployment pipelines.

// Back-of-Envelope Estimation

Powers of 2

2^10	1K	~1 thousand
2^20	1M	~1 million
2^30	1B	~1 billion
2^40	1T	~1 trillion

Time Conversions

1 day	86,400 sec	≈ 10⁵ sec
1 month	2.6M sec	≈ 2.6×10⁶
1 year	31.5M sec	≈ 3×10⁷
1 decade	315M sec	≈ 3×10⁸

Typical Object Sizes

ASCII char 1 byte

Integer 4 bytes

Long / Double 8 bytes

UUID 16 bytes

Tweet text 300 bytes

URL 100 bytes

User record (basic) ~1 KB

Photo (compressed) ~200 KB

Video (1 min, 1080p) ~100 MB

Estimation Template

QPS = DAU × actions_per_day / 86,400
Storage/year = QPS_write × avg_size × 86,400 × 365
Bandwidth = QPS × avg_request_size
Cache memory = QPS_read × avg_response_size × cache_window_seconds

// CAP Theorem + PACELC

Consistency

Every read receives the most recent write or an error.

Availability

Every request receives a (non-error) response — but it might not be the latest.

Partition Tolerance

System continues operating even when network messages are lost/delayed.

CAP Rule: You can only guarantee 2 of the 3. In practice, partition tolerance is non-negotiable in distributed systems. So the real choice is: CP (sacrifice availability) or AP (sacrifice consistency).

System	Type	Trade-off
Zookeeper, HBase	CP	Unavailable during network partition to ensure consistency
Cassandra, DynamoDB, CouchDB	AP	May return stale data to remain available during partition
MySQL (single node)	CA	Not distributed; partitions don't apply
Spanner, CockroachDB	CP (with tunable consistency)	External consistency via TrueTime
Redis (cluster)	AP	Primary may serve stale reads; async replication

// Consistency Models (Weakest → Strongest)

Eventual Consistency

If no new updates, all replicas eventually converge. May read stale data. Use when: social media likes, shopping cart, DNS.

Monotonic Read Consistency

A user will not read older data after reading newer data. Use when: timeline feeds, user profiles.

Read Your Writes

After a write, the same user will always see that write. Use when: user settings, profile updates.

Causal Consistency

Operations that are causally related are seen in the same order by all. Use when: comment replies, message threads.

Strong Consistency (Linearizability)

All operations appear instantaneous and in real-time order. Use when: financial transactions, inventory, distributed locks.

// Database Types — When to Use What

Type	Example	When to Use	Trade-offs
Relational (SQL)	MySQL, PostgreSQL	Complex queries, transactions, strict schema, joins	Harder to scale horizontally; rigid schema
Wide-Column	Cassandra, HBase	High write throughput, time-series, IoT, log data	No joins; eventual consistency; complex data modeling
Document	MongoDB, DynamoDB	Flexible schema, JSON-like docs, rapid iteration	Limited joins; document size limits
Key-Value	Redis, DynamoDB	Sessions, caches, leaderboards, counters	No complex queries; all in memory for Redis
Graph	Neo4j, Neptune	Social graphs, recommendation engines, fraud detection	Not suited for non-graph queries
Time-Series	InfluxDB, TimescaleDB	Metrics, monitoring, IoT sensors	Not general purpose; optimized for time-based queries
Search Engine	Elasticsearch	Full-text search, log analysis, autocomplete	Eventually consistent; not a primary DB

// Caching Strategies — Trade-offs

Cache-Aside (Lazy Loading)

App reads cache → miss → read DB → populate cache → return

✓ Pros

· Only caches what's actually read

· Cache failure is non-fatal

· Works with most DBs

✗ Cons

· Cache miss = 3 operations (extra latency)

· Data can be stale

· Thundering herd on cold start

Use when: General purpose reads. Most common pattern.

Write-Through

App writes → write to cache AND DB synchronously → confirm

✓ Pros

· Cache always consistent with DB

· No stale reads after write

✗ Cons

· Write latency = cache + DB latency

· Cache filled with unread data

Use when: When fresh reads are critical (financial, inventory).

Write-Back (Write-Behind)

App writes → write to cache → async flush to DB

✓ Pros

· Very fast writes

· DB load reduced

✗ Cons

· Data loss risk if cache fails before flush

· Complex implementation

Use when: High write throughput, acceptable data loss risk (analytics, counters).

Read-Through

App reads cache → miss → CACHE reads DB → returns to app

✓ Pros

· App doesn't need to know about DB

· Simpler app code

✗ Cons

· Cache must support the pattern

· Still slow on first read

Use when: CDNs, Varnish, managed cache services.

// Load Balancing Algorithms

Algorithm	How It Works	Best For	Downside
Round Robin	Request 1→server1, 2→server2, 3→server3, repeat	Identical servers, equal request size	Ignores server load and request weight
Weighted Round Robin	Server gets requests proportional to its weight/capacity	Servers with different capacities	Doesn't account for current load
Least Connections	Route to server with fewest active connections	Long-lived connections (WebSocket, DB)	Requires tracking connection counts
IP Hash	hash(client_IP) % server_count	Session affinity / sticky sessions	Uneven distribution if popular IPs
Consistent Hashing	Virtual ring; requests map to nearest server node	Distributed caches, horizontal scaling	Hotspots if servers are unevenly distributed
Resource Based	Probe CPU/memory; route to least-loaded	Variable-cost operations	Overhead of health checks

// Message Queue Comparison

System	Throughput	Ordering	Retention	Use Case
Kafka	10M+ msg/sec	Per partition	Days–weeks (configurable)	Event streaming, log aggregation, CDC
RabbitMQ	~50K msg/sec	FIFO per queue	Until acknowledged	Task queues, RPC, fan-out
AWS SQS	~3K req/sec std	Best-effort	Up to 14 days	Decoupling microservices
AWS SQS FIFO	~3K msg/sec	Strict FIFO	Up to 14 days	Order-critical workflows
Redis Streams	~1M msg/sec	Per stream ID	Configurable	Real-time analytics, leaderboards

// Key Rules of Thumb

80/20 Rule

20% of content serves 80% of reads → cache aggressively. Cache hit ratio target: >95%.

Read-Write Ratio

Twitter: 100:1 read/write. Uber: ~1:1. Wikipedia: ~10:1. Design for your ratio.

Replication Factor

Always replicate data 3× for fault tolerance (1 write, 2 replicas minimum).

Hot Shard Problem

Consistent hashing + virtual nodes (150-200 per server) evenly distributes load.

DB Connection Pool

Rule of thumb: 2× vCPUs per connection pool size for most OLTP workloads.

CDN for Assets

Static assets should NEVER be served from origin. CDN reduces latency by 10-100×.

Circuit Breaker

After 5 failures in 10 sec, open circuit for 30 sec. Prevents cascade failures.

Timeouts + Retries

Always set timeouts. Retry with exponential backoff + jitter. Max 3 retries.

Index Everything Queried

Every WHERE clause, ORDER BY, and JOIN condition should have an index.

Sharding Key Rule

Good shard key = high cardinality + evenly distributed + immutable after creation.

← Overview URL Shortener →