Horizontal Scaling

The Definition

 HORIZONTAL scaling (scale OUT): more nodes running the service
 
 [ LB ] ──► app-1 ─┐
        ──► app-2 ─┼── shared stateless tier, N identical workers
        ──► app-3 ─┘
 
 capacity = nodes × per-node capacity (ideally linearly)
 10x load? add ~10x nodes. no code change IF services are stateless

Why It Won

Property	Consequence
No ceiling	Add nodes as long as demand grows
Linear-ish cost	Commodity instances; price per unit work flat
Fault tolerance	Node death = capacity dip, not outage
Rolling everything	Deploys, upgrades, rollouts across many nodes
Elasticity	Autoscaling matches supply to demand hourly

Vertical’s ceilings — physical, economic, failure — all dissolve. This is why “web scale” is synonymous with horizontal.

The Price: Statelessness

Horizontal only works when any node can serve any request:

 BLOCKS horizontal scaling:
 - in-memory sessions      → move to Redis/JWT
 - local file uploads      → move to object storage (S3)
 - singleton schedulers    → leader election or managed jobs
 - sticky routing needs    → usually a statefulness smell
 
 rule: STATE lives in dedicated stores; compute tiers stay
 disposable. node = cattle, not pet.

The Limits of Linear Scaling

Real systems hit coordination walls:

 - shared DB becomes bottleneck → app scales, database doesn't
   (the classic wall: solve with replicas/caching/sharding)
 - fan-out amplification: each request touches more services
 - coordination overhead: consensus/locks grow with cluster size
 - hot data: one partition serves everyone regardless of nodes
 
 scaling the APP is easy; scaling the STATE is the actual problem
 most interview designs are really about

Load Balancing Is the Enabler

 requirements for effective horizontal pools:
 - health checks evict dead nodes automatically
 - distribution strategy fits traffic (round-robin default,
   least-connections for uneven request costs)
 - connection draining on deploys
 - autoscaler signals (CPU, RPS, queue depth) tuned to reality
 
 without good LB + health checks, extra nodes add noise,
 not capacity

Combining With Vertical

 they're complements, not rivals:

 per-node sizing (vertical choice) × node count (horizontal)
 
 undersized nodes → coordination/fan-out overhead dominates
 oversized nodes   → worse bin-packing, bigger blast radius each
 sweet spot: medium nodes, many of them — cloud-native default

Interview Framing

“Handle 100x traffic” answers start horizontal: stateless tier behind an LB with autoscaling, then immediately name what DOESN’T scale that way — the database — and pivot to replicas/caches/shards. The phrase “state out of the app tier” is the key signal; designs that keep session memory on nodes fail the follow-up instantly.