Rate Limits

Why Limits Exist

Every API is shared infrastructure. Without limits, one buggy customer loop (or attacker) consumes capacity meant for everyone:

 unbounded:  tenant X deploys retry loop → 50k rps → DB saturates
             → ALL tenants degraded → your outage, their bug
 
 rate limits define FAIR SHARE and CAP BLAST RADIUS:
   X capped at 1000 rps → their excess 429s; everyone else fine

Limit Dimensions

 per API key / tenant     fairness between customers
 per user                 single-account abuse
 per IP                   anonymous/edge protection
 per endpoint class       expensive search ≠ cheap status ping
 global budget            protect a fragile backend absolutely

 layered in order: IP at edge (cheap), key+user at gateway,
 endpoint-specific near the service itself

Algorithms

Algorithm	Mechanism	Weakness
Token bucket	Refill rate r, burst capacity b	Needs state per key
Leaky bucket	Constant outflow queue	Smooths but queues latency
Fixed window	Count per minute	Burst ×2 across boundaries
Sliding window log	Exact timestamps	Memory heavy
Sliding window counter	Blend of fixed + weighted	Approximate

 token bucket (industry default):
   bucket holds 100 tokens, refills 10/sec
   each request spends 1; empty bucket → 429
   allows BURSTS (empty the 100 fast) while averaging to refill rate
 
 fixed window trap:
   limit 100/min; 100 requests at 00:59 + 100 at 01:01
   = 200 requests in 2 seconds, both windows individually "legal"

Distributed Enforcement

Limits must hold across N gateway instances:

 options:
 - centralized counter (Redis INCR with TTL) — simple, adds hop
 - local buckets synced periodically — approximate, no hop
 - consistent-hash keys to one owner instance — exact, sharded

 pick per strictness: billing-relevant limits need accuracy;
 abuse-mitigation tolerates approximation

The Response Contract

Being limited must be actionable, not just refused:

 HTTP 429 Too Many Requests
 Retry-After: 12                        seconds until likely success
 X-RateLimit-Limit: 1000                the deal
 X-RateLimit-Remaining: 0               where you are
 X-RateLimit-Reset: 1724500000          when counters reset

 clients that honor Retry-After convert rejection into
 delayed success; undocumented limits create blind retries

Client-Side Etiquette

 read the headers BEFORE firing: remaining low? slow down.
 on 429: wait Retry-After, then resume — never tight-retry 429s
 (429 retry storms are self-inflicted DDoS against your own quota)
 
 proactive client-side throttling beats reactive server punishment
 for everyone involved

Interview Framing

“Prevent one user from melting the system” = rate limiting question. Scored shape: dimensions chosen by identity model, token bucket with named parameters, distributed-counter note, and the 429 + Retry-After contract. The bonus point is connecting limits to load shedding as defense-in-depth — limits are policy, shedding is emergency surgery.