Identify Single Points of Failure

The Audit Method

A single point of failure (SPOF) is any component whose death takes the system down. Finding them requires no tools — only the discipline of walking every box and asking one question:

 FOR EACH BOX IN THE DIAGRAM:
 
 "what happens if THIS dies right now?"
 
 answer classes:
   nothing user-visible      → redundant already ✓
   degraded but alive        → acceptable, note the mode
   outage                    → SPOF ✗ fix required
 
 then ask the follow-up everyone forgets:
 "what happens when it COMES BACK?"

RideShare Audit Walk

Component dies	Immediate effect	Verdict
One app server	LB health-checks it out; capacity -1/N	fine (N+1 sized)
The load balancer	Nothing reachable at all	SPOF → run pairs/managed multi-AZ
Postgres primary	Writes fail system-wide	SPOF → standby + automated failover
Redis cache	Misses fall to DB; latency spikes, DB must absorb storm	degraded — size DB for miss load
One AZ	Whatever lives only there dies with it	→ replicate across AZs
DNS provider	Total unreachability despite healthy fleet	→ secondary DNS provider

Three genuine SPOFs found in a four-box diagram — this audit is never academic.

Redundancy Patterns Per Failure Class

 STATELESS components          STATEFUL components
 ─────────────────────         ─────────────────────
 N+1 behind LB                 replication:
 any instance serves any       primary + standby (failover)
 request; death = capacity     quorum (N/2+1 agree)
 event, not availability       sharding + replicas per shard
 event                         (death = data event, not just capacity)

 COORDINATION components
 ─────────────────────
 LB pairs (VRRP) or cloud-managed multi-AZ
 DNS secondaries, multiple notification channels

Stateful redundancy is where the hard design lives — failover mechanics, split-brain prevention, data loss windows (RPO). Those get their own deep-dives later.

The Comeback Problem

Failures aren’t complete until recovery is clean:

 node returns after 30 minutes offline:
 - thundering herd: cache repopulation stampedes the DB
 - stale state: in-memory queues lost mid-flight
 - split brain: two nodes both believed primary during partition
 
 mitigations: warm-up rates on rejoin, fencing tokens/epochs,
 idempotent message processing, gradual traffic ramp

Production incidents routinely happen during recovery, not failure.

Blast Radius Thinking

When full redundancy is too expensive, shrink blast radius instead:

 cell-based isolation: users sharded into independent cells;
                       one cell's total loss affects only its users
                       ▼
 Cell 1 (DB+A+B)  Cell 2 (DB+A+B)  Cell 3 ...
 ── no shared fate across cells; failures are contained explosions ──

Trade-off: cross-cell global operations become harder. Availability bought with architectural constraint.

Interview Framing

“SPOF check” is one of the fastest seniority signals available: after drawing anything, spend sixty seconds walking boxes aloud (“LB is paired; primary has standby; cache loss degrades to sized-for misses”). Interviewers probe comebacks (“primary fails over — what do clients see?” — connection errors for seconds, then rerouting). A diagram with no stated death stories invites the hardest questions.