Online vs Offline Processing

Two Worlds, One System

Every substantial system secretly contains two:

 ONLINE (interactive)               OFFLINE (batch/stream)
 user is waiting                    nobody is waiting
 SLA: p99 milliseconds–seconds      SLA: minutes–hours completion
 sized for peak concurrency         sized for total work / window
 failure = visible outage           failure = delayed data
 examples: match a ride,            examples: nightly invoices,
 charge a card, render feed         ML training, analytics ETL,
                                    report generation

The architecture’s shape follows from which world each task belongs to — misfiled tasks are the root of most “why is checkout slow at 2am?” mysteries.

The Classification Test

One question per task:

 "must the HTTP response contain this result?"
 
 YES → online path. budget it into request latency.
 NO  → offline path. queue it; schedule it; batch it.
 
 RideShare examples:
 fare estimate shown pre-request     ONLINE (the response IS the answer)
 receipt email                       OFFLINE
 driver earnings summary             OFFLINE (page refresh tolerates lag)
 fraud check before capture          ONLINE (must gate payment)
 monthly fraud model retraining      OFFLINE

Precomputation: Moving Work Across the Line

The highest-leverage optimization in system design is reclassifying online work as offline:

 NAIVE FEED (all online)              PRECOMPUTED FEED (hybrid)
 on every feed view:                  offline fan-out writes each
   fetch my posts + all follows'      follower's feed list on post;
   posts; rank; merge                 online: read precomputed list
 
 read latency: 500ms+                 read latency: ~20ms
 write cost: trivial                  write amplification: celebrity
                                      problem (10M followers = 10M writes/post)

 hybrid reality: precompute for normal users, pull-based merge for
 celebrities — the fan-out trade gets its own lesson later in the course

Same pattern everywhere: search indexes, leaderboards, recommendation matrices — all are offline computation buying online speed.

The Failure Semantics Differ

Design responses differ by world:

Concern	Online	Offline
Dependency down	Degrade gracefully (cache, fallback)	Retry for hours; alert on lag
Overload	Shed load, admit control	Slow down; backlog absorbs
Correctness bug	Immediate user harm; rollback fast	Bad data flows downstream; needs backfill
Success metric	Latency percentiles	Job completion time, freshness lag

Backfills deserve respect — offline bugs corrupt silently and repair requires replay machinery (idempotent, resumable pipelines).

The Boundary Component

Queues sit exactly on this line and enforce it:

 ONLINE side                QUEUE            OFFLINE side
 request handlers  ──enqueue── [ ] [ ] [ ] ──dequeue── workers
 (never block on                  buffering,
  worker fate)                    smoothing spikes
 
 the queue converts load spikes from outages into delayed results —
 usually the correct trade when both sides exist

Interview Framing

Strong designs announce the split explicitly: “synchronous: match, authorize, record trip; asynchronous: receipts, ratings aggregates, analytics.” When interviewers ask “where does X happen?”, answering with the classification test (“does the rider wait for it? no — queue”) scores precisely because it demonstrates the line as a decision procedure, not decoration.