Menu

Earn Premium with Referrals

Invite your friends and earn Premium rewards through our referral program.

See how it works and start inviting friends.

TCP Optimization at the Edge
HLD

TCP Optimization at the Edge

Splitting connections and tuning congestion control — how edges make slow networks feel fast.

Why Long Paths Are Slow

 TCP throughput is capped by physics + algorithms:

 throughput ≈ window / RTT

 transatlantic RTT ~90ms:
 - each loss costs a full RTT to discover, another to repair
 - congestion windows grow slowly over high-RTT paths
 - mobile networks add packet loss that TCP reads as congestion
   (it isn't — radio interference is)

 a user on 3G in Nairobi fetching from Virginia suffers
 EVERY one of these effects simultaneously.

Weapon 1: Connection Splitting

 break one long path into two optimized short ones:

 [user] ──short/lossy──► [edge POP] ──backbone──► [origin]

 each leg tuned independently:

 user↔edge leg (the hostile one):
   modern CC (BBR-style): models bandwidth, not just losses
   aggressive initial windows
   TLS 1.3 / QUIC: fewer handshake RTTs
   
 edge↔origin leg (the controllable one):
   long-lived pooled connections: handshakes amortized
   backbone routing: monitored, uncongested paths
   big healthy windows over fat pipes

 user experiences the SUM of two good legs,
 not the product of one bad one.

Weapon 2: Congestion Control Modernization

 classic loss-based CC (Reno/CUBIC):
   interprets ANY loss as congestion → backs off hard
   → mobile users throttled by radio noise forever

 BBR-class model-based CC:
   measures actual bottleneck bandwidth + RTT
   ignores non-congestion losses
   → 2-25x throughput gains on lossy/high-RTT paths reported

 CDNs run tuned kernels/QUIC stacks at edges;
 your origin rarely can — another reason to terminate there.

Weapon 3: Protocol Upgrades at Ingress

 HTTP/3 / QUIC at the edge specifically helps:

 - 0-RTT resumption: repeat visits skip handshake entirely
 - no TCP slow start: QUIC manages its own windows smarter  
 - connection migration: wifi→5G handoff keeps transfers alive
   (connection IDs survive IP changes!)
 - per-stream loss recovery: one lost packet doesn't stall
   every stream (TCP head-of-line blocking eliminated)

 browsers speak it when edges advertise it;
 origins never need to know.

The Composite Numbers

 measured improvements for far/mobile users (typical):

 TTFB:            40-60% faster
 video startup:   30-50% faster (startup = several RTTs of setup)
 throughput on
 2% loss paths:   2-4x with BBR-class CC vs CUBIC
 rebuffering:     drops proportionally to throughput gains

 none of this requires application changes —
 it's infrastructure you inherit by terminating at edges.

Interview Framing

“Mobile users in emerging markets report slowness despite caching” is the planted question. Scored diagnosis: high-RTT + lossy paths punish TCP setup and loss-based congestion control; fixes are edge termination with modern CC (BBR mention), QUIC/HTTP-3 ingress (0-RTT, migration, no HOL blocking), connection splitting with pooled origin legs. Explaining WHY radio loss ≠ congestion is the insight that separates network-aware engineers.

My Private Notes

Notes are auto-saved locally to this device.