Cost vs Performance

The Curve That Governs Spending

Performance improvements cost money non-linearly. The cheap wins come first; each further increment costs more:

 latency achieved vs relative monthly cost (illustrative)

 cost ▲                                    ██ 10x cost
      │                              ██████
      │                        ███████           ← each 2x speedup
      │                 ███████                   costs multiples more
      │          ███████
      │   ██████
      └──────────────────────────────► faster
        p95: 800ms  400ms  200ms  100ms  50ms

 first moves: indexes, caching, right-sizing     → nearly free
 middle moves: replicas, bigger instances, CDNs  → linear-ish
 last miles: NVMe everything, in-memory datasets, multi-region active-active → multiples

Engineering judgment = knowing where on this curve the product belongs.

When Performance Is Worth Anything

Some paths print money directly:

Path	Why the last 100ms matters	Evidence class
Checkout / payment submit	Cart abandonment correlates with delay	A/B measured revenue lift
Ride match	Users re-request or open competitor apps	Match-funnel metrics
Search-as-you-type	Feel of instant defines product	Engagement data
Ad auctions	Latency = lost auctions at bid time	Direct revenue math

For these, spending 5–10x on infrastructure can return multiples — but measured, never assumed.

When It Absolutely Isn’t

Path	Reality
Admin dashboards	Nobody’s revenue waits on them
Nightly reports	Batch window exists precisely to be slow
Analytics queries	Analysts accept seconds; caching here is waste
Cold-start paths behind rare flags	Optimize when usage appears

Over-provisioning these is the most common silent budget leak — performance theater applied where no user stands.

The Decision Procedure

 1. MEASURE the current number (percentiles, not averages)
 2. PRICE the next increment (what does 2x faster cost monthly?)
 3. VALUE the improvement (revenue lift? retention? support load?)
 4. COMPARE against alternative uses of the same money
    (that $8k/mo might be two engineers' week on a better algorithm)
 5. DECIDE with an owner: "checkout gets NVMe + read replicas;
    admin stays shared-small"

Step 4 is the one teams skip — infrastructure spend competes with headcount and roadmap, not with nothing.

The Reversibility Note

Cost decisions should stay revisable:

• Prefer managed scaling knobs over hardware commitments.
• Tag spending by component so regressions are visible (“egress doubled after v2 launch”).
• Schedule re-review: a 10x-justified spend at 1M users may be indefensible at 10M — or trivially affordable. The curve moves under you.

Interview Framing

Senior candidates volunteer this trade unprompted: “we could cut feed p95 from 200ms to 80ms with an in-memory hot set — that roughly triples cache spend; justified for logged-in home, not for the settings page.” One sentence demonstrates measurement discipline, pricing awareness, and per-path judgment — the entire lesson in spoken form.