Data Consistency Across Services

What You Gave Up, Precisely

 microservices + database-per-service + sagas =
 CONSISTENCY IS PER-SERVICE; the FEDERATION is eventual.

 concrete consequences to design around:

 - orders confirmed BEFORE inventory fully settled (ms–s window)
 - dashboards lag writes (replica/ETL lag)
 - cross-service invariants ("no order without stock")
   are ENFORCED BY PROCESS (sagas), not constraints

 the failure mode isn't using eventual consistency —
 it's pretending you didn't.

Preserving Critical Invariants Anyway

 some rules cannot bend. tier them:

 HARD INVARIANTS (never violated, even briefly):
   money conservation, stock ≥ 0 at capture time,
   unique usernames
 → enforce INSIDE the owning service's local txn
   (single-writer = real constraints available!)
   cross-service pieces join via synchronous pre-checks
   or reservation patterns (hold-then-commit).

 SOFT INVARIANTS (converge quickly, tolerate windows):
   denormalized counts, search indexes, caches, feeds
 → events + replicas + reconciliation sweeps.

 RESERVATION PATTERN for hard+cross-service:
   inventory.reserve(hold 30s) → pay → confirm | expire-hold
   converts distributed atomicity into timed local leases ✓

The Reconciliation Safety Net

 eventual systems need PROOF they eventually converge:

 □ INVARIANT SWEEPS: periodic jobs recompute truth:
     sum(payments) == balance shown? reserved-but-unconfirmed
     stock aging > threshold? flag & heal.
 □ DRIFT DETECTION between replica and source streams
   (checksums, counts per window)
 □ COMPENSATION queues: saga leftovers (stuck PENDINGs)
   surfaced, aged, resolved — not silently rotting
 □ AUDIT trails sufficient to REPLAY repairs

 reconciliation is the difference between
 "eventually consistent" as an ARCHITECTURE and as an EXCUSE.

Honest UX for Eventual Windows

 surface the model instead of hiding it:

 - OPTIMISTIC UI: render expected state immediately;
   reconcile on confirmation event ("✓ Order placed")
 - PENDING as a first-class status users understand
 - STALENESS indicators where stakes warrant
   ("inventory updated 20s ago")
 - read-your-writes routing for THIS user's fresh data
   (version tokens/sticky windows from global-consistency)

 users forgive eventual consistency they can SEE.
 they file lawsuits over consistency that lies silently
 about money and commitments.

Data	Consistency tier	Enforcement
Ledger balances	hard	local txn in owner
Stock at capture	hard-ish	reservations
Order history views	soft	replicas+sweeps
Recommendation state	softest	rebuild anytime

Interview Framing

“Your services are eventually consistent — what could go wrong and how do you bound it?” scored shape: invariant tiering table (hard/soft) with enforcement homes named, reservation pattern demonstrated for the cross-service hard case, reconciliation machinery listed as first-class, UX honesty patterns included. This question separates people who ADOPTED eventual consistency from people who OPERATE it — the safety nets are the tell.