Menu

Earn Premium with Referrals

Invite your friends and earn Premium rewards through our referral program.

See how it works and start inviting friends.

Why Databases Exist
HLD

Why Databases Exist

The properties that separate a database from a file — durability, concurrent access, and structured queries.

The Naive Alternative Fails Fast

 "just write JSON files to disk":

 app crash mid-write      → corrupted half-file
 two servers writing      → last-writer-wins clobbering
 "find orders > $100"     → read EVERY file, parse all
 10k reads/sec            → disk seeks melt
 delete user's data       → find every reference manually
 
 each failure has a name:
   durability · concurrency · queryability · throughput · integrity
 databases are the accumulated engineering answer to all five.

Property 1: Durability

 committed data survives crashes:

 WAL (write-ahead log):  changes journaled to append-only log FIRST,
                         fsync'd; data pages written later
 crash between?          log replay restores state — the log IS truth
 
 this is why databases fsync (slow!) while apps buffer (fast):
 durability is bought with deliberate latency.

Property 2: Controlled Concurrency

 hundreds of connections reading/writing simultaneously:

 transactions + isolation levels define what interleavings
 are legal:
   two agents selling one seat → exactly one succeeds
 
 locking / MVCC machinery makes that sentence TRUE at scale,
 which ad-hoc code never achieves (check-then-act races).

Property 3: Queryability

 declare WHAT you want, not HOW to find it:

 SELECT city, AVG(fare) FROM trips
 WHERE created_at > '2026-08-01'
 GROUP BY city;

 the PLANNER decides: indexes vs scans, join order, memory use.
 decades of optimizer work available via one declarative sentence —
 versus hand-writing every access path yourself.

Property 4: Integrity

 constraints live WITH the data, enforced for every writer:

 PRIMARY KEY      identity guaranteed unique
 FOREIGN KEY      references never dangle  
 CHECK            business rules at storage layer
 UNIQUE           no duplicate emails, ever

 application bugs come and go; database constraints
 catch what slips past every code path. defense in depth.

The Full Stack a Database Provides

LayerWhat it solves
Storage enginePages, B-trees, WAL, compression
Transaction managerACID semantics
Lock/MVCC managerConcurrent access
Query plannerDeclarative efficiency
ReplicationAvailability, read scaling
Backup/recoveryDisaster tolerance

“Choosing a database” is really choosing WHICH of these matter most for your workload — the next lessons tour the families.

Interview Framing

“Why Postgres over files/Redis?” tests fundamentals vocabulary. Scored shape: name durability-via-WAL, concurrency-via-transactions, declarative queries, constraints-as-integrity — then admit when simpler tools WIN (files for blobs, Redis for ephemeral). Knowing WHY the heavyweight exists includes knowing when it’s unnecessary.

My Private Notes

Notes are auto-saved locally to this device.