Everything in the Beginner track quietly assumed you had one machine. The moment there are two, an unfixable trade-off appears — one that shapes every database, every queue, every cache you'll work with in production. Welcome to distributed systems.
Twenty modules of foundation — request/response, storage, scaling, reliability, the design framework. All of it built on a quiet assumption: there is a system. Singular. One MySQL primary. One Redis. One API server (or, fine, a few, but treated as interchangeable). The conversation was always about what the system does, not what happens when the system disagrees with itself. That last sentence sounds odd because the previous track gave you no reason to imagine it. It's about to become unavoidable.
Once you have more than one node — replicas, shards, microservices, multi-region writes — every "obvious" thing you knew about programming gets quietly violated. The system can be in two different states at the same time. A write can be acknowledged and then disappear. A read can return data that no client ever wrote. Two correct programs running on two correct machines can produce contradictory truths and both be right. This isn't bug-land; it's the territory.
The first concept that forces all of this into the open is the CAP theorem. It's the most-quoted, most-misstated result in distributed systems folklore. People who can quote it often get it wrong; people who don't quote it sometimes apply it correctly by intuition. The goal of this module is to give you the precise version, the production-relevant version, and a simulator that makes the trade-off impossible to forget.
Brewer's CAP theorem (formalized by Gilbert & Lynch, 2002) says: any networked shared-data system can guarantee at most two of the following three properties at the same time. This sounds like a buffet, but it isn't — one of the three is not really optional in practice. Get clear on each definition first, then we'll see why the choice is narrower than it looks.
The theorem says: during a network partition, you cannot have both Consistency and Availability. Pick one. This is the entire content of CAP, and it's much narrower than the way it's usually quoted.
That last row is worth re-reading. The "C" in CAP is not the "C" in ACID. CAP's consistency is about distributed observability: can two clients reading from two different nodes at the same moment see the same value? ACID's consistency is about constraints: does the database state satisfy the schema's invariants (foreign keys, NOT NULL, unique)? They use the same word for completely different ideas. Mix them up at your peril.
The third row in the myths table is the one that bites. In any real distributed system — once your data lives on more than one machine, especially across availability zones, regions, or data centers — partitions will happen. Network cables get cut. Switches reboot. Cloud regions lose connectivity. Even within a single datacenter, microsecond delays under load can functionally partition nodes (one node hasn't heard from another in long enough that it's effectively isolated). Partition tolerance isn't a thing you choose; it's a thing you survive.
That collapses CAP from a three-way pick to a two-way one. Once you accept that P is mandatory, the only real choice during a partition is:
When a partition happens, the minority side stops accepting writes (and often reads too). It returns errors. The majority side continues serving. "Better unavailable than wrong."
Use when: the data must be correct (money, inventory, identity). A stale read or lost write would cause real harm.
When a partition happens, both sides keep accepting reads and writes. They diverge temporarily. When the partition heals, conflicts are reconciled (last-write-wins, vector clocks, CRDTs, custom merge). "Better stale than unavailable."
Use when: staleness is recoverable. A user's shopping cart, a like count, a cache entry — all tolerate momentary inconsistency.
Two things to internalize. First, "AP" doesn't mean "no consistency forever" — it means during a partition, prefer availability. After the partition heals, AP systems work to reconverge. The technical name for this is eventual consistency (M.22 will go deeper). Second, the choice can be made per operation. DynamoDB lets you choose: for this read, do you want a fast eventually-consistent answer (AP), or are you willing to wait for a strongly-consistent one (CP)? Production systems don't pick CP-or-AP globally; they pick it per call.
Now you've heard the theory. Time to make the choice physical. The lab below puts you in control of a 3-node cluster — partition it, choose your mode, and watch the consequences of CP vs AP play out write by write.
Below: a 3-node cluster. Pick a network topology (healthy, or one node isolated, or a brain split). Pick CP or AP mode. Send writes to any node and watch what survives. The event log shows the negotiation; the verdict explains what just happened. The CP/AP trade-off should become so visceral after this lab that you'll never confuse it again.
Pick a topology, choose a mode, send some writes. The verdict will explain what happened and why — covering quorum math, divergence, and what real systems do in the same situation.
CAP describes what happens during a partition. But partitions are rare — minutes per year in a well-run system. CAP says nothing about the other 99.9% of the time. Daniel Abadi's 2010 extension fills the gap: PACELC. It reads as: "during a Partition, choose between Availability and Consistency. Else (no partition), choose between Latency and Consistency." The "ELC" half is the one most engineers haven't met, and it's the one that shapes day-to-day system feel.
some replica might be slightly behind.The EL/EC distinction is what determines whether your "strongly consistent" database feels snappy or sluggish at low load. Spanner's elegance is real, but the price is paid in every read latency — every operation does a coordination round trip. DynamoDB defaults the other way: tell me you need consistency, and I'll do the work; otherwise, fast and slightly stale is the default. Both are correct designs for different needs. PACELC just makes the trade-off visible.
Internalize this: when someone says "we need a strongly consistent database," ask "strongly consistent during partitions, or also during normal operation?" The first is CAP-C. The second is PACELC's EC. Different cost, different system, different conversation.
The distributed-systems vocabulary that shows up in every paper, every system doc, every postmortem. Master these and a whole literature opens up.
floor(N/2) + 1 — a strict majority. Ensures only one side of a partition can make progress.last-write-wins, vector clocks, application-level merge, CRDTs.Test the distributed-systems intuition. Click an answer; explanation drops in instantly.
Perfect. The CAP/PACELC framing is yours, and the trade-off is no longer a textbook diagram. Next: a closer look at consistency itself — the spectrum from linearizable down to "almost certainly eventually."
The trade-offs at the heart of distributed systems. Every module in this track will refer back to these.
Partitions will happen in any networked system. CAP collapses to a CP-vs-AP choice during partitions — and that's the only choice worth thinking about.
Real systems don't choose CP-or-AP globally. They choose per call, per query, per write. The mature view: tell me what this specific operation needs.
CAP covers partitions. PACELC covers the rest of the time. Even healthy systems trade latency for consistency — the choice is just less visible.