Replication
A database on one machine has one machine's limits. Every read and every write lands in the same place, and when that place is busy, everyone waits.
Replication copies the data onto more machines. Reads can now spread out: any copy can answer a question, so the work splits across all of them.
Writes are different. They still go to one machine, the primary. Sending every change through a single point is what keeps them in order, so all the copies apply the same edits in the same sequence.
The copies are called replicas, and they always run a little behind. When the primary takes a write, it has to send that change out, and sending takes time. For a moment the primary holds a newer value than the replicas do. A read routed to a replica in that moment gets an answer that used to be true. That gap is lag.
Watching the lag
One balance, three copies, and the lag between them
One key holding one integer, a primary, and two replicas with authored lags of 2 and 4 steps. The balance starts at 100. Nothing here is timed or sampled: every value, arrival, pending list, stale count, and loss set is computed from the step index over a frozen twenty-operation stream. Real systems replicate over a network whose delay varies, use quorums rather than all-replicas acknowledgement, and can often recover a failed primary rather than discarding it.
- primary
- 100
- Replica 1
- 100
- Replica 2
- 100
- reads served stale
- 0
- step
- 0 / 20
When the primary dies, a replica takes its place. The figure promotes the one that had received the most writes, since it is closest to current. Whatever had not reached it is gone. There is no other copy of those writes to restore them from.
Asynchronous mode is where this bites. The primary confirmed each of those writes the instant it received them, so the client was already told they succeeded. They were acknowledged, then lost anyway.
Synchronous mode closes the gap. A write is not confirmed until both replicas hold it, so nothing acknowledged can vanish in a failover. The cost shows in the step counter: every write now waits, and the same run takes longer. That wait is the price of the guarantee.
One last thing, because it is easy to get wrong. A replica is not a backup. A backup is a copy from a moment in time that you keep so you can return to it. A replica is a live mirror. Delete a row on the primary and every replica deletes it too, faithfully, within moments. Replication saves you from a machine dying. It does not save you from a mistake, because it copies the mistake.