The whole course ▾

System Design · Lesson 11 of 39 · Week 3

Replication: copies of your data

Updated 22 Oct 2026 · By Akshay, software engineer, India · 6 min read

QuickBite's one database answers every read and write, and lunchtime is too much for it. Today: replication, copies that share the load.

Got a doubt? Ask in the comments under the video.Subscribe

QuickBite in this lesson

QuickBite's database can't keep up with reads.

QuickBite is the food delivery app we grow through the course, from one server to millions of orders.

One database, every read

One database, every read
App servers
Orders DB
one machine
reads + writes
95% busy
and if it dies?

Yesterday, indexes made each query fast. But at lunch, QuickBite's database gets so many queries that even the fast ones wait in line.

Here's QuickBite at lunch: many app servers, all talking to one orders DB. For every order placed, there are about twenty reads: menus the cache missed, order status checks, history pages, riders' apps asking for new jobs. Writes are easy for the database. It's the reads, thousands a second, that push it to its limit.

At the lunch peak, its processor sits at ninety-five percent, and queries that took a millisecond take a hundred. Worse, it's one machine. If it dies, QuickBite stops taking orders, completely, until someone fixes it. Two problems: too many reads, and a single point of failure. One idea fixes both.

Copies of the data

Copies of the data
IN ONE LINE
Writesgototheleader;readsspread
acrossfollowersthatreplayits
changelog.
One database, every read
App servers
Orders DB
one machine
reads + writes
95% busy
and if it dies?
Follower
copy 1
Follower
copy 2
change log
change log
leader: takes writes
reads
reads

The idea is to keep copies of the same database on several machines. That's called replication.

One machine stays in charge: the leader. Every write still goes to it. Next to it we add two followers. They hold full copies of the data, and they stay up to date by replaying the leader's change log: every insert, update and delete, in the same order. Followers never take writes from the app. They only copy what the leader did.

Now the app sends its writes to the leader, and its reads to the followers. Twenty reads for every write, spread across two followers, or five, or ten: you add followers until the reads fit. These copies are called read replicas, and adding them is the classic first step in scaling a read-heavy database.

In one line: writes go to the leader, and reads spread across followers that replay its change log. And as a bonus, we now have spare copies of everything, which will matter the night the leader dies.

Wait for the copies, or not?

Wait for the copies, or not?
Synchronous
Every write waits for the followers
One slow follower slows everyone
A dead follower can block writes
Asynchronous
The leader confirms straight away
Writes stay fast
A slow follower hurts no one
THE PRICE OF ASYNCHRONOUS
Followersrunbehind:
usuallymilliseconds,
sometimesseconds.That
gapisreplicalag.

Here's the decision that matters most. When the leader tells the app a write is done, have the followers got it yet?

Synchronous: the leader waits until the followers confirm they have the change, and only then tells the app it's done. Nothing is lost if the leader dies, but every write waits for the slowest follower, and a dead follower can block writes completely. Asynchronous: the leader confirms as soon as it has the change itself, and the followers catch up a moment later. Writes stay fast. Most systems choose asynchronous, often with one synchronous follower kept as a safety copy.

But asynchronous has a price, and it has a name: replica lag. Followers are always a little behind the leader. Usually by milliseconds. Under heavy load, sometimes by seconds. And a few seconds is plenty of time for a very confusing bug. Watch it as a number on your dashboards, because it grows exactly when you're busiest.

Where did my order go?

Where did my order go?
Customer
Leader
Follower
place order
1
copy, later
2
my orders?
3
none yet!
4
200 ms behind
Read your own writes
1
Your reads go to the leader
2
For a few seconds after you
write
3
Or: a follower that has caught up
DECIDE PER FEATURE
Lagisfineformenus
andratings;notfor
yourownorderora
payment.

Here's that bug. Almost every app that adds read replicas meets it, usually on its busiest day.

A customer places an order. The write goes to the leader, which confirms it, and the app says: order placed. A moment later, the app opens the orders page, and that read goes to a follower, which is two hundred milliseconds behind and hasn't received the order yet. So the customer sees: no orders. They panic, and order again. Two biryanis.

The fix has a name: read your own writes. Right after a customer writes something, send that customer's reads to the leader, which is never behind, for a few seconds. Everyone else keeps reading from followers. A finer version remembers where the customer's last write sits in the change log, and reads from any follower that has caught up past it. Either way, nobody watches their own change disappear.

So decide feature by feature. A menu that's a second old? Nobody notices. A restaurant's rating? Fine. But your own order, or a payment you just made, must come from the leader. Saying that distinction out loud is exactly what interviewers want to hear. The rule of thumb: if a user would notice their own change missing, read it from the leader.

When the leader dies

When the leader dies
Failover's sharp edges
Async: the newest writes can be lost
Two leaders at once: split brain
A false alarm triggers it
Where did my order go?
Customer
Leader
Follower
place order
1
copy, later
2
my orders?
3
none yet!
4
200 ms behind
promoted: new leader
Read your own writes
1
Your reads go to the leader
2
For a few seconds after you
write
3
Or: a follower that has caught up
DECIDE PER FEATURE
Lagisfineformenus
andratings;notfor
yourownorderora
payment.

Replication fixed the reads. It also gives us a way to survive the leader dying.

At two in the morning, the leader's disk fails, and it dies. Writes stop. The fix is called failover: a follower is promoted to be the new leader, the app's writes are pointed at it, and the other followers start copying from it instead. Managed databases do this automatically, often within a minute or two. And when the old leader comes back, it must rejoin as a follower, never as a second leader. And the app has to find the new leader too, usually through one database address that the failover moves to the new machine.

But failover has sharp edges. With asynchronous copying, the newest writes may never have reached the follower, and they're lost. If the old leader was only cut off, not dead, you can end up with two leaders taking writes at once: split brain, which systems prevent by making sure the old one steps down. And a false alarm, a leader that was only slow, can trigger a failover nobody needed.

In the interview

In the interview
SAY THIS IN THE INTERVIEW
Leader-follower,
asynchronous,withread
replicas;auser'sowndata
readfromtheleader.
Follow-ups to expect
->
What's lost when the leader
dies?
->
How do you handle replica lag?
->
Why not several leaders?
THE COMMON MISTAKE
Addingreplicasand
forgettinglag:users
can'tseewhatthey
justwrote.

Replication shows up in nearly every design, usually in a single sentence. Make it a good one.

Say it like this: a leader and followers, replicating asynchronously, with reads spread across replicas, and a user's own recent data read from the leader. Then say what happens on failover. That's one breath, and it answers the next three questions before they're asked.

Then expect these. What's lost when the leader dies? With asynchronous copying, possibly the last few writes, so say so. How do you handle replica lag? Read your own writes. And why not several leaders, each taking writes? Each region can, which helps global apps, but two leaders can change the same row at once, so conflicts need resolving. That's lesson thirty-eight.

The common mistake: adding read replicas and never mentioning lag. Interviewers wait for it, and the candidate who brings it up first stands out.

Quick check

Quick check
?
Writes are fine, but reads
are overloading the
database. What do you add?
Read replicas: followers that
copy the leader and take the
reads.
?
A user changes their address,
refreshes, and sees the old
one. Why, and the fix?
Replica lag. Read the user's own
data from the leader for a few
seconds after a write.
Remember
Writes to the leader, reads to
followers
Asynchronous is fast, but
followers lag
Failover promotes a follower
Quick check

Writes are fine, but reads are overloading the database. What do you add?

Show the answer

Read replicas: followers that copy the leader and take the reads.

Quick check

A user changes their address, refreshes, and sees the old one. Why, and the fix?

Show the answer

Replica lag. Read the user's own data from the leader for a few seconds after a write.

Remember: writes go to the leader, and reads to the followers. Asynchronous copying is fast, but followers lag, so read your own writes from the leader. And failover promotes a follower when the leader dies. Next, the problem replication can't solve: data too big for one machine.

Practise with flashcardsYour progress stays on this device.
New System Design videos at 6 pm

Lessons on weekdays, the week's revision on Saturday, a mock interview on Sunday.