The whole course ▾
System Design · Week 3 revision
Week 3 revision: Data
The data week in ten minutes: indexes, replicas, shards, hash rings and quorums, then six questions to test yourself.
Got a doubt? Ask in the comments under the video.Subscribe
Lesson 10Indexes
This week was all about QuickBite's data, and it started with one slow page: a customer's order history, hidden among three hundred million orders.
Lesson ten, in three lines. Without an index, a query does a full scan: sixty gigabytes read to find twelve rows. An index is a sorted copy of one column with pointers back to the rows, kept as a B-tree, so any lookup takes about four page reads, and ranges come free. And a two-column index only helps when the query uses its leftmost column, like a phone book sorted by surname.
The interview line: index what your queries filter and sort by, and say which index each table needs. And remember the price: every index is one more write on every insert, which is why write-heavy stores use LSM trees instead.
Lessons 11 and 12Copies and slices
Then the database couldn't keep up, first with reads, then with sheer size. Here's the picture both lessons built.
Lesson eleven gave the database copies: a leader that takes the writes, and followers that replay its change log and take the reads. Copying asynchronously keeps writes fast, but followers lag, so a user's own data is read from the leader. Lesson twelve split the data itself: the app servers ask a shard router, which sends each customer to one shard by a hash of the customer ID. And each shard has its own shard leader and follower, exactly as before.
The interview lines: asynchronous read replicas, with read your own writes for a user's own data; and when one machine can't hold it all, shards by a hash of the customer ID. Then name the query that doesn't fit, the restaurant dashboard, and how you'd serve it: a second copy, split by restaurant.
Lesson 13The hash ring
Then a fifth cache node turned into an outage, and taught us why dividing by the number of machines is a trap.
Lesson thirteen, in three lines. With hash mod N, going from four nodes to five moves eighty percent of the keys, and a cache full of misses hits the database. On a hash ring, each key belongs to the first node clockwise, so a new node takes just one slice, about one Nth of the keys. And virtual nodes, many points per machine, keep the slices even.
The interview line: consistent hashing with virtual nodes, so adding a node moves only about one Nth of the keys, and a dead node's load spreads evenly. Always say what moves; that's what interviewers listen for.
Lesson 14Consistency and quorums
And on Friday, the hardest question of the week: what happens when copies disagree.
Lesson fourteen, in three lines. Strong consistency, every read sees the latest write, for money and order status; eventual consistency for likes and ratings. CAP, said precisely: during a network partition, each copy either refuses to answer or answers with possibly stale data. And a quorum: with N copies, if writes reach W and reads ask R, and R plus W is more than N, every read sees the latest write.
The interview line: strong consistency where money moves, eventual where it doesn't, and quorums where you need correctness and availability together. And never say pick two out of three. Partitions aren't optional.
What's nextNext week: async
Look at everything we've built so far: it all answers while the customer waits. Next week is about the work that doesn't have to happen right away.
Message queues, so the confirmation email doesn't slow down the order. Kafka, where one order event feeds analytics, notifications and fraud checks. Idempotency, because duplicates always happen, and a double-tapped Pay button must never charge twice. Live rider tracking on the map. And rate limiting, for the day a bot sends QuickBite a flood of logins.
Test yourself
An index on (restaurant_id, created_at). Can "restaurant 9's orders tonight" use it?
Show the answer
Yes: it starts with restaurant_id, then reads one time range.
A rider accepts a job, but their app still shows it as open. You use read replicas. Why?
Show the answer
Replica lag: the read hit a follower that hadn't caught up. Read your own writes.
You shard users by signup date. What goes wrong?
Show the answer
A hot shard: every new signup lands on the newest shard.
A 50-node cache on a hash ring loses one node. How much of the cache is gone?
Show the answer
About 1/50 of the keys, 2%; virtual nodes spread its load across the rest.
N = 5 copies and W = 3. What's the smallest R that always sees the latest write?
Show the answer
R = 3, because R + W must be more than 5.
During a network split, the likes counter keeps answering with old numbers. Consistent or available?
Show the answer
Available, and that's the right choice: a stale like count harms no one.
Lessons on weekdays, the week's revision on Saturday, a mock interview on Sunday.