The whole course ▾
System Design · Week 2 revision
Week 2 revision: Scaling
Scaling week in ten minutes: servers, load balancers, caches and CDNs, then six questions to test yourself.
Got a doubt? Ask in the comments under the video.Subscribe
Lessons 5 and 6More servers, one front door
This week QuickBite went from one server melting at lunch to an app that scales. Let's rebuild it on one board, lesson by lesson.
Lesson five: one server couldn't take lunch, so we scaled out. Customers now reach a pair of load balancers, an active one and a standby. They spread requests across stateless app servers, which keep sessions in a shared session store, so any server can answer anyone. Lesson six added the details: least connections to choose a server, and health checks to skip dead ones. And everything still ends at one orders DB.
And the numbers behind it. One server handles about two hundred requests a second, and QuickBite's lunch peak is five hundred. So you divide the peak by one server's capacity, round up, and add one spare, plus one, so a single failure doesn't hurt. Three plus one: four servers.
The interview line from those two lessons: stateless app servers behind a layer seven load balancer, with health checks and a standby, so nothing in front of the database is a single point of failure.
Lessons 7 and 8Caching
Then the database became the bottleneck, and two lessons took most of its reads away.
Lesson seven, in four lines. Cache-aside: check the cache, and on a miss, read the database and save the answer. At a ninety percent hit rate, the database sees a tenth of the reads, and the average menu loads in about three milliseconds. On a change, write the database first, then delete the key. And give everything a TTL, while LRU drops the least recently used items when memory runs out.
Lesson eight showed how a cache fails. A hot key, like the celebrity chef's menu, melts the one machine that holds it: fix it with replicas or a short local copy. A stampede, when one popular key expires and thousands miss at once: fix it with coalescing, early refresh and jitter. And a cold cache after a restart: warm the hot keys first.
The caching line to say in an interview: cache-aside in Redis, with a TTL; on a write, update the database and then delete the key; and coalesce misses on hot keys, so a single expiry can't stampede the database. Then give the hit rate you expect, and what it does to the database.
Lesson 9Files and the CDN
And on Friday, the photos finally moved out of the database.
The restaurant app asks the app server for a presigned URL, then uploads the photo straight to object storage, so the bytes never touch our servers. Customers load photos from the nearest CDN edge, which fetches from storage only on a miss. Photos are cached for a year, every new version gets a new name, and thumbnails are two hundred times smaller than the original.
Why the CDN is worth it: a photo from storage across the country takes around two hundred milliseconds, and from a nearby edge around twenty. With ninety-five percent of requests answered at the edge, storage sees just one request in twenty.
And the line to say whenever a design has images or video: files in object storage through presigned URLs, only metadata in the database, and delivery through a CDN, in a few sizes.
What's nextNext week: the database
Look at the board one more time: everything still ends at one orders database. That's next week.
Next week is all about data. Indexes, so lookups stay fast as the tables grow. Replication, so there are copies to read from and to fail over to. Sharding, when one machine can't hold it all. And consistent hashing, then the CAP trade-off that every distributed database makes.
Test yourself
Peak traffic is 900 requests a second; one server handles 200. How many servers?
Show the answer
Six: five to carry the load, plus one spare.
Users get logged out after you add a second server. Why?
Show the answer
Sessions lived in one server's memory. Move them to a shared store.
Requests take anywhere from 50 ms to 5 seconds. Which load balancing method?
Show the answer
Least connections: new requests go to the least busy server.
Your hit rate falls from 95% to 50%. What happens to the database?
Show the answer
Ten times the reads: misses go from 5% to 50%.
At exactly 1 PM the database CPU hits 100%. The cache looks fine. What happened?
Show the answer
A stampede: a hot key expired and every request missed at once.
A restaurant changes a photo, but customers see the old one for days. Fix?
Show the answer
Give the new photo a new file name.
Lessons on weekdays, the week's revision on Saturday, a mock interview on Sunday.