The whole course ▾
System Design · Lesson 7 of 39 · Week 2
Caching: making reads fast
QuickBite's menu is read a hundred times for every change, and the database answers every single read. Today: caching.
Got a doubt? Ask in the comments under the video.Subscribe
QuickBite's menu is read a hundred times for every change.
QuickBite is the food delivery app we grow through the course, from one server to millions of orders.
A hundred reads, one change
QuickBite survives lunch now: four stateless app servers behind a pair of load balancers. But yesterday's lesson ended with a warning, and today it comes true.
Watch what customers actually do. They open a restaurant and read its menu. Thousands of people, all reading the same few hundred menus, through the app servers, and every single read becomes a query on the orders DB. Each one takes about twenty milliseconds of the database's time.
A restaurant changes its menu maybe once a day. In that same day, the menu is read a hundred times or more for every change. So the database spends most of its effort computing the same answer, over and over, for data that didn't change. That's exactly the kind of work a cache removes.
A cache is a small, fast store, usually in memory, that keeps answers you've already computed. Reading from memory takes about a millisecond across the network, against twenty for the database query. And every read the cache answers is a read the database never sees.
Cache-aside, step by step
The most common way to use a cache has a name you should say in interviews: cache-aside.
We add a cache, Redis, next to the database. When a menu request arrives, the app first asks the cache for the key, menu forty-two. If it's there, that's a hit, and the answer comes back in a millisecond. If it isn't, that's a miss: the app reads the database as before, then saves the answer in the cache, so the next reader gets a hit. The app does all the work, and the cache sits to the side. Hence: cache-aside.
Now the maths that makes caching worth it. A hit costs about one millisecond, and a miss about twenty-one, the database plus the cache check. At a ninety percent hit rate, the average read takes about three milliseconds instead of twenty. And the database load drops to one read in ten. Same database, ten times the traffic.
Where caches live
Redis isn't the only cache between a customer and the database. There are four layers, and a good answer knows them all.
Closest to the user, the phone itself keeps what it just showed, so scrolling back up costs nothing. Next, a CDN keeps copies of files near the user, like food photos, which is lesson nine. Then the application cache, Redis or Memcached, which is today's star. And finally, the database keeps hot pages in its own memory buffers. Each layer you hit early saves every layer behind it.
And the layers multiply. Say the phone answers thirty percent of requests from its own copy, so seventy percent go on. The CDN answers half of those, so thirty-five percent remain. Then Redis answers ninety percent of what's left, and only about three and a half percent of all requests ever reach the database. That's how big apps survive with surprisingly small databases.
Keeping it fresh
Here's the catch with every cache: it can be wrong. A restaurant raises a price, and the cache still holds the old menu.
There are three ways to handle writes. With cache-aside, the app updates the database and deletes the cached menu, so the next read fills it fresh. With write-through, every write goes to the cache and the database together: always fresh, but every write is slower. With write-back, writes land in the cache first and reach the database later: very fast, but if the cache crashes before it writes back, that data is gone. For a menu, cache-aside is the right call.
And whichever you choose, give every entry a time to live, a TTL, like ten minutes. When it runs out, the cache simply forgets the entry, and the next read fetches fresh data. Even if you forget to delete something, it can never be wrong for longer than its TTL.
When the cache is full
Memory is expensive, so the cache is much smaller than the database. Something has to go.
The usual rule is LRU: least recently used. The cache keeps track of when each item was last read. When it's full and a new item arrives, it drops the item nobody has asked for in the longest time. Last month's closed café goes; today's popular lunch menu stays, because people keep asking for it. Popular items stay in by being popular, which is exactly what you want.
A good cache holds data that's read often and changes rarely: menus, restaurant pages, ratings. Don't cache a rider's live location, which changes every few seconds, so the cache would always be wrong. And don't cache payment status, where a stale answer means charging someone twice.
In the interview
Caching comes up in nearly every interview, usually right after the database starts to struggle.
Say this: the menu is read-heavy, so I'll add a cache-aside layer in Redis, with a TTL. At a ninety percent hit rate, the database sees a tenth of the reads. Notice that the sentence gives a reason, a pattern and a number.
Then come the follow-ups. How do you keep it fresh? Delete on write, plus a TTL. What if the cache goes down? Reads fall back to the database, slower but correct, which is why you never store anything only in the cache. And what exactly do you cache? Menus, not live locations or payments.
The common mistake is saying add a cache and stopping there. Interviewers want three things: what you cache, how it stays fresh, and the hit rate you expect.
Quick check
Your cache hit rate drops from 95% to 50%. What happens to the database?
Show the answer
It gets ten times the reads: misses go from 5% of reads to 50%.
A restaurant changes a price, but customers see the old one for ten minutes. Why?
Show the answer
Nobody deleted the cached menu on the write, so it stayed until its 10-minute TTL ran out. Delete the key on every write.
Remember three things. Cache what's read often and changes rarely. Use cache-aside, delete on write, plus a TTL. And the hit rate decides everything. QuickBite's menus now load in three milliseconds.
Lessons on weekdays, the week's revision on Saturday, a mock interview on Sunday.