The whole course ▾
System Design · Lesson 20 of 39 · Week 5
Microservices, gateways and service discovery
QuickBite's one codebase now has 80 engineers, and every deploy is a risky, all-or-nothing event. Today: microservices.
New System Design videos go up at 6 pm. Subscribe on YouTube so you don't miss it.
Got a doubt? Ask in the comments under the video.Subscribe
QuickBite splits orders, menus and payments.
QuickBite is the food delivery app we grow through the course, from one server to millions of orders.
One app, eighty engineers
Until now, QuickBite's code has been one application: one codebase, deployed as one unit. That served us well for a long time. Here's when it stops.
Here's QuickBite today: a load balancer in front of many copies of one QuickBite app, which contains everything: orders, menus, payments and riders. They all share one database. With five engineers, this was perfect: one thing to build, one thing to test, one thing to deploy. Now there are eighty engineers in eight teams, all changing the same code, every day.
And it shows. Every change must be tested with every other change, so deploys go out only twice a week, and each one is nervous. A bug in the menu code blocks an urgent payments fix from shipping. The menu pages need fifty servers at lunch while payments needs five, but they can only scale together. And one memory leak in one small feature can take down the whole app.
Split it, carefully
The fix is to split the app into services, each owned by one team. But how you split matters far more than whether you split.
Split along what the business does. An orders service, a menus service, a payments service: each owned by one team, deployed on its own schedule, scaled on its own. And the crucial rule: each service owns its data. The orders DB, menus DB and payments DB are separate, and no service reads another's tables. If orders needs a dish's price, it asks the menus service. That rule is what lets each team change its data without breaking anyone.
And here's the advice most interviewers want to hear: start with a monolith. Splitting too early means paying all the costs of a distributed system before you have the problems it solves. Split when teams keep blocking each other, or when parts need very different scaling, and then split one piece at a time, starting with the one that hurts most.
One front door
Now the phone app would be talking to three services instead of one. It shouldn't have to know that.
So put an API gateway in front, as the single front door. The mobile app calls one address. The gateway looks at each request's path, slash orders, slash menus, slash pay, and sends it to the right service. The app never learns how many services there are, so teams can split or merge them later without shipping a new app.
Besides routing, the gateway does the jobs every service would otherwise repeat. It routes by path. It checks who you are once, at the door, and passes a trusted user ID inside. It applies rate limits, from lesson nineteen. And it handles TLS, logging every request, and timeouts. Each service behind it gets clean, trusted requests, and can focus on its own job. Teams usually use one from their cloud provider, or open-source ones like Kong or Nginx, rather than writing their own.
Finding each other
Inside, services call each other. But services move: new copies start, old ones die, and addresses change all day long.
When the orders service needs a price, which address does it call? There might be three copies of menus right now, on addresses that didn't exist an hour ago. The answer is a service registry. Every copy of the menus service registers itself when it starts, and keeps sending heartbeats while it's healthy. The orders service asks the registry for a healthy address, then calls it directly. A copy that stops sending heartbeats is dropped from the list.
In practice, platforms do this for you. In Kubernetes, the menus service gets a stable name, and DNS, the same DNS from lesson one, plus the platform's routing, sends each call to a healthy copy. Bigger systems add a service mesh, which also handles retries and encryption between services, without changing their code. Either way, the idea is the same: services find each other by name, never by an address typed into the code.
Call, or publish?
Services talk to each other in two ways, and choosing between them shapes how the whole system fails.
Synchronously: orders calls menus, and waits. Simple, and you get an answer now. But if menus is down, orders fails too, and a chain of five calls adds up five services' delays, and their failures. Asynchronously: orders publishes order placed to Kafka, from lesson sixteen, and moves on, and whoever cares reacts. A service that's down simply catches up later. So: call when you need an answer right now, and publish an event for everything else.
And be honest about the price. Every call between services crosses the network, and can fail or be slow: that's tomorrow's lesson. Data split across services can't be joined, or changed in one transaction: that's Wednesday's. And following one request through ten services needs tracing: that's Friday's. Microservices trade code problems for distributed-system problems. Which is exactly why the rest of this week exists.
In the interview
In an interview, the boxes you draw are services, so how you split them is one of the first things judged.
Say it like this: services split by business capability, each owning its own data, behind an API gateway, talking by calls when they need an answer and by events otherwise. And if the company is small, say you'd start with a well-organised monolith. Knowing when not to use microservices is a senior signal.
Then expect these. Why not one shared database? Because then every team's change can break another team, and you've lost the independence you split for. How do services find each other? A registry, or the platform's DNS. And should this service call, or publish an event? Call if you need the answer now; publish if others just need to know. And how big should a service be? Small enough for one team to own completely, and no smaller.
The common mistake: splitting the code into services, but leaving them all on one shared database. You pay every cost of microservices, and get none of the independence. A distributed monolith.
Quick check
Five engineers, one product, a year old. Should they split into microservices?
Show the answer
No: keep a well-organised monolith, and split later when teams or scaling demand it.
The orders service needs a dish's price. Should it read the menus database directly?
Show the answer
Ask the menus service through its API: each service owns its data.
Remember: start with a monolith, and split by business capability when it hurts. Each service owns its data. And a gateway guards the front door, while discovery connects the services inside. Tomorrow, what happens when one of those services gets slow.
Lessons on weekdays, the week's revision on Saturday, a mock interview on Sunday.