The whole course ▾
System Design · Lesson 15 of 39 · Week 4
Message queues
Every QuickBite order waits for an email and a text, so when the SMS provider slows down, ordering does too. Today: queues.
Got a doubt? Ask in the comments under the video.Subscribe
QuickBite's order confirmation emails and texts.
QuickBite is the food delivery app we grow through the course, from one server to millions of orders.
Everything, right now
Week four is about work that doesn't have to happen while the customer waits. And right now, QuickBite does all of it while the customer waits.
Here's QuickBite's checkout today. The customer places an order. The order service saves it in the orders DB, then calls the email provider to send a receipt, then the SMS gateway to send a text, and only then answers: confirmed. Saving takes about ten milliseconds. The email and the text take a few hundred more, on a good day.
Then, at dinner time, the SMS gateway slows down. Each text now takes eight seconds, and every checkout waits for it. The order service's threads get stuck waiting, new orders pile up behind them, and customers watch a spinner, then give up. A text message, the least important step of all, just took down ordering. The problem isn't the slow provider. The problem is that we waited for it.
Put a queue in between
The fix is an old one. Don't do the slow work now. Write it down, and let someone else do it soon.
Now the order service saves the order in the orders DB, drops a small message into a queue, saying order four-two-one was placed, and answers the customer straight away. Fifteen milliseconds in all, whatever the SMS gateway is doing. The message is tiny, and the queue keeps it safe, on disk, until someone deals with it.
On the other side, notification workers take messages off the queue, one at a time, and do the slow work: a receipt through the email provider, a text through the SMS gateway. If the gateway is slow tonight, messages simply wait in the queue a little longer, and nobody's checkout does. If the queue grows, we start more workers. Ten workers send ten texts at once.
In one line: a queue lets the fast part finish now, and the slow part happen soon, separately. The service that writes messages is called the producer, and the workers that read them are consumers. Neither needs to know the other exists, or even be running at the same time. In practice, teams use a managed queue like Amazon SQS, or run RabbitMQ themselves; the ideas are the same.
Delivered at least once
Now the important question: what happens when a worker crashes halfway through a message?
A message's life has four steps. The queue stores it safely, often on several machines. A worker takes it, and the queue keeps it hidden from other workers for a while, say thirty seconds. That's called the visibility timeout. When the worker finishes, it acknowledges the message, and only then is it deleted. If the worker crashes first, the timeout runs out and the message reappears for another worker. Nothing is lost.
That's called at-least-once delivery, and it's what almost every queue gives you. Never lost, but sometimes delivered twice. A worker might send the text, then crash just before acknowledging, and the next worker sends it again. Two texts: annoying, but harmless. For a payment, it's not harmless at all, which is why Wednesday's lesson is about exactly that.
Retries and dead letters
Some messages fail once, and some fail every single time. A good setup treats them differently.
When a send fails, say the gateway answers with an error, the message goes back on the queue to be tried again. But not straight away: each retry waits longer than the last, one second, then two, then four, then eight. That's exponential backoff, and it gives a struggling provider room to recover, instead of hammering it. A message that fails five times, maybe a phone number that doesn't exist, moves to a separate queue: the dead letters.
Why bother with dead letters? Because without them, one broken message can be retried forever, wasting workers and, in some queues, blocking the messages behind it. In the dead-letter queue, engineers can inspect what went wrong. And once the bug is fixed, they can replay those messages back into the main queue, so no customer is quietly forgotten.
What a queue costs
So should everything go through a queue? No. Here's the trade.
Calling directly means the caller waits for the slowest step, a slow provider slows everything upstream, and if the server crashes mid-call, the work is simply lost. A queue means the caller answers straight away, a dinner-time spike just waits in line instead of overloading workers, and the work survives crashes. For receipts, texts, emails, image resizing, and reports, the queue wins easily. Even if the SMS gateway is down for an hour, the texts go out when it's back.
But the price is real. The work happens later, so the customer can't be told right away that the text failed. You now run one more system, and you must watch how long its line is, because a growing queue is the first sign of trouble. And anything the user must wait for, like whether their card was accepted, can't go on a queue at all.
In the interview
Queues appear in almost every design, the moment you draw something slow.
Say it like this: anything the user doesn't need to wait for goes on a queue, delivered at least once, with retries, exponential backoff, and a dead-letter queue. That one sentence covers what interviewers check: you know what to make async, and you know messages can fail and repeat.
Then expect these. What if a message arrives twice? The consumer must handle duplicates safely, which is Wednesday's lesson. The queue keeps growing: now what? Add workers, and scale them on the queue's length. And does the order of messages matter? Many queues don't promise it, so if it matters, say how you'd keep it. That's tomorrow's lesson.
The common mistake is the opposite one: putting a queue in front of something the user must wait for. The customer can't be told your order is confirmed, we'll charge your card later, when the card might be declined.
Quick check
A worker sends a text, then crashes before acknowledging. What happens?
Show the answer
Another worker gets the message after the timeout and sends it again: at-least-once.
One message fails every time, and workers keep retrying it. What's the fix?
Show the answer
Retry with backoff a few times, then move it to a dead-letter queue.
Remember: queue the work the user doesn't need to wait for. Delivery is at least once, so expect duplicates. And failing messages get backoff, then a dead-letter queue. Tomorrow, a different kind of queue, one that many services can read at once.
Lessons on weekdays, the week's revision on Saturday, a mock interview on Sunday.