The whole course ▾

System Design · Lesson 19 of 39 · Week 4

Rate limiting

Updated 22 Oct 2026 · By Akshay, software engineer, India · 6 min read

At 3 AM, a bot tries ten thousand passwords a minute on QuickBite's login. Today: rate limiting, and how to stop a flood.

Got a doubt? Ask in the comments under the video.Subscribe

QuickBite in this lesson

A bot hammers QuickBite's login.

QuickBite is the food delivery app we grow through the course, from one server to millions of orders.

A flood at the front door

A flood at the front door
Bot
10,000 tries a
minute
Load
balancer
Login service
Users DB
POST
/login
check
password
CPU 100%
accounts at risk

Everything we built this week makes QuickBite faster and sturdier. Today we protect it, starting at the front door: the login page.

At three in the morning, a bot starts guessing passwords: ten thousand login attempts a minute, taken from a list of leaked passwords, through the load balancer to the login service. Each attempt checks a password hash from the users DB, and password hashing is slow on purpose, so every guess costs real processor time. Real customers trying to log in for a late-night order wait, then time out.

Two things are at risk: the service, which is drowning, and the accounts, because one of those guesses may be right. And floods aren't always attacks. A buggy app stuck in a retry loop, or a partner calling our API too eagerly, can do the same damage by accident. So we need a rule: no single caller gets more than its fair share, and we enforce it before the expensive work starts.

The token bucket

The token bucket
Token bucket
1
Each caller has a bucket of 10 tokens
2
Each request takes one token
3
Tokens refill: 1 a second
4
Empty? Reject with 429
WHY IT'S POPULAR
Allowsshortbursts(up
tothebucketsize),but
capstheaveragerate.

That rule is called rate limiting. The most common way to do it is a bucket of tokens.

Picture each caller, say each IP address, holding a bucket of ten tokens. Every request takes one token. Tokens refill at a steady rate, one a second, until the bucket is full again. And if the bucket is empty, the request is rejected straight away, with status four hundred and twenty-nine: too many requests. A real customer never notices. A bot hits the wall within seconds.

It's popular because it matches how people behave. A quick burst of requests is fine, because the bucket was full. But over time, nobody can go faster than the refill rate. There are only two numbers to tune, the bucket size and the refill rate, and very little to store: for each caller, a token count and the time it was last updated.

Counting in windows

Counting in windows
Fixed window
Count per clock minute: max 60
Resets on the minute
Double bursts at the edges
Sliding window
Counts the last 60 seconds
No double bursts at edges
Smooth, fair limits
WHICH ONE?
Tokenbucketformost
APIs;slidingwindowwhen
limitsmustbestrict;fixed
windowwhensimpleisenough.

The other family of limiters counts requests in windows of time. They're simpler to explain, and one of them has a sneaky flaw.

A fixed window counts per clock minute: at most sixty in the minute starting at twelve o'clock, then the count resets. Simple. But a client can send sixty at eleven fifty-nine and fifty-nine seconds, and sixty more right at twelve: a hundred and twenty in two seconds. A sliding window always counts the last sixty seconds from right now, so there's no edge to exploit. A common trick estimates it cheaply, from the current and the previous minute's counts.

So which one? A token bucket for most APIs, because it allows natural bursts. A sliding window when the limit must be strict, like a paid plan's quota. And a fixed window when simple is good enough. In an interview, name one, and say why it fits.

Where the limit lives

Where the limit lives
Clients
API gateway
checks limits
Redis
shared counters
Services
requests
take a
token
1
allowed
2
429 +
Retry-After
< 1 ms per check
Choose the key
->
Per IP: anonymous traffic
->
Per user or API key: logged in
->
Per endpoint: login stricter than
menus

Next: where to check the limit, and how ten servers agree on one count.

Check limits at the edge, in the API gateway that sits in front of every service, so rejected requests cost almost nothing and never reach the services. But QuickBite runs ten gateway servers, and a bot's requests are spread across all of them. If each kept its own count, the bot would get ten times the limit. So the counts live in one shared place: Redis, the same in-memory store we used for caching in lesson eight.

Each check is one quick Redis operation, under a millisecond: take a token for this caller. It must be a single atomic step, or a small script that runs as one, so two gateways can't both take the last token. And if Redis itself is slow or down? Most systems fail open: they let requests through, and lean on other defences, rather than lock every customer out because a limiter broke.

And choose what you count per. Per IP address for anonymous traffic, like the login page. Per user, or per API key, once someone is logged in, so a whole office behind one IP isn't punished. And per endpoint: login gets a much stricter limit than browsing menus, because one is expensive and dangerous, and the other is cheap and cached.

Saying no, politely

Saying no, politely
A good 429
Status 429: too many requests
Retry-After: wait 30 seconds
Headers: how many requests are left
BEYOND PER-IP LIMITS
Forlogins,alsolimitper
account,andadda
CAPTCHAafterrepeated
failures.

A rate limiter says no a lot. How it says no matters, for honest clients and for attackers.

Answer with status four hundred and twenty-nine, so clients know exactly what happened. Add a Retry-After header, wait thirty seconds, so honest apps back off instead of hammering, with the exponential backoff from lesson fifteen. And many APIs add headers saying how many requests are left in the current window, so well-behaved clients can slow down before they ever hit the limit.

One warning. A serious attacker uses thousands of IP addresses, a few guesses each, and slips under every per-IP limit. So logins also get limits per account: five failures, then slow down or lock for a while. And after repeated failures, a CAPTCHA, which is cheap for a human and expensive for a bot.

In the interview

In the interview
SAY THIS IN THE INTERVIEW
Atokenbucketperuseror
IP,checkedatthe
gateway,countedinRedis;
429withRetry-After.
Follow-ups to expect
->
The bot uses 10,000 IPs?
->
What if Redis is down?
->
Different limits per plan?
THE COMMON MISTAKE
Countingperserver:
with10servers,the
reallimitis10times
toohigh.

Rate limiting comes up as a full question, design a rate limiter, and as a detail in almost every public API.

Say it like this: a token bucket per user, or per IP for anonymous traffic, checked at the API gateway, with counters in Redis so every gateway server shares them, and a four-twenty-nine with Retry-After when the bucket is empty. That covers the algorithm, the place, the shared state, and the client.

Then expect these. The bot uses ten thousand IP addresses? Limits per account, and CAPTCHAs. What if Redis is down? Fail open, with a rough local limit on each gateway as a backstop. And different limits per plan, free versus paid? Key the bucket by API key, and give each plan its own bucket size and refill rate.

The common mistake: each server counting on its own. With ten servers behind a load balancer, the limit you thought you set is ten times too generous. Shared counters, or nothing.

Quick check

Quick check
?
A limit of 100 a minute, in fixed
windows. What's the most a
client can send in 2 seconds?
200: 100 at the end of one
minute, and 100 at the start of
the next.
?
A bucket of 10 tokens, refilling
1 a second, is full. 15 requests
arrive at once. How many pass?
10: one per token. The other 5
get a 429.
Remember
Token bucket: bursts OK,
average capped
Check at the gateway, count
in Redis
429, with Retry-After
Quick check

A limit of 100 a minute, in fixed windows. What's the most a client can send in 2 seconds?

Show the answer

200: 100 at the end of one minute, and 100 at the start of the next.

Quick check

A bucket of 10 tokens, refilling 1 a second, is full. 15 requests arrive at once. How many pass?

Show the answer

10: one per token. The other 5 get a 429.

Remember: a token bucket allows bursts and caps the average. Check limits at the gateway, with shared counts in Redis. And say no politely: four-twenty-nine, with Retry-After. That's week four: queues, Kafka, idempotency, live updates, and rate limiting.

Practise with flashcardsYour progress stays on this device.
New System Design videos at 6 pm

Lessons on weekdays, the week's revision on Saturday, a mock interview on Sunday.