The whole course ▾
System Design · Mock interview 20 of 20
Design ChatGPT: serving a large language model
Design the serving system for ChatGPT.
Before you read the interview, answer the question yourself. Give yourself 10 minutes: say your design out loud, or sketch it on paper. What would you ask? Which numbers matter? Which boxes, and what's the hard part? Then read on, and score yourself with the feedback at the end.
New System Design videos go up at 6 pm. Subscribe on YouTube so you don't miss it.
Got a doubt? Ask in the comments under the video.Subscribe
Requirements
Estimation
API and data
High-level design
Deep dive: keeping GPUs busy
Deep dive: when demand beats supply
Wrap-up
Strong: GPUs as the scarce resource, continuous batching, KV cache care, admission and priorities. Next time: the cost per answer. Verdict: strong hire.
Score yourself
Interviewers score these seven things. Give your own attempt a tick for each one you did.
- Asks before drawing. Agrees what the system does, and how well: scale, speed, consistency.
- Turns users into requests a second and bytes a day, and says what the numbers decide.
- A small, clear contract, and a data model that fits how the data is read.
- A working path end to end before any detail. Every box has a reason.
- Picks the hard parts, compares two options, and chooses one with a trade-off said out loud.
- What breaks, what happens then, and how users notice. Retries, duplicates, limits.
- Thinks out loud, checks in with the interviewer, and takes a hint without losing the thread.
0 of 7
Lessons on weekdays, the week's revision on Saturday, a mock interview on Sunday.