The whole course ▾
System Design · Lesson 31 of 39 · Week 7
Design Dropbox: files in sync everywhere
You edit one slide in a 200 MB presentation, and it syncs to your other laptop in seconds. How? Today: design Dropbox.
New System Design videos go up at 6 pm. Subscribe on YouTube so you don't miss it.
Got a doubt? Ask in the comments under the video.Subscribe
One folder, every device
Yesterday, YouTube moved huge files one way: from one creator to millions of viewers. Today's system moves files in every direction, between your own devices, and has to keep them all identical.
The requirements. One folder that's the same on your laptop, your phone and the web. You can edit offline, on a plane, and everything syncs when you land. You can share a folder with a colleague, and see each other's changes. And old versions are kept, so a file you broke yesterday can be restored.
And the shape of the problem. A hundred million users, at about ten gigabytes each, is around an exabyte: a billion gigabytes. Most changes are small edits to big files: one slide, one paragraph, one photo added to an album. And many users keep exactly the same files: the same installers, the same shared documents.
That shape decides the design. If a one-slide edit re-uploads two hundred megabytes, the system wastes almost all its bandwidth, and your laptop's battery. So never send, or store, a whole file again when only part of it changed. Work in blocks.
Blocks named by their content
Here's the idea at the heart of every sync service, and of Git, too.
The client splits every file into blocks of four megabytes. Each block is named by a hash of its contents, a SHA-256 fingerprint: change one byte, and the name changes completely. A file is then just an ordered list of block hashes. So when you edit one slide, only a few blocks get new hashes, and only those few need to travel.
And deduplication comes for free. Two blocks with the same content have the same hash, so the system stores that block once, however many files contain it. Before uploading, the client simply asks: do you already have these hashes? Copy a big folder to a new place, and nothing is uploaded at all, because every block is already there.
Two kinds of storage
Blocks and the lists that describe files are very different kinds of data, so they live in very different places.
The desktop client watches your folder. When a file changes, it splits and hashes it, and asks the metadata service: which of these hashes are new? The service checks the metadata DB. The client uploads only the new blocks to block storage, which is object storage keyed by hash. Then it commits: this file, version eight, is this list of blocks. The commit is the moment the change becomes real.
Now your other devices find out. Each one keeps a connection open to a notification service, a long poll or a WebSocket, from lesson eighteen. After a commit, it tells your other laptop: something's new in this folder. The laptop asks the metadata service for changes since the version it has, version seven, sees the new block list, and fetches only the two blocks it doesn't already have.
Notice how different the two stores are. Metadata is small, but must be strongly consistent and transactional, because two devices committing at once must never corrupt a folder: it lives in a sharded relational database. Blocks are enormous, but immutable: written once, never edited, so plain object storage handles them perfectly.
When two devices disagree
You edit a document on the plane. Your colleague edits the same document at the office. Then you land.
Every commit says which version it started from. Your colleague's commit arrives first, built on version seven, and becomes version eight. Your commit arrives next, also built on version seven, but the file is now at eight. That's a conflict, caught by the same version check as lesson twenty-eight. So the service keeps both: the latest stays as the file, and yours is saved beside it as a conflicted copy.
Why not merge the two automatically? Because a sync service can't understand every file format there is: a spreadsheet, a photo, a video project. A wrong merge would silently destroy someone's work. Keeping both and letting a person decide is less elegant, and much safer. Real-time co-editing, like Google Docs, is a different design: lesson thirty-five.
Details that matter
A few details separate a demo from a service people trust with their files.
Insertions are a trap: add a paragraph near the start of a file, and with fixed four-megabyte blocks, every block after it shifts, and looks new. So many systems choose block boundaries from the content itself, with a rolling hash, so only the blocks near the edit change. Old versions are cheap: they're just old block lists pointing at blocks that are still stored. And a block is deleted only when no version of any file uses it anymore.
And the lesson that teams who build these services keep repeating: blocks are the easy part to scale, because they never change. Metadata is where the hard problems live: consistency, conflicts, sharing permissions, and millions of tiny commits. Shard it by user or by shared folder, so each folder's history stays on one shard.
In the interview
Design Dropbox, or Google Drive, is a favourite because it tests data modelling, not just boxes.
Say it like this: files are split into blocks named by their content hash, so only changed blocks are uploaded and identical blocks are stored once. Metadata, the block lists and versions, lives in a strongly consistent, sharded database, and blocks live in object storage. Devices hear about changes through a notification channel, and conflicts become conflicted copies.
Then expect these. Why blocks of about four megabytes? Small enough that an edit resends little, big enough that a huge file isn't millions of tiny pieces. Two people edit the same file? A version check, and a conflicted copy. And how do devices hear about changes? A long-lived connection to a notification service, then a fetch of everything since their last version.
The common mistake is treating files as single objects: re-uploading the whole thing on every save, or storing file contents in the database. Blocks and hashes are what make sync fast and cheap.
Quick check
You fix a typo in a 200 MB file, changing one 4 MB block. How much is uploaded?
Show the answer
Just that one block, about 4 MB, plus a small metadata commit.
Two laptops edit the same file offline, then reconnect. What happens to the second commit?
Show the answer
Both versions are kept: the second becomes a conflicted copy.
Remember: files are lists of hashed blocks. Only new blocks travel, and identical ones are stored once. And metadata stays strongly consistent, while conflicts are kept as copies, never guessed at. Tomorrow, a system that reads the whole web.
Lessons on weekdays, the week's revision on Saturday, a mock interview on Sunday.