📁 Design Dropbox

Prerequisites: Design WhatsApp, Data Models | You'll be able to: design the presigned-URL upload/download path and say precisely why file bytes must never traverse your application servers; build resumable uploads, cross-user dedup, and delta sync out of one primitive — the content-addressed chunk; reason about multi-device sync conflicts as a multi-leader replication problem and say what "this file is synced" actually commits to.

🧨 The problem (why this exists)

"Design Dropbox" — or Google Drive, or OneDrive; the brief is a cloud file-storage service: store files, get them back from any device, share them, and keep every device's copy in sync automatically. This is the fifth rep of the delivery framework, and it earns its slot by breaking an assumption the previous reps never questioned. The URL shortener moved a few hundred bytes per request; Ticketmaster and the news feed moved kilobytes; WhatsApp already shunted its media sideways to blob storage as a supporting detail. Here the payload is the product, and a single object can be 50 GB — orders of magnitude past anything a request/response body was designed to carry. That one number invalidates the default architecture: you cannot POST it, you cannot buffer it, you cannot afford to route it through your compute tier at all.

Functional requirements:

1. Users can upload a file from any device.
2. Users can download a file from any device.
3. Users can share a file with other users and see files shared with them.
4. Files sync automatically across a user's devices.

Below the line: editing files, viewing files without downloading them. This design also scopes out designing blob storage itself — we consume S3-style storage as a building block; its internals are a different interview.

Non-functional requirements — quantified:

1. Highly available — explicitly prioritizing availability over consistency, per the non-functional requirements discipline of naming the CAP stance. A useful calibration: a stock trade must be read-consistent across continents; a file appearing in Germany seconds before it's visible in the US harms nobody.
2. Files as large as 50 GB.
3. Secure and recoverable — lost or corrupted files can be restored.
4. Upload, download, and sync as fast as possible — low latency on all three.

Below the line: per-user storage limits, file versioning, virus scanning.

Two of these NFRs are quietly at war. "As fast as possible" pushes bytes toward the shortest path between client and storage; "recoverable, from any device, in sync" demands bookkeeping that knows — durably, transactionally — which bytes exist and which version is current. The design's central move is to split the system along exactly that line and never let the halves blur: a data plane that moves bytes, a metadata plane that moves facts about bytes.

💡 Intuition first

Build it the obvious way first, because every wrong instinct in this interview is the same instinct: treat a file like any other request payload. POST /files with the bytes in the body; the File Service writes them to local disk and records a row in its database. Download streams them back. This works in a demo, and it dies precisely as follows.

It dies three deaths. First, the box fills up — files accumulate forever, and sharding them across app servers by hand reinvents, badly, the distributed storage layer blob stores already are. Second, the box is a single point of loss — NFR 3 says recoverable; one failed drive says otherwise. Third — and this survives even after you swap local disk for S3 — the bytes traverse your compute tier twice. Client to server, server to blob storage: every gigabyte crosses the network twice and holds an application thread for the duration. The arithmetic that makes this fatal rather than wasteful: a 50 GB file on a 100 Mbps uplink is 50 × 8 / 0.1 Gbps = 4,000 seconds — about 1.1 hours — for a single request. Nothing in the chain tolerates that: default request-body limits are under 2 GB on Apache and NGINX and as low as 10 MB on managed API gateways, and an hour-long request is a timeout somewhere with probability approaching 1. One dropped packet at minute 59 and the user starts over.

So the naive design's failure teaches the two constraints the real design is built on: file bytes must flow directly between the client and blob storage, with your servers handling only coordination — and a 50 GB transfer must be divisible, so that progress survives interruption. Everything in the deep dives is the working-out of those two sentences.

⚙️ How it works

🧱 Core entities: the split is the design

Three entities anchor this design — File, FileMetadata, User — and the whole architecture hides in the first comma. This lesson promotes a fourth, elsewhere treated as a mere field:

• File — the raw bytes. Lives in blob storage, and only there. The application never holds or parses it; from the data-models standpoint it's not a record at all — an opaque value the rest of the system refers to by name.
• FileMetadata — the facts about the bytes: name, size, MIME type, owner, status (uploading / uploaded), and the chunk manifest — the ordered list of chunks constituting the file, each with its fingerprint and upload status. This row is the file's identity as far as the system is concerned; the bytes are its shadow.
• Chunk — a 5–10 MB slice of a file, identified by the hash of its content (a SHA-256 fingerprint). The unit of transfer, resume, dedup, and delta sync — one primitive, four features.
• User — plus a SharedFiles mapping (userId → fileId) so "files shared with me" is an indexed lookup, not a scan of every file's share list. The path from share-list-in-metadata → inverse cache → this normalized table is a straightforward indexing argument whose ending we adopt.

The metadata store is a document-shaped, low-relational workload — DynamoDB is a natural reach, though PostgreSQL would do fine too; per storage engines, nothing here turns on that choice, and saying so is worth more than the pick.

🔌 The API — and why it refuses to carry the file

Naively (the starting point, per API design): POST /files with the bytes, GET /files/{fileId} returning them, a share endpoint, a changes endpoint for sync. The final API keeps the shapes but evicts the bytes — every file-carrying endpoint becomes a credential-issuing endpoint:

POST /files/presigned-url    { fileFingerprint, chunkFingerprints[], name, size, mimeType }
                             → { fileId, uploadUrls: presigned PUT per *missing* chunk }
GET  /files/{fileId}/presigned-url
                             → { downloadUrl }            // presigned GET, or signed CDN URL
POST /files/{fileId}/share   { targetUserEmail }
GET  /files/changes?since={cursor}
                             → [ changed FileMetadata ]   // the sync feed

A presigned URL is a time-limited, signed grant: the File Service asks blob storage to mint a URL authorizing exactly one operation — PUT this object, or GET that one — for a few minutes, and hands it to the client, which then talks to blob storage directly. Authorization stays server-side (no URL gets minted for a file the metadata store says you can't touch); the bytes take the short path. User identity rides in headers (session token / JWT), never in bodies.

🗺️ High-level architecture

Two planes, drawn as one picture. Metadata operations — fingerprints, manifests, shares, the change feed — flow through the File Service to the metadata DB. Bytes flow client ↔ blob storage (uploads and cold downloads) or client ↔ CDN (hot downloads), on presigned/signed URLs. The dashed lines are the point: the heavy arrows never touch the green box.

The sync agent on each device closes the loop: it watches the local folder via OS file-system events (FSEvents on macOS, FileSystemWatcher on Windows) and on a change pushes local → remote through the upload path. In the other direction, devices learn of changes over a hybrid channel: a WebSocket for fresh files (recently edited, near-real-time sync matters) and cheap periodic polling of /files/changes for stale ones — the remote metadata store acting throughout as the source of truth both directions converge on.

🤿 Deep dives

🚚 Moving big files — why bytes never traverse your app servers

This is the ladder for what's often tagged the handling-large-blobs pattern; each rung fails in a way that names the constraint the next rung satisfies.

Rung 1 — file through the app server, onto local disk. Dies as described in the intuition: disk ceiling, single point of loss, double-transfer tax. Broken baseline.

Rung 2 — file through the app server, into blob storage. Blob storage genuinely solves storage: effectively unlimited capacity, redundancy that makes NFR 3's "recoverable" someone else's solved problem, lifecycle policies for free. But the bytes still cross your tier twice, and a subtler wound opens: upload and metadata write are two operations against two systems, and either can succeed while the other fails — bytes with no record, or a record with no bytes. The prescription — commit metadata only once the upload is confirmed — becomes deep dive 2's manifest-commit discipline.

Rung 3 — presigned URLs: the client uploads directly (the chosen answer). Three steps: the client asks the File Service for an upload grant; the service writes a FileMetadata row with status uploading and returns the presigned URL; the client PUTs the file straight to blob storage. Then the crucial coda — blob storage, not the client, reports completion: an S3 event notification fires on upload and the backend flips the status to uploaded. Downloads mirror it: GET /files/{fileId}/presigned-url checks SharedFiles for permission, then returns a short-lived presigned GET.

Rung 4 — CDN for hot downloads. Blob storage lives in a region; users don't. Add a CDN so a frequently fetched file is served from an edge node near the requester, security carried over — signed CDN URLs with a short expiry (~5 minutes), validated at the edge. CDNs bill for what they cache, so cache strategically: cache-control headers and invalidation keep only genuinely hot files at the edge. Contrast WhatsApp, which declined a CDN for media — an attachment has at most 100 recipients, so edge caching buys nothing. Same pattern, opposite verdict; the deciding variable is read fan-out — the sharpest one-line demonstration of trade-off thinking this design offers.

Notice what the File Service has become at the ladder's top: it never touches a byte of file content — it checks permissions, mints grants, and records facts, which is what lets it stay small, stateless, and horizontally boring while the data plane carries the tonnage. In networking terms: your servers stay in the control path and exit the data path — every large-blob system you'll ever design (video upload, ML artifacts, photo backup) re-instantiates this split. One honest boundary: your servers now see content only after it lands (if ever), so validation, scanning, or transcoding becomes an asynchronous post-upload step driven by the same event notifications, not an inline gate. Name the consequence rather than discovering it.

🧩 Chunking, dedup & delta sync — the content-addressed chunk

A 50 GB object is not just too big to proxy — it's too big to treat as atomic. A presigned single PUT still takes 1.1 hours on a 100 Mbps link, still exceeds request-size limits, and still restarts from zero on interruption. So the client — and it must be the client — splits the file into 5–10 MB chunks before anything is transferred. Chunking on the server is the classic trap: the whole file would have to reach the server first, which was the problem. Chunks upload in parallel (saturating the client's bandwidth), each is cheap to retry, and a progress bar falls out for free — chunks done over chunks total.

Now the idea that turns chunking from a transport trick into the design's second pillar: identify every chunk — and the file itself — by a hash of its content (a SHA-256 fingerprint; the file's fingerprint becomes its fileId). Names lie — two users' report.pdf collide; a renamed file changes name while the bytes didn't. Content hashes tell the truth in both directions: same bytes, same identity, regardless of name, owner, or device. Once identity is content, three features collapse into one lookup against the chunk manifest:

• Resume. An interrupted upload is just a manifest with some chunks marked uploaded and some not. On retry the client sends its fingerprint list, the server answers with what's missing, and only those chunks move. Nothing was "saved"; the manifest is the resume state.
• Dedup. If any user ever uploaded a chunk with this hash, the bytes are already in blob storage — record the reference, skip the transfer. The second person to upload a popular 2 GB installer uploads approximately nothing. (Cross-user dedup has real costs — see In production.)
• Delta sync. Edit one region of a 50 GB file and re-hash: unchanged chunks produce unchanged fingerprints, so only changed chunks upload. A one-chunk edit moves 5 MB instead of 50 GB — 10,000× less, from the same mechanism that gave you resume.

The upload protocol, end to end — and note who is trusted to say what:

Steps 7–9 are the trust boundary. The obvious alternative — the client PATCHes the manifest after each successful chunk PUT — is the one this design rejects: a client that reports upload status can lie, and a lying (or merely buggy) client leaves the manifest claiming chunks blob storage doesn't hold — corrupt state that surfaces later as a failed download. S3 event notifications make blob storage itself the witness: a chunk is uploaded when the store says it arrived, and the client is trusted with nothing but bytes.

The final commit — step 10 — gives rung 2's "transactional care" its precise form. A file becomes visible — downloadable, shareable, sync-announced — in exactly one place: the metadata transaction that flips its status once every chunk in the manifest is confirmed. Until that flip, the bytes are durable but invisible; an upload abandoned forever is orphaned garbage to collect, never a half-file a user can download. The flip is the file's linearization point — everything upstream of it may fail, repeat, or arrive out of order. Same shape as WhatsApp's "sent means the transaction committed": the user-visible promise pinned to a single durable write.

Two footnotes at the expert edge. First, chunk size is a genuine trade-off — the Trade-offs table works it through; 5–10 MB is the number used here, adaptive sizing the refinement. Second, fixed-boundary chunking has a weakness this design inherits: insert bytes near the front of a file and every subsequent boundary shifts, every downstream hash changes, and delta sync degenerates to a full re-upload. Rule of thumb, not from source: production sync engines use content-defined chunking — rolling hashes place boundaries at content landmarks so an insertion only disturbs its neighborhood (the rsync lineage); neither source covers it, so in an interview it's a flagged "beyond scope, here's the name." Finally, the productized analog: S3 Multipart Upload is this protocol — client-side parts, parallel PUTs, server-side assembly — and the advice is to mention it and be able to rebuild it from parts, since "I'd call the API" answers nothing about what the API does.

🔄 Sync & conflicts across devices — the metadata store as the coordination point

The first two deep dives moved one file from one device. Requirement 4 is stranger than it looks: the same user's laptop, phone, and desktop each hold a local copy of the folder, each accepts edits while offline, and all are supposed to converge. Stop and name what that is. Each device is a full replica accepting writes without coordinating with anyone — multi-leader replication, exactly. DDIA's sync-engines treatment says an offline-capable app with a local replica per device is multi-leader at an extreme: each device is in effect a "region" with an extremely unreliable link, and replication lag isn't milliseconds but hours or days — however long the laptop stays in the bag [ch. 6 p. 220]. The client-side sync agent is what DDIA calls a sync engine — capture local changes, queue offline, ship when connected, merge remote changes in [ch. 6 p. 221]. Dropbox is the textbook's happy case: sync engines work best when each replica's data can be fully downloaded and held locally — fine for a user's own files, hopeless for an entire product catalog [ch. 6 p. 222] — and it collects the local-first payoff: reads and writes hit local disk first, the UI never blocks on a round-trip, offline isn't a special mode, and local operations skip the per-call error handling RPCs need [ch. 6 pp. 221–222].

The topology, though, is deliberately not peer-to-peer. Devices never reconcile with each other; every device syncs against the remote metadata store, designated the source of truth — commit local changes there fast, and let every other device converge on it. That star-through-the-hub shape is what keeps the system comprehensible: "current version of file X" has exactly one authoritative answer, the metadata row, guarded by deep dive 2's manifest-commit discipline. Devices are leaders for accepting writes; the hub is where write order gets decided.

Then two devices edit the same file while both are offline, and no topology can save you — you must choose a conflict policy. Precision first, because interviewers probe exactly here: the edits conflict not because they happened at the same wall-clock moment but because they're concurrent in the causal sense — each made in ignorance of the other; neither happens-before the other [ch. 6 pp. 222, 238–239]. A device offline for a week can conflict with an edit made six days ago. Clocks don't define this; knowledge does.

This design resolves it with last write wins: both versions reach the server; the later-timestamped upload becomes current. And here the framing and DDIA genuinely part ways, so we surface both. DDIA's verdict on LWW is unsparing: among concurrent writes "last" is undefined, so LWW means one write is picked as winner and the others silently discarded — convergent, at the price of data loss; and with wall-clock timestamps, skew can crown a genuinely earlier write [ch. 6 pp. 224–225]. For a chat app deciding message order, dropping one ordering is invisible. For a file store, the discarded loser is someone's day of work. The correctness-preserving alternative is DDIA's sibling pattern: on detected concurrency, keep both copies and surface them for a human (or the application) to merge — CouchDB's approach [ch. 6 p. 225], recognizably the "conflicted copy" sync products drop next to the original. Note what's not on the menu: automatic merging — CRDTs and operational transformation converge concurrent edits losslessly [ch. 6 pp. 226–228], but only because they understand the data's structure, and this design's File is an opaque blob by construction; type-aware merging is the road to Google Docs, a different product and a different interview. Ruling: LWW is the accepted answer in the interview's main line, and even that framing gestures at the safety net (versioning: don't overwrite; add a version and move the pointer — scoped below the line). But say the DDIA-grounded sentence out loud: LWW silently discards a concurrent write; with versioning or conflicted copies the loser stays recoverable. Knowing what your policy destroys is the difference between choosing a trade-off and stumbling into one.

Now the expert layer worth pushing past: what does "this file is synced" even mean? The green checkmark is a compound claim whose parts commit at different moments:

1. Durable — every chunk of the new version is confirmed in blob storage (S3 events; all manifest entries uploaded). The bytes can no longer be lost. Not yet visible.
2. Visible — the manifest transaction committed; "current version" changed. This instant, and no other, is when the upload happened system-wide: shares, downloads, and the change feed all read this row.
3. Propagated — some other device heard the change event and pulled the chunks. Per-device, unbounded in time (that laptop in the bag), never guaranteed for all devices at once.

The checkmark on the uploading device honestly means 1 + 2 — the source of truth has it; it cannot mean 3, which isn't a single fact and never completes for a device that stays offline. WhatsApp pinned each tick to a machine-checkable event; this is the same honesty with different events, and the manifest commit is the hinge — flip visibility before durability is total and a device can be told to download chunks that don't exist. Order of operations is the correctness argument, and "walk me through what each state of the status field promises" is how a strong candidate closes this deep dive.

The whole design as a walkthrough — three boards rather than one picture: the system in context, its containers, and the code level inside the sync agent. Any box carrying a link badge drills down a level; the ◀ ▶ ⌂ controls and the board menu walk back out.

🛠️ Hands-on: run this design

A runnable implementation lives at _proof-of-concepts/07-case-studies/05-dropbox/ in the repo root — the sync agent (Chunker, ManifestDiffer, SyncEngine, mirroring the code view) against a FastAPI file service backed by a content-addressed chunk store in Postgres.

cd _proof-of-concepts/07-case-studies/05-dropbox
./run            # build + start the file service (8360) + Postgres (8361)
./run test       # mypy --strict + pure agent tests + HTTP two-device sync
./run stop

./run test makes content-addressing pay off: device A pushes a file, device B pulls an identical copy from the chunk store; editing only the file's tail re-uploads just the changed chunk (the first chunk, unchanged, is reused); and pushing the same content at a new path uploads zero chunks — dedup, because the name is the hash.

🧱 Component reference

⚖️ Trade-offs

The three decisions this design turns on, in the thinking-in-tradeoffs frame:

Conflict policy, per deep dive 3:

Chunk size — small vs large, at fixed file size:

The 5–10 MB choice sits deliberately toward the small end — resumability and delta precision are the NFRs that bite — with adaptive sizing by network conditions as the refinement.

🔢 Numbers that matter

Every figure here ends in a decision, per the estimation discipline:

🏭 In production

Operational reality for this design's shape — sourcing flagged; none of it claims to describe Dropbox-the-company's internals.

Incomplete uploads are a garbage stream, not an event. Clients start uploads and vanish — laptop lids close mid-chunk, forever. Each abandonment leaves durable-but-invisible chunks (deep dive 2's safe failure mode) and an uploading manifest row: a steady leak of paid storage at fleet scale. Reap it — expire stale in-progress manifests and delete chunks no manifest references. Blob-storage lifecycle policies help here; S3 specifically can auto-abort incomplete multipart uploads after N days [web: AWS S3 lifecycle documentation]. The subtle bug class is reaping a chunk a new upload just claimed via dedup — reference-counting chunks across manifests is the real discipline, and where content-addressing collects its operational tax (rule of thumb, not from source).

Dedup ratios are workload-dependent — and dedup fights encryption. Cross-user dedup shines on mass-shared content (installers, media, the forwarded PDF) and does little for personal photos, unique by nature; neither source quantifies ratios, so treat any figure as a rule of thumb, not from source. The structural conflict is worth naming: the compress-before-encrypt point rests on encryption destroying redundancy, and the same fact means client-side encryption with per-user keys makes identical files hash differently — end-to-end-encrypted storage and cross-user dedup are directly at odds.

Tier cold bytes. File storage is write-once, read-maybe. DDIA notes the economics — object storage optimized for infrequent access is the cheap home for old data [ch. 6 digest, p. 197]; blob stores productize this as storage classes trading retrieval latency for cost, and demoting long-unread chunks is standard practice — tiering specifics are a rule of thumb, not from source.

Watch sync lag the way WhatsApp watches delivery lag. The user-perceived NFR is "my other device has it fast," and "sync is slow" has four different owners — instrument chunk-upload throughput, S3-event-to-manifest-mark latency, manifest-commit-to-change-event latency, and change-event-to-peer-download completion separately. The signature failure is a manifest stuck at 9,999/10,000: one lost event notification leaves a file forever almost-visible, so run a reconciliation sweep that re-lists blob storage against uploading manifests — measure the backstop to see the losses (rule of thumb, not from source; the same logic as WhatsApp's inbox sweep).

Conflicted copies accumulate. The sibling policy trades silent loss for visible clutter: users who edit on two devices generate conflicted copies faster than they merge them. Still the right side of the trade for user files — but plan the UX, and expect "why are there four copies of my thesis" tickets (rule of thumb, not from source).

🪤 Pitfalls & interview traps

Trusting the client's word for upload state. The client-orchestrated PATCH ("I uploaded chunk 7, promise") hands your source of truth to the least trustworthy machine in the system; a malicious or buggy client manufactures manifests claiming bytes blob storage never saw. "Who says a chunk is uploaded?" has exactly one right answer, and it isn't the uploader.

Flipping visibility before durability is total. Mark the file uploaded when the client says it finished — or worse, at presign time — and a sync peer can be told to fetch chunks that don't exist. The flip fires only when every manifest entry is storage-confirmed; everything before it must be repeatable and invisible. "What exactly does your status field promise, and when?" is the metadata-honesty probe.

Saying "last write wins" like it's free. LWW is the accepted interview answer and it silently discards one of two concurrent versions — a fact you're expected to volunteer, with the mitigation ladder (versioning below the line; conflicted-copy siblings). Trap inside the trap: "last" by wall-clock timestamp inherits every clock-skew pathology — a fast-clocked device wins conflicts it shouldn't [ch. 6 pp. 224–225].

Treating sync as a feature instead of a replication topology. Candidates bolt "and it syncs" onto a client-server design and never notice they've built multi-leader replication — every device an offline-capable leader, lag measured in days, conflicts guaranteed by construction [ch. 6 p. 220]. Name the topology and the follow-ups — why star-through-the-hub, why conflicts are causal not temporal, why CRDTs don't apply to opaque blobs — become questions you've already answered.

The leveling bar. Mid-level: clean API and entities, a working high-level design for upload/download/share; presigned URLs and chunking may need prompting, but you can reason to them when pushed ("you're uploading the file twice — how do we avoid that?"). Senior: through the high-level design fast, because the large-file deep dive is where you plan to live — chunking, fingerprints, resumability, trade-offs argued proactively. Staff+: hands-on fluency (multipart upload by name and by mechanism) and depth where the breakdown goes quiet — the trust boundary, the commit discipline, what sync really is. The gradient is the same as every rep: who drives, and how deep the "why" goes.

✅ Check yourself

🔬 PoC — Proof of concepts

Run it yourself. Dropbox — content-addressed sync — chunk a file, address each chunk by its hash, and sync only the chunks that changed; the deduplication and delta-sync at the heart of the design. From _proof-of-concepts/07-case-studies/05-dropbox/, run ./run.

Study real implementations.

• restic — content-defined chunking and content-addressed, deduplicated storage in a readable Go codebase; exactly this POC's core, production-grade.
• MinIO — the S3-compatible blob store the chunks land in; the durable object tier every file-sync service needs.
• SeaweedFS — an alternative store tuned for billions of small objects (chunks are small and numerous), with cloud tiering.

📚 Sources

• DDIA2 ch. 6 pp. 220–228 (sync engines, local-first, conflicts) — offline devices as multi-leader replicas with hours–days lag, each a "region" [p. 220]; the sync engine defined, offline-first/local-first distinguished [p. 221]; local-read/write advantages and the download-in-advance fit ("a user's own files") [pp. 221–222]; concurrency as mutual unawareness [p. 222]; LWW's real meaning — a concurrent write silently discarded, clock-skew sensitivity [pp. 224–225]; siblings / keep-both (CouchDB) [p. 225]; CRDTs and OT and why they need structured data [pp. 226–228].
• DDIA2 ch. 6 pp. 238–239 (happens-before) — the causal definition of concurrent writes backing the conflict discussion.
• Flagged inline: content-defined chunking / rolling hashes (beyond both sources); S3 incomplete-multipart lifecycle specifics [web: AWS S3 lifecycle documentation]; dedup ratios, chunk reference-counting, cold-tiering specifics, sync-pipeline monitoring, and conflicted-copy UX as rules of thumb; the dedup-vs-encryption tension derived from the compression/encryption discussion above.