One app · nine stages · 50 users to 50 million
Every box in a system diagram is there because something broke. Step through the build and watch each one break: the load balancer, the session store, the index, the cache, the replicas, the shards — each arriving the moment the app can no longer live without it.
Nine stages, each triggered by a failure
Nobody asks for a cache. Users ask for a fast app. Read each stage as a chain: something breaks, you add a component, and the component hands you a new problem. The rail is a real sequence — you cannot get to shards before you have run out of one machine.
← → to step
—
—
—
—
Where the milliseconds actually go
The finished topology, one request at a time. Reads and writes take different paths on purpose: the feed is optimised down to a single millisecond of memory lookup, while the upload is allowed to take a second and a half because uploading feels like work.
Stage 08 topology
Pick a scenario and run it.
Hit, miss, evict, go stale
Four slots, eight photos, least-recently-used eviction. Request photos by hand or fire a realistic burst where a handful of celebrity posts soak up most of the traffic. Then edit a cached bio and watch the cache confidently serve the old one — the failure mode that is far worse than an empty cache.
Cache slots
Request a photo
Tape
Consistency drill
The database and the cache now hold the same row twice. Sooner or later they disagree.
| Data | Policy | Why |
|---|---|---|
| Home feed | TTL 30 s | A few seconds of staleness is invisible while scrolling. |
| Trending list | TTL 60 s | Recomputed constantly, wrong by a rank nobody notices. |
| Profile card | TTL minutes | Small, read constantly, changes almost never. |
| Follower count | TTL 60 s | 41 instead of 45 for a minute costs nothing. |
| Privacy & permissions | no TTL — invalidate | Stale here means showing private photos to the wrong person. |
Why adding one machine can move all your data
Thirty-six user IDs spread across shards. With plain user_id % N, adding one shard rewrites the rule for almost everybody — billions of rows would have to physically move. Switch the strategy and add a shard again: consistent hashing takes a slice from one neighbour and leaves the rest alone.
Add or remove a shard to see how much data has to move.
Keys · shard assignment
Load per shard
Altitude, requirements, shape
High-level design
Big pieces and the path between them: app servers, database, cache, queue, CDN. Where does a request go, where does data live, what breaks first when traffic grows.
Graded on: data flow, component choice, and whether you can name the cost of each choice.
Low-level design
One feature, in code. What happens on tap, which function handles it, how you avoid counting the same user twice, which classes and data structures hold it.
Graded on: objects, data structures, edge cases. No servers required.
“I’ll stay at the high level — components, data flow and trade-offs — and we can zoom into any one of them.” Interviewers rarely stop you when you answer at the wrong height; they just let you spend twenty minutes there. Mismatched altitude is one of the most common ways strong candidates lose.
The five questions · ask these before you design anything
0 / 8
A feature you can point at and test is functional. A quality everybody feels but nobody can point at — speed, durability, scale, availability, cost — is non-functional. Two apps with identical feature lists can be completely different systems.
Sign-up, upload, feed, likes and comments all live in one program and call each other with plain function calls. No network in the middle, nothing in the middle to fail.
You cannot scale one part on its own. The feed needs ten servers and uploads need two, but you scale the whole program to ten either way — and you pay for eight servers of upload capacity nobody asked for.
Change the sign-up page and you redeploy everything, including the feed.
There are two honest reasons, and neither is fashion.
Every function call becomes a network call that can be slow, time out, or fail halfway. One user request may touch five services, so debugging means reading five logs and understanding how they interact.
Most companies you will join run a monolith plus a handful of services that were split off — not one giant program, and not three hundred tiny ones. A five-person startup running thirty services spends more time debugging the network than building the product.
| Vertical — a bigger box | Horizontal — more boxes | |
|---|---|---|
| Effort | Zero code changes. Pay more, reboot. | Needs a load balancer and stateless servers. |
| Ceiling | Hard. The biggest machine on the market is the end. | None in practice. Add the eleventh machine. |
| Price curve | Small → medium is cheap. Second-biggest → biggest can double the bill for 20% more power. | Roughly linear in cheap machines. |
| Failure | One big server is still one server. It dies, you are down. | One of ten dies, nine keep serving. |
| Right for | A 50-user internal tool. Buying yourself another year. | A consumer app heading for a million users. |
Scale up first because it is simple; scale out when you hit the ceiling or can no longer afford a single point of failure. Most real systems end up doing both.
Seven prompts · tap to see the answer that scores
The pattern in every one of these: name the cause, give the fix, then volunteer the wrong answer and why it is wrong. Bringing up the failure before the interviewer asks is what separates “knows the feature” from “has thought about it”.
Break it yourself
The four hands-on labs from the course, each one a failure you cause on purpose. Links live in the source video’s description.
Run one app in two containers behind nginx, push a pile of requests through, then shut one container down while it is still busy and watch the balancer route around the corpse.
Fill a table with 5M rows, run a query, watch it crawl. Work out why, add a single index, run it again — milliseconds.
Put Redis in front of the database, measure before and after, then change the row underneath and watch your app confidently hand out the old value.
Set up a primary and a replica, write a row and read it from the copy a beat too early. Then kill the primary, promote the replica — and watch the replica dutifully copy your delete.
Say the word, then say why