Thinking in systems

The first time I was asked to "architect" something at work I drew six boxes on a whiteboard, connected them with arrows, and immediately got stuck. I knew the names of the boxes — API gateway, microservice, queue, database — but I didn't actually know what to do with them. I was arranging vocabulary, not solving a problem.

What I'd missed is that systems design isn't about boxes. It's about four questions you ask before you ever pick up a marker. They feel trivial the first time you read them and then take five years of production scars before you ask them automatically. So we'll start there.

The four questions

Open a fresh whiteboard for any new feature and, before you reach for a single box, answer these out loud.

Who sends the traffic? Not "users" — that's a non-answer. Twenty thousand mobile clients hitting the API fifty times a day is a very different problem from a single nightly cron job doing one batch import. The answer drives every capacity calculation and every auth decision downstream. Get it specific or you'll over-build and ship late.

What is the actual logic? Reads dominate writes by a hundred to one in social products and ten to one in banking. A timeline feed is nothing like a payment transfer; they happen to use the same word "request" but everything about their shape — cacheability, idempotency, latency budget — is different. Name the verb. Then name what makes this verb hard.

Where does state live? "The database" is the answer of someone who has not yet been on call at 3am. State is plural. There is a source-of-truth (probably Postgres or DynamoDB), there is a cache in front of it (Redis), there are blobs in object storage (S3), there are queues holding deferred work, there is browser localStorage. Each one has its own consistency story, its own failure mode, and its own cost. Naming them up front is half the design.

What can fail? This is the question most teams answer with "we have HA" and then discover, the first time an availability zone hiccups, that "HA" was an aspiration rather than a property. The real list — primary crashes, slow dependency, partial network partition, deploy that ships a bug at 2pm Friday — fits on one whiteboard and should sit next to your design from day one.

If you can answer those four for any feature, you can build it. If you can't, you're guessing, and the system will eventually return the guess as an outage.

A working definition

A system is a set of components that cooperate to move data from a user's intent to a durable result. Every failure you'll ever see is two of those components disagreeing about the path the data should take. That's it. That's the entire job.

The five-layer skeleton

If you stare at enough production architectures you'll notice the same five layers, in the same order, almost every time. New systems are mostly variations on this skeleton:

Layer	What it does	Typical tech
Edge	Terminates TLS, blocks junk, throttles per-IP, caches static	CloudFront, Fastly, Cloudflare, Envoy
Compute	Holds the stateless business logic	App servers, Lambda, workers
State	Source of truth that survives a restart	Postgres, DynamoDB, S3
Cache	Absorbs reads so state isn't the bottleneck	Redis, Memcached, CDN
Pipeline	Carries async work between services	Kafka, SQS, RabbitMQ

When you start a new design, fill in those five slots before you do anything else. The interesting trade-offs surface immediately — "we need cross-region consistency" lights up the state row in red, and suddenly you're talking about Spanner instead of Postgres.

You haven't designed a system until you can explain what happens when each layer goes down. "It works on the happy path" is not a design — it's a demo.

Putting numbers on it before you draw

The four questions become real the moment you attach arithmetic to them. You don't need precision. You need an order of magnitude, because order of magnitude is what decides the architecture.

Say the feature is a notifications feed. The first question — who sends the traffic — gives you: 5 million daily active users, each opening the app and refreshing the feed about 20 times a day. That's all you need to start.

Quantity	Math	Result
Feed reads per day	5M × 20	100M reads/day
Average reads per second	100M ÷ 86,400	~1,160 RPS
Peak (×3 over average)	1,160 × 3	~3,500 RPS
Writes (a notification per user per day)	5M ÷ 86,400	~58 WPS

The read

ratio falls out immediately — roughly 2,000 reads for every write. That single number answers half the design before you've drawn a box. A 2,000 read-heavy workload screams for a cache in front of state, and the writes are so light that the source of truth barely notices them. Now the skeleton is filling itself in: edge for static assets, a Redis cache absorbing the 3,500 peak RPS, Postgres handling 58 writes a second in its sleep.

Get this arithmetic wrong by an order of magnitude and you build the wrong system. A team that assumes "a few hundred RPS" provisions one database and skips the cache; the real 3,500 RPS arrives and the database — designed for the writes, never the reads — falls over at launch.

A war story: the layer nobody owned

A payments team shipped a "recent transactions" panel. They answered three of the four questions well. Who sends the traffic: every logged-in user, on every dashboard load. What's the logic: read the last 50 transactions. Where does state live: Postgres, the system of record.

They never seriously answered the fourth — what can fail — for the dependency between layers. The panel read straight from the primary Postgres on every dashboard load. No cache, because "it's just a small query." For months it was fine.

Then a marketing email went out at 9am and dashboard traffic spiked 8x in ten minutes. The recent-transactions query — unindexed on the exact sort it needed — went from 4ms to 90ms under load, then queued. Postgres connection pool saturated. And because the same primary served the actual payment writes, those started timing out too. A read panel nobody considered load-bearing took down the ability to take money.

The post-mortem cost wasn't the fix — adding an index and a 30-second Redis cache took an afternoon. The cost was 40 minutes of failed payments during the highest-traffic window of the quarter, and the uncomfortable realization that the team had a five-layer system with the cache layer simply left blank. Nobody had drawn the box, so nobody owned what happened when the read path and the write path fought over the same database.

What can fail, made concrete

"What can fail" is the question that separates a design from a demo, so it's worth turning into a checklist you run against every box you draw. For each layer, ask: what happens when it's gone, and what happens when it's slow? Slow is usually worse than gone, because a dead dependency fails fast and a slow one ties up every caller waiting on it.

Layer down	Symptom	Common defense
Edge	TLS errors, no static assets, no rate limiting	Multi-POP CDN, fail-open for cached content
Compute	5xx everywhere	Multiple instances, health checks, autoscale
State (primary)	Writes fail, reads may fail	Replica promotion, [CONCEPT]replication
Cache	Reads stampede the database	[CONCEPT]caching-patterns, singleflight
Pipeline	Async work backs up or is lost	Durable queue, dead-letter, replay

The row that surprises teams most is the cache one. A cache that goes down doesn't fail quietly — it hands its entire read load to the database that was never sized for it, all at once. Designing for "cache down" is designing for a sudden 10x on your source of truth.

Why this lesson sits at the front

Everything else in this book — load balancers, caches, consensus, queues — is a concrete answer to one of the four questions. Caching patterns answer "where does state live?". Consensus answers "what can fail?". The reason it's worth reading the lessons in order is that each one only makes sense once you've decided what slot in the skeleton it's filling. Skip this lesson and you'll memorize patterns without knowing when to reach for them.

Start with [CONCEPT]back-of-envelope once you can name the four; it teaches the arithmetic that turns them into numbers. Then [CONCEPT]load-balancing and [CONCEPT]caching-patterns are the first concrete moves almost every system makes.