Sharding strategies

The day you start thinking about sharding is the day you've already exhausted the easier moves. You've added read replicas, you've put a cache in front of the hottest reads, you've vertically scaled until the bill from your cloud provider raised an eyebrow somewhere. The write workload is still climbing, the single primary is still groaning, and somebody on the team has just used the word "shard" out loud for the first time. The room gets a little quieter.

This is the right reaction. Sharding is the heaviest hammer in the toolbox, and not because it's hard to implement — hash(key) % N is twelve characters. It's hard because sharding changes what your data model is allowed to do. Cross-table joins become cross-network fan-outs. Foreign keys stop being enforceable. Transactions stop being atomic across the things you used to atomically transact. Every report that used to be a GROUP BY becomes a small distributed system that has to merge partial results from N machines.

So before you shard, you owe it to your future self to be honest about whether you've genuinely run out of easier options. A modern Postgres on a machine with 256 GB of RAM and NVMe storage will hold hundreds of millions of rows and serve tens of thousands of writes per second. Buying a bigger box is operationally trivial compared with splitting your data, and most teams that "needed to shard" in 2015 don't need to in 2025 because the hardware caught up.

Do you even need this? The back-of-envelope

Before the room agrees to shard, write the math that proves one box can't hold the year. Say you run a system that's growing toward 40,000 writes/second sustained and 8 TB of hot data:

The RAM line is the one that actually forces your hand here, not the write ceiling — the working set blew past cache long before writes maxed out, so reads were already hitting disk. That's the honest trigger. If your own numbers come out at "1.2 shards", you don't have a sharding problem; you have a bigger-box problem. Buy the box.

Picking the shard key

If you do shard, the only decision that matters is the shard key — the column whose value determines which shard a row lives on. Almost every other choice flows from this one. There are three families of strategy, and which family suits you depends on the queries that matter to you.

Range partitioning sorts the key and assigns contiguous ranges to shards. a-h → shard 0, i-p → shard 1, q-z → shard 2. The appeal is that range scans stay local: "all users registered last week" hits one shard. The failure mode is skew. If your traffic follows a Zipfian distribution (and most user-keyed traffic does), range 0 gets hammered while ranges 2 and 3 idle. Time-series databases lean hard on range partitioning because the keyspace is genuinely uniform — every minute gets roughly the same write volume. Most other systems should be skeptical.

Hash partitioning runs the key through a hash and takes the result modulo N. Distribution is uniform regardless of what the keys look like, which is wonderful for write balance and terrible for range scans — any query that touches more than one key now has to fan out to every shard because the hash destroyed locality. This is what Redis Cluster does (CRC16(key) % 16384 slots), what Cassandra does, and what most NoSQL systems default to. Pick this when you mostly do point lookups by primary key and your range scans are rare or batch.

Directory / lookup partitioning keeps a separate metadata store that maps key → shard. The router consults the directory on every request (cached, of course). The freedom this buys you is enormous: you can move any single key to any shard at any time without re-hashing anything. The cost is the extra hop and the operational weight of running yet another database. Used by systems where one key's footprint changes dramatically over time — multi-tenant SaaS where a tenant might grow from 1 GB to 10 TB, or social-graph sharding where a celebrity might suddenly need their own shard.

The thing nobody tells you about resharding

The day you pick a shard key feels like the hard decision. It isn't. The hard decision is what happens when you need to add a shard.

Naive hash modulo is catastrophic here. Going from 4 shards to 5 means hash(key) % 5 is different from hash(key) % 4 for almost every key in your database. You'd have to move roughly 80% of your data to maintain the invariant. Production clusters never do this; they use one of two tricks.

Consistent hashing arranges shards around a virtual ring and assigns each key to the next shard clockwise. Adding a shard only disturbs the keys that fall in its arc — about 1/(N+1) of the total. This is what DynamoDB, Cassandra, and most distributed caches use. The trade is that load distribution gets uneven; production systems mitigate this with virtual nodes (each shard owns many points on the ring), but you'll still see hot shards if your key distribution is skewed.

Virtual buckets (also called slots or vbuckets) precompute a large fixed number of buckets — Redis Cluster uses 16384, Couchbase uses 1024 — and map each bucket to a shard. Adding a shard just means redistributing some buckets, which is conceptually like moving boxes: cheap to coordinate, easy to track. Going from 4 shards to 5 means each existing shard donates roughly 1/5 of its buckets to the new one. Every router knows which bucket lives where; a bucket move is a single metadata update plus the data copy.

The cross-shard query tax

A query that needs data from multiple shards becomes a small distributed system. The router fans the query out, every shard processes its partition in parallel, and the router merges the partial results. The good news is parallelism — N shards do N times the work simultaneously. The bad news is latency: the final answer arrives when the slowest shard finishes, so p99 of the merged query is max(p99 of each shard), not the average.

This is fine for a join across two shards. It's a problem at ten. At a hundred, it's a death sentence — the probability that every single shard responds within your latency budget goes to zero, and you've built a system that's slower than the single-box one it replaced.

Production systems handle this by avoiding cross-shard queries on the hot path. They denormalize so the join lives inside one shard, or they precompute the aggregate in a separate materialized view, or they accept that operational reports are 10× slower than user queries and run them off-peak. The teams that pretend the tax doesn't exist eventually pay it as a customer-facing latency regression.

A war story about picking the wrong key

A team I worked with sharded a job-board database on employer_id. The reasoning was clean: most queries were "list all jobs for this employer", and employer-scoped queries would stay on one shard. They shipped it, traffic grew, and the system was fine for eight months.

Then one of their employers ran a hiring spree and posted forty thousand jobs in a quarter — about 200× more than the next biggest. Suddenly one shard held 35% of the entire dataset. Its disk was 80% full while three of its neighbours sat at 12%. Its query latency was 2-3× higher because the hot indexes wouldn't fit in RAM. They couldn't add a node to take pressure off that shard, because the shard key had already partitioned the keyspace.

The eventual fix was painful: introduce a directory layer, split employer_id ranges by hand so the big employer got their own shard, and accept a two-week migration. The lesson the team internalized is that any shard key is a bet on the key distribution not changing. If the distribution can become skewed, design in escape hatches (directory, or sub-keys, or per-tenant placement) before you need them.

When to walk away

• The dataset fits on a beefy box. Modern Postgres on 256 GB of RAM serves hundreds of millions of rows. Buy hardware before you split.
• Reads dominate. Replicas plus a cache solve most read-heavy problems without a single schema change.
• You don't have a clean shard key. Without one, every query becomes a scatter-gather and you've lost the entire benefit.
• You're sharding for "future scale" the product doesn't have yet. Defer until you can write the back-of-envelope that proves you'll outgrow one box this year, not someday.

[CONCEPT]replication and [CONCEPT]caching-patterns usually exhaust their headroom before sharding is justified. [CONCEPT]indexes-deep-dive is what you should optimize before sharding — a missing index can hide as a scale problem for years.