Redis Cluster - Native Sharding with Hash Slots

Overview

Redis Cluster is the native distributed mode of Redis. It automatically partitions data across multiple master nodes using 16384 hash slots. Unlike Redis Sentinel (which is HA-only), Redis Cluster provides both sharding AND built-in failover -- no external process needed.

Architecture

Smart Client

Redis Cluster uses "smart clients" (redis-py, jedis, ioredis) that maintain a local cache of the hash slot to node mapping. The client computes CRC16(key) % 16384 to determine which master owns a key, then sends the command directly. No proxy is needed.

3 Shards (6 nodes total)

• Shard 1: Master 1 (slots 0-5460) + Replica 1
• Shard 2: Master 2 (slots 5461-10922) + Replica 2
• Shard 3: Master 3 (slots 10923-16383) + Replica 3

Each master owns roughly 1/3 of the 16384 hash slots.

Cluster Bus (Gossip Protocol)

Every Redis Cluster node opens a second TCP port (main port + 10000, e.g. 16379) for node-to-node communication. Nodes exchange PING/PONG messages carrying cluster metadata, failure reports, and configuration updates.

Key Concepts

Hash Slot Routing

slot = CRC16(key) % 16384

The client computes this locally and routes directly. On first connect, the client runs CLUSTER SLOTS to get the full mapping.

MOVED vs ASK Redirects

MOVED = permanent reassignment. ASK = temporary during migration.

Failure Detection: PFAIL vs FAIL

Similar to Sentinel's SDOWN/ODOWN but built into the cluster:

• PFAIL (Probable Failure): One node's subjective view (like SDOWN)
• FAIL: Majority of masters agree (like ODOWN). Triggers automatic failover.

Built-in Failover (No Sentinel)

When a master is marked FAIL:

1. Its replica increments currentEpoch (logical clock)
2. Replica requests votes from all masters
3. Masters vote once per epoch (replica with highest replication offset preferred)
4. Majority vote = replica promotes itself
5. New configuration broadcast via gossip

Online Resharding

Slots can be moved between masters while the cluster is running:

1. Target node marks slot as IMPORTING
2. Source node marks slot as MIGRATING
3. Keys are migrated one by one (MIGRATE command)
4. During migration, ASK redirects handle in-flight requests
5. After completion, MOVED redirect is the new permanent state

Scenarios

1. Smart Client Routing -- CRC16 hash, slot lookup, direct routing
2. MOVED Redirect -- Stale cache, permanent redirect, cache update
3. Gossip Protocol -- PING/PONG, PFAIL/FAIL consensus
4. Automatic Failover -- Epoch voting, replica promotion
5. Online Resharding -- Slot migration with ASK/MOVED transitions