Chat System Design

The illustrative workload is 1 billion registered users, 100 million concurrent connections and 50 sent messages/user/day: 5 billion messages/day, about 57,870/s average and a 170,000/s planned peak. A planning density of 20,000 established connections per WebSocket pod requires about 5,000 pods before failure headroom. A 30-second presence heartbeat at 100 million online users is 3.33 million updates/s; 64 presence shards average about 52,100/s each and must be load-tested. Presence is advisory, not a delivery ledger.

Durable acceptance and delivery

Each device supplies a stable client_msg_id. In one database transaction the message service verifies an event-time membership snapshot, enforces unique sender_device_id plus client_msg_id, allocates a monotonic chat_seq, stores the message and inserts an outbox event. Only the committed result earns ACK-DURABLE. A WebSocket write alone is not an acceptance ACK.

The outbox relay publishes by chat_id. Kafka orders records inside a partition, and one record goes to one consumer in each consumer group; it does not broadcast a chat record to every WebSocket pod in one group. The delivery consumer expands authorized recipients, deduplicates recipient_id plus msg_id, consults the connection registry and sends to the owning pod. Offline devices catch up from a durable cursor even if APNs/FCM drops or collapses a wake-up notification.

chat_seq is assigned at durable commit and defines total order within one chat. Timestamps, Snowflake IDs and per-sender counters do not totally order concurrent senders. A later sequence can appear first because of downstream retries or parallel recipient delivery, so clients buffer gaps and advance a durable cursor. This is not attributed to one WebSocket/TCP connection reordering messages; WebSocket fragments are delivered in sender order.