System Design · Design Question
Design a Chat System
1:1 and group messaging: WebSockets, fan-out, persistence, presence, and delivery receipts.
1. Requirements
Functional
- 1:1 and group chats; send/receive text (media later).
- Online presence; typing optional.
- Message history; delivery / read receipts.
- Push notification when recipient offline.
Non-functional
- Low latency delivery for online users.
- At-least-once to device; idempotent message ids.
- Order per conversation (not global across all chats).
Estimation snippet
Scope: 1:1 + small groups (<100), text only
DAU 50M; 40 msgs/user/day → ~23k msg writes/s average (peaks 5–10x)
Online: 10% concurrent → 5M WebSocket connections → gateway fleet + conn map
Storage: 50M * 40 * 200B ≈ 400GB/day raw → shard by conv_id; TTL/archive cold
2. API
REST:
POST /conversations { members: [...] }
GET /conversations/{id}/messages?before=cursor&limit=50
POST /conversations/{id}/messages { clientMsgId, text }
WebSocket:
client → server: send_message | ack | typing
server → client: message | receipt | presence
3. Data model
users(user_id, ...)
conversations(conv_id, type, created_at)
members(conv_id, user_id, last_read_msg_id, role)
messages(conv_id, msg_id, sender_id, text, created_at, client_msg_id)
PK (conv_id, msg_id) -- msg_id time-sortable Snowflake
device_sessions(user_id, conn_server, device_id) -- for routing
4. High-level design
Clients <--WSS--> Connection Gateway fleet
| pub/sub (Redis / Kafka)
v
Chat Service → Message Store (Cassandra / SQL shard by conv_id)
|
+--> Push Service (APNs/FCM) if offline
+--> Presence Service (heartbeat + TTL keys)
5. Deep dive — online delivery
- Client sends message with
clientMsgId(dedupe). - Chat service persists, assigns server
msg_id. - Lookup members’ active connections → publish to gateway(s).
- Gateway pushes frame; client ACKs; update delivery receipt.
Fan-out for group size G:
small G: write-time fan-out to each inbox / connection
large G: store once; members pull / fan-out on read (or hybrid)
6. Presence
Heartbeat every 30s → Redis SET user:42:online EX 60
Disconnect / TTL expiry → offline event (with grace to avoid flap)
7. History and sync
- Paginate by
(created_at, msg_id)cursor. - Multi-device: each device tracks checkpoint; fetch gap on reconnect.
- Cassandra/Dynamo wide-row per conversation is a common pattern; SQL works early.
8. Scaling
- Connection gateways are stateful for sockets; sticky LB or conn map in Redis.
- Shard messages by
conv_id. - Kafka for durable fan-out and push pipeline.
- Separate channels for receipts to keep text path fast.
9. Edge cases
- Duplicate sends → unique
(sender, clientMsgId). - Recipient in multiple devices → deliver to all; read receipt once.
- Group of 10k → do not naively open 10k sync WS writes; use hybrid fan-out.
10. Media messages (extension)
Upload media to object storage first; chat message carries the URL and thumbnail. Virus scan asynchronously; gate download behind auth. Do not push multi-MB binaries through the WebSocket.
Client → upload session → S3
→ WS send { type:image, mediaId, w, h }
Server persists message; recipients fetch media via CDN signed URL
11. Moderation and retention
- Optional async moderation queue on message text.
- Retention / delete-for-everyone needs careful fan-out of tombstones.
- E2E encryption changes server visibility — only mention if asked.
Quick revision
- WebSocket gateways + pub/sub to reach online users.
- Persist first (or in parallel) for history and offline catch-up.
- Order and shard by conversation id.
- clientMsgId for idempotency; server Snowflake msg_id.
- Presence via heartbeat + TTL; push for offline.
- Small groups: write fan-out; huge groups: pull / hybrid.
- Conn map ties user → gateway instance.