Om's Brain

6. Whatsapp

9 min read

1️⃣ Requirements

whatsapp-req.excalidraw

2️⃣ APIs Design

Since we need low latency, bidirectional communication, and real-time push, we use WebSockets instead of HTTP polling.

Client ➔ Server Commands (WebSocket Frames) Server ➔ Client Commands (WebSocket Frames)

3️⃣ HLD

whatsapp-hdl.excalidraw

Database Design (DynamoDB / NoSQL)

We use a NoSQL database like DynamoDB to handle massive horizontal scale.

1. Chat Table

  • PK: chatId
  • Attributes: name, metadata, createdAt

2. ChatParticipant Table

  • PK: chatId
  • SK: participantId (which is userId)
  • GSI (Global Secondary Index):
    • PK: participantId
    • SK: chatId
    • Query: Retrieve all chats for a given user.

Message Delivery Flows

1. Real-time Delivery (Online Flow)

If the recipient is online, the message travels through the WebSocket connection. Each Chat Server maintains an in-memory connection map:

unordered_map<userId, WebSocketConnection> activeConnections;

If the recipient is connected to the same server, we push directly. If connected to a different server, we route via Redis Pub/Sub (see Deep Dive 1).

2. Offline Messages (Inbox Pattern)

If a user or device is offline, messages cannot be delivered immediately. We store them in a persistent table until they reconnect.

Message Table
  • PK: messageId
  • Attributes: chatId, senderId, content, timestamp
Inbox Table
  • PK: recipientClientId (Client-specific queue)
  • SK: messageId
  • Purpose: Tracks undelivered messages for every device/client.
Sender ➔ Chat Server ➔ Write to Message Table ➔ Write to Inbox Table ➔ Attempt WebSocket Push
ACK Flow
  1. Client receives message.
  2. Client sends ACK {messageId}.
  3. Server deletes corresponding entry from the Inbox Table.
Reconnect Flow
  1. Client establishes WebSocket connection.
  2. Server queries Inbox Table for all pending messageIds for this recipientClientId.
  3. Server fetches message bodies from the Message Table and pushes them to the client.
  4. Client ACK-backs, and Server deletes those records from the Inbox Table.

Media Attachments Flow

Uploading large binary files directly through WebSocket servers degrades chat performance. Instead, we use S3 Pre-signed URLs.

Client                       Chat Server                       Amazon S3
  │                               │                                │
  │ 1. Request upload URL         │                                │
  ├──────────────────────────────>│                                │
  │                               │ 2. Generate pre-signed URL     │
  │ 3. Return pre-signed URL      │                                │
  │<──────────────────────────────┤                                │
  │                                                                │
  │ 4. Upload file binary directly using pre-signed URL            │
  ├───────────────────────────────────────────────────────────────>│
  │                                                                │
  │ 5. Send message with S3 File URL & Metadata                    │
  ├──────────────────────────────>│                                │

5️⃣ Deep Dive

🔥 Deep Dive 1: Scaling Chat Servers & Message Routing

Problem

With millions of concurrent users, connections are spread across multiple chat servers. If User A (on Server 1) wants to message User B (on Server 2), Server 1 cannot write to B’s WebSocket directly.

User A (Server 1)  ──X──>  User B (Server 2)

Solution Comparison

OptionProsCons
Consistent Hashing (Route users to specific servers)Simple, predictable routing.Complex rebalancing when servers scale up/down; connection drops.
Redis Pub/Sub (Dynamic routing)Highly flexible, extremely fast, lightweight.Requires a Pub/Sub infrastructure layer.

✅ Preferred: Redis Pub/Sub Routing

  1. Each Chat Server subscribes to Redis channels representing the userIds of all clients currently connected to it.
  2. When User A sends a message to User B:
    • Server 1 receives it and publishes the message to the Redis channel userB.
    • Redis routes this message to Server 2 (which is subscribed to userB).
    • Server 2 receives the Redis event and pushes the message to User B’s active WebSocket connection.

Why not Kafka? Kafka is built for persistent log storage. Having a topic per user is not feasible when scaling to billions of users. Redis Pub/Sub channels are ephemeral, lightweight, and perfect for dynamic routing.

🔥 Deep Dive 2: Reliability of Redis Pub/Sub

Redis Pub/Sub guarantees at-most-once delivery. If Server 2 crashes or disconnects momentarily, messages published to Redis channels are lost.

Why is this still safe?

We decouple our delivery path:

  • Redis Pub/Sub = Fast Path (real-time push)
  • Inbox Table (DynamoDB) = Reliable Path (durability guarantee)

Before publishing to Redis, the message is always persisted in the database. If the fast path fails (e.g., connection lost, server crashed), the message remains in the Inbox Table. Upon reconnection or heartbeats detecting loss, the client pulls the message from the reliable path.

🔥 Deep Dive 3: Managing WebSocket Failures & Heartbeats

Problem

If a user closes their app, walks into an elevator, or loses internet, the TCP socket might stay open on the server for minutes (TCP timeout). This wastes server resources and leads to “false online status”.

Solution

  1. Heartbeats (Ping/Pong):
    • The client sends a small PING frame to the server every 10–30 seconds.
    • The server responds with PONG.
    • If the server misses 2 consecutive PINGs, it terminates the WebSocket connection, updates the database, and publishes the offline status.
  2. ACK Timers:
    • If the server sends a message to a client and receives no ACK within 5 seconds, it flags the socket as unhealthy, disconnects, and routes future messages to the offline queue.

🔥 Deep Dive 4: Preventing Lost Messages (Sequence Numbers)

Problem

Even with heartbeats, a socket can silently fail during an active message burst, leading to missed messages without triggering a reconnect immediately.

Solution: Sequence Numbers

  • Every message in a chat is assigned a monotonically increasing sequence number (e.g., 101, 102, 103…).
  • The client tracks the highest sequence number it has received.
  • If the client receives message 104 but the last received was 102, it immediately detects that 103 is missing.
  • The client sends a request to the server: GET /messages/sync?lastSeq=102 to retrieve the missing message(s).

🔥 Deep Dive 5: Multi-Device Sync

Schema Changes

  • We modify the Inbox Table partition key to be recipientClientId (unique per device) instead of recipientId.
  • A user table maps userId ➔ list of active clientIds.

Sync Flow

  1. When User A sends a message to User B, the server looks up all active clients for B (e.g., B_Phone, B_Laptop).
  2. The server duplicates the inbox entry for each client ID.
  3. If one device is online (B_Phone) and one is offline (B_Laptop), the message is delivered and cleared from B_Phone’s inbox queue, while remaining in B_Laptop’s queue.
  4. When B_Laptop reconnects, it replays its pending inbox queue.

🔥 Deep Dive 6: Message Ordering in Distributed Systems

Problem

Messages sent at almost the exact same time from different clients can arrive in different orders due to network latency and clock skew.

Solutions

  1. NTP Synchronization: Sync all chat server clocks via Network Time Protocol to minimize drift.
  2. Ingestion-Time Timestamping: The chat server assigns the definitive timestamp when the message first hits the backend.
  3. Sequence IDs: For group chats, a single coordinator/sequencer or database atomic counter assigns sequence numbers.
  4. Client-side Sorting: Clients always sort incoming messages by timestamp / sequence_id before rendering them to the user.

🔥 Deep Dive 7: Presence Status & Last Seen

Presence Database

A high-throughput cache (like Redis) stores the presence states:

  • Key: presence:{userId}
  • Value: { status: "ONLINE" | "OFFLINE", lastSeen: timestamp }

Heartbeat Integration

  • When the server receives a WebSocket PING heartbeat from a user, it refreshes the TTL in Redis (e.g., 60 seconds).
  • If the heartbeat stops, the TTL expires, and the user status transitions to OFFLINE.

Status Propagation (Fan-out)

  • Option A (Pub/Sub): When a user changes state, publish the update to their presence channel. Friends subscribing to this channel get the update in real-time.
  • Option B (Lazy Load / Pull): Only request presence status when the user opens a specific chat window. This dramatically reduces backend load by avoiding fan-out storm of status updates to inactive contacts.

6️⃣ ❌ Bad Math Good Math

❌ Bad Math

Doing useless calculations at the very beginning of the interview just to fill time:

  • “Let’s assume WhatsApp has 2 Billion DAU. If each user sends 10 messages of 100 bytes, we need 2TB storage per day. Moving on…”

✅ Good Math

Using estimation math to influence architectural choices.

1. WebSocket Server Count

  • Let’s estimate how many physical servers we need to support concurrent users.
  • Assumptions:
    • 100 Million concurrent active connections.
    • A single chat server can sustain 100,000 concurrent TCP/WebSocket connections (limited by memory overhead of socket buffers, e.g., 30KB - 60KB per socket, consuming ~6GB RAM).
  • Required Servers: [ \frac{100,000,000 \text{ connections}}{100,000 \text{ connections/server}} = 1,000 \text{ servers} ]
  • Design Impact: We cannot use simple DNS routing. We must use a dedicated L4 NLB cluster to distribute the load across 1,000 servers.

2. Message Throughput & Redis Routing Cluster

  • Can a single Redis node route all real-time messages?
  • Assumptions:
    • 10 Billion messages per day.
    • Average messages per second: [ \frac{10,000,000,000 \text{ messages}}{86,400 \text{ seconds}} \approx 115,000 \text{ messages/sec} ]
    • Peak throughput (5x average): ~575,000 messages/sec.
  • Design Impact: A single Redis node cannot handle 575k pub/sub messages/sec. We must use a Redis Cluster and shard the pub/sub channels using hash(userId).

7️⃣ 🚀 Scaling Optimizations Summary

  • Layer-4 NLB: Pins long-lived TCP/WebSocket connections with minimal L7 processing overhead.
  • Redis Pub/Sub Cluster: Dynamically routes messages between distributed servers based on recipient user IDs.
  • Dual-Path Delivery: Employs Redis Pub/Sub as the fast path and DynamoDB Inbox Table as the reliable recovery path.
  • Inbox Queue Per Client: Separates queue per device ID to allow seamless multi-device message synchronization.
  • Client ACK & Sequence Sync: Protects against message loss during silent connection drops.
  • S3 Pre-signed URLs: Offloads media uploads from WebSocket connection streams to maintain chat responsiveness.
  • Lazy Presence Checks: Avoids fan-out storms by querying presence data only when a chat screen is actively open.

8️⃣ 🎤 Interview Conclusion

Verify that the system meets the core requirements defined in the beginning.

✅ Functional Requirements Covered

  • Real-Time Messaging: Supported using WebSockets + Redis Pub/Sub.
  • Offline Delivery: Managed using DynamoDB Message and Inbox tables.
  • Media Attachments: Enabled via direct S3 uploads with pre-signed URLs.
  • Presence Status: Tracks heartbeats in Redis with client-controlled lazy pulling.

✅ Non-Functional Requirements Covered

  • Low Latency (<200ms): Guaranteed by keeping messages in memory (WebSockets & Redis Pub/Sub) on the fast path.
  • Durability: Enforced by writing messages to persistent database stores before initiating delivery.
  • Multi-device Sync: Achieved by tracking inbox queues at the client-device grain (recipientClientId).

Mindmap

Graph View

Backlinks