A popular Twitch stream has 200,000 concurrent viewers. Every one of them can send chat messages. A message from one viewer needs to reach all other viewers within a second. That’s fundamentally different from the fan-out problem in a news feed, and the solutions are different too.

Why News Feed Fan-Out Doesn’t Apply#

News feed fan-out is about delivering content to followers: write to each follower’s timeline, or read and merge at request time. The fan-out is bounded by follower count, which is fixed per user.

Live stream chat is a different problem: every message goes to every current viewer, and viewer count changes dynamically by the minute. You can’t pre-compute fan-out targets. The room membership is ephemeral and changes constantly.

The Chat Server Architecture#

Chat servers maintain WebSocket connections to viewers. At 200K concurrent viewers per stream, you can’t hold them all on one server. Chat traffic is distributed across a fleet of chat servers, each holding connections for a subset of viewers.

When a viewer sends a message, it goes to their chat server. That server needs to broadcast it to all other chat servers, each of which then pushes it to their connected viewers. This is a pub/sub pattern within the chat server fleet.

Viewer A (on Chat Server 1) sends a message:
  Chat Server 1 → publishes to Pub/Sub topic "stream:12345"
  Chat Server 2 → subscribed to "stream:12345", receives message, pushes to its viewers
  Chat Server 3 → same
  Chat Server 4 → same
graph TD A[Viewer Sends Chat Message] --> B[Chat Server 1: their server] B --> C[Pub/Sub: stream channel] C --> D[Chat Server 2: 50K viewers] C --> E[Chat Server 3: 50K viewers] C --> F[Chat Server 4: 50K viewers] D --> G[Push to WebSocket connections] E --> G F --> G style A fill:#000000,stroke:#00ff00,stroke-width:2px,color:#fff style B fill:#000000,stroke:#00ff00,stroke-width:2px,color:#fff style C fill:#000000,stroke:#00ff00,stroke-width:2px,color:#fff style D fill:#000000,stroke:#00ff00,stroke-width:2px,color:#fff style E fill:#000000,stroke:#00ff00,stroke-width:2px,color:#fff style F fill:#000000,stroke:#00ff00,stroke-width:2px,color:#fff style G fill:#000000,stroke:#00ff00,stroke-width:2px,color:#fff

The Message Rate Problem#

200K viewers each sending one message per minute is 3,300 messages per second. During exciting moments, message rates can spike to 50,000 per second. At that rate, displaying every message is useless: messages scroll too fast to read.

The solution: message throttling and aggregation. When message rate exceeds a threshold, only a sampled subset of messages is shown to viewers. The sampled set still represents the variety of reactions, and the chat remains readable. Individual viewers can still send messages, they just might not appear in the flood.

Emote spam (flooding the same emoji) is handled separately: count identical consecutive messages, display one with a count badge instead of repeating the same message 5000 times.

At Oracle#

We had system event notifications that needed to reach operator dashboards. At peak, 50 events per second came in from network elements. Displaying all of them made the UI unusable. We aggregated events by type and severity into summary counts, refreshing every 5 seconds, with a detail view that showed the raw stream on demand. Same pattern as chat throttling: summary view at scale, detail view when you want it.

What I’m Learning#

At live stream scale, chat delivery is really a broadcast problem: one sender, many receivers, ephemeral membership. The pub/sub pattern among chat servers is the standard solution, but the message throttling layer is what makes it usable as an actual product feature.

Have you built real-time broadcast systems where the volume of messages required selective delivery to keep the experience useful?