A popular Twitch stream has 200,000 concurrent viewers. Every one of them can send chat messages. A message from one viewer needs to reach all other viewers within a second. That’s fundamentally different from the fan-out problem in a news feed, and the solutions are different too. Why News Feed Fan-Out Doesn’t Apply News feed fan-out is about delivering content to followers: write to each follower’s timeline, or read and merge at request time.

Standard HLS has 15-30 seconds of latency between the streamer’s action and the viewer seeing it. For a live sports match, that’s an eternity. Viewers checking social media see the goal before they see it on stream. Low-latency HLS and related approaches cut this to 2-5 seconds. Where the Latency Comes From Standard HLS segments are 6-10 seconds long. The viewer’s player buffers 2-3 segments before playing. That’s 12-30 seconds of buffering before a single frame plays.

A streamer broadcasts at 1080p60 on a 6Mbps upload. A viewer on a mobile phone with a weak signal can’t play 6Mbps video. A viewer on a 4K TV would love more. The platform needs to serve the same stream at multiple quality levels simultaneously. That’s what transcoding at ingest does. The Ladder A “bitrate ladder” is the set of quality levels a stream is available in. A typical ladder:

Broadcasting live video starts before CDN distribution, before transcoding, before viewers. It starts with getting the video from the streamer’s machine to your servers reliably. That first hop, from broadcaster to origin, is where RTMP fits. RTMP: Designed for Live RTMP (Real-Time Messaging Protocol) was created by Macromedia in the early 2000s for Flash video. It’s old. It’s also still the standard ingest protocol for live streaming because it solves the ingest problem well.

A global leaderboard for a game with 500 million players across 20 countries needs to aggregate scores from everywhere. A player in Mumbai competing for rank 1 globally is competing against a player in São Paulo. The data is split across regional deployments for latency reasons. Combining it without introducing consistency problems is the interesting part. Why Regional Splits Exist Storing all player scores in one region means cross-region write latency for score updates.

A Redis sorted set on a single node handles tens of millions of members with O(log N) operations. That’s enough for most games. Then you have 500 million players and a score update on every match completion. Suddenly one node isn’t enough. The Single-Node Limit A Redis sorted set for 500 million players needs roughly 5-8 GB of memory just for the skip list nodes and hash table entries. More importantly, every score update and rank query hits one CPU core, because Redis is single-threaded per shard.

Redis sorted sets support O(log N) insert, O(log N) rank lookup, and O(log N + M) range queries. A balanced BST like a red-black tree could also do this. Redis uses a skip list instead. For years I just accepted that without understanding why. How a Skip List Works A skip list is a linked list with express lanes. The base level is a sorted linked list of all elements. Above it are increasingly sparse “express lanes”: level 2 has every 2nd element, level 3 every 4th, and so on, each determined probabilistically at insert time.

The namenode knows where data lives. It doesn’t move data. Once a client has chunk locations from the namenode, all data flows directly between the client and datanodes. That separation is what makes HDFS scale. The Write Path Writing a file: The client contacts the namenode, which allocates chunk IDs and assigns 3 datanodes for each chunk based on rack-aware placement. The client sends data to the first datanode. The first datanode simultaneously writes to disk and forwards data to the second datanode.

Three replicas gives you fault tolerance. It also means every byte of storage you buy, you get one-third of it for actual data. For cold storage with petabytes at rest, that cost is hard to justify. Erasure coding is how you get fault tolerance without paying 3x. The Idea Take a chunk of data. Split it into k data fragments. Generate m parity fragments using the data fragments. Store all k+m fragments on different nodes.

Storing 3 replicas of a chunk on 3 different machines sounds like good fault tolerance. It’s not, if all 3 machines are in the same network rack. A rack losing power or a top-of-rack switch failing takes down all 3 replicas simultaneously. You’ve built 3 copies but achieved the fault tolerance of 1. How Racks Fail In a data center, servers are grouped into racks of 20-40 machines. Each rack shares a top-of-rack (TOR) switch and often shares power distribution.

HDFS has a single namenode. Every read and write starts there. For a system designed to scale to petabytes, having one machine that every client must contact sounds like a terrible idea. It kind of is. Understanding why they did it anyway, and how they mitigated it, is the interesting part. What the Namenode Stores The namenode holds all filesystem metadata in memory: the directory tree, file-to-chunk mappings, chunk-to-datanode mappings, and file permissions.

A 10GB video file on a single disk is just a file. A 10GB video file in a distributed system is a problem. How do you store it? Which machine does it go on? What happens when that machine dies? The answer used in GFS and HDFS: split the file into fixed-size chunks, store each chunk on a different machine, and keep the mapping from file to chunks in a separate metadata server.

A route is a sequence of road segments, each with a travel time estimate. Add them up, you have an ETA. That’s the naive version. It’s also wrong often enough that Google spent years making it less wrong. The Simple Version and Its Failures Sum the edge weights on the shortest path. If the router says the trip is 45 minutes, tell the user 45 minutes. This works fine when traffic weights are accurate and conditions are stable.

Every phone running Google Maps is a sensor. It sends GPS coordinates every few seconds. Aggregate enough of those pings from enough phones on the same road, and you can calculate actual vehicle speeds. That’s where the red and yellow lines on the map come from. The Pipeline Each GPS update includes a location, timestamp, and device ID. The system needs to: Map the coordinate to a road segment. A phone at (37.

When you pan across Google Maps, you’re not downloading one giant image. You’re downloading hundreds of small squares stitched together by your browser. Each square is a tile. The tile system is what makes maps feel fast. The Tile Grid The world map is divided into a grid at each zoom level. At zoom 0, the entire world is one 256x256 pixel tile. At zoom 1, it splits into 4 tiles.

Dijkstra’s algorithm finds the shortest path in a weighted graph. Every CS student learns it. What they don’t learn is that vanilla Dijkstra on a continental road graph is too slow to use in production, and fixing that takes some clever preprocessing. Why Dijkstra Is Too Slow Dijkstra explores nodes in order of increasing distance from the source. In the worst case, finding a route from one end of a continent to the other means exploring most of the graph before finding the destination.

The road network is a graph. Intersections are nodes, road segments are edges, weights are travel time. A CS textbook covers that in two sentences. What the textbook skips is how you store a graph with hundreds of millions of nodes and query it in milliseconds. The Naive Approach Breaks Fast A normalized relational schema works: intersections table, road_segments table with from_id, to_id, travel_time_seconds. Routing algorithms traverse edges. Each traversal step is a query on road_segments filtered by from_id.

You shard your database to scale. You pick a shard key. If you pick something unrelated to tenant, queries for one tenant’s data scatter across all shards. If you pick tenant ID, all of one tenant’s data lands on one shard, and a large tenant can overwhelm it. Why Tenant ID Makes Sense as a Shard Key Tenant isolation is the priority in a multi-tenant system. If all of Tenant A’s data is on Shard 2, a query for Tenant A’s records goes to Shard 2 only.

Tenant A generates 10x the normal query load for 20 minutes. Your database CPU spikes. Tenant B, doing nothing unusual, sees 5-second query times. Tenant B’s SLA is breached. Tenant A didn’t do anything wrong. This is the noisy neighbor problem. Why It Happens Shared infrastructure means shared resources. CPU, memory, I/O, and network bandwidth are fungible. When one tenant consumes more than their share, others get less. In a single-tenant system, this is your own problem.

You’re building a SaaS product. Do you give each customer their own database? Put everyone in one? Somewhere in between? The answer affects cost, isolation, compliance, and how much operational pain you take on for the life of the product. The Three Models Shared database, shared schema: all tenants in the same tables, with a tenant_id column. One database to manage. Lowest cost. The risk: a bug that forgets the tenant_id filter leaks one customer’s data to another.

Sometimes the change you need to make breaks compatibility. You can’t add a default. The field type genuinely needs to change. You can’t keep the old schema. Here’s what you do instead. The New Topic Strategy The cleanest approach: create a new topic with the new schema. Producers write to both old and new topics in parallel. Consumers migrate to the new topic one by one. When all consumers are on the new topic, stop writing to the old one.

Kafka retains messages for days or weeks. Your consumer code will be updated independently of your producer. That means old messages need to be readable by new consumer code, and new messages need to be readable by old consumer code. You can’t just change a field. What Backward and Forward Mean Backward compatibility: a new consumer can read messages written with the old schema. If you add an optional field with a default, old messages (which don’t have that field) are still valid.

Service A writes a Kafka message with field user_id. Service B reads it. Service A’s team renames it to userId next sprint. Service B starts throwing deserialization errors at runtime. Neither team knew about the other. The Problem In a microservices system passing messages through Kafka, producers and consumers evolve independently. There’s no enforced contract. A producer can change a field name, add a required field, or change a data type, and the consumer finds out when deserialization fails in production.

100 nodes in a cluster. Every node needs to know when another node fails. If every node heartbeats every other node, that’s 9,900 heartbeat streams. At scale, this becomes the majority of your network traffic. How SWIM Works SWIM (Scalable Weakly-consistent Infection-style Membership) uses gossip-based dissemination with indirect probing. Instead of every node monitoring every other node, each node monitors a small random subset. When a node suspects another has failed, it asks a few other nodes to probe the suspect on its behalf.

Node 3 is down. A write comes in that belongs there. You could reject it. Or you could accept it, hold it somewhere safe, and deliver it when Node 3 comes back. What Hinted Handoff Does In a distributed database with replication, each write goes to a coordinator node, which forwards it to the nodes that own the data. If an owner is unreachable, the coordinator stores the write temporarily with a hint: “this write is intended for Node 3.

Your query is always “give me all events for user X, sorted by time.” A row-oriented database gives you rows where you pay for every column you didn’t ask for. Wide-column stores flip the model: you design the schema around your query, not the other way around. How It Works In a wide-column store like Cassandra or HBase, the primary key has two parts: the partition key and the clustering key.

Deploy the new version. Test it. Switch traffic. If something breaks, switch back. Instant rollback. Sounds ideal. The database migrations are where it gets complicated. The Pattern Blue-green runs two identical production environments. Blue is live. Green is idle. You deploy your new version to green. You test it against real infrastructure but with no live traffic. When you’re confident, you flip the load balancer to point to green. Green is now live.

CI passed. Staging tests passed. You’ve reviewed the code three times. Then you ship to production and something you never predicted breaks at scale. What Canary Means A canary release sends a small fraction of real traffic to the new version before switching everyone over. 1% of users hit v2, 99% hit v1. You watch your metrics. If v2 behaves well, you expand: 5%, then 20%, then 100%. If metrics degrade, you route that 1% back to v1 and investigate without anyone else affected.

You ship a feature. Three minutes later, on-call pings you: error rate spiked. You need to roll back. A full redeploy takes 20 minutes. With a feature flag, rollback takes 30 seconds. What a Flag Is A feature flag is a conditional in your code. If the flag is on, the new code path runs. If it’s off, the old behavior runs. The flag is a config value read at runtime, not at deploy time.

Every team writes the same retry logic. The same circuit breaker boilerplate. The same mTLS handshake setup. The platform team changes the retry policy and now has to update 30 services. There’s a better way. The Sidecar Pattern A sidecar is a separate process running in the same pod as your service. It intercepts all network traffic in and out. Your service code is unchanged. The sidecar handles retries, timeouts, circuit breaking, load balancing, and observability.