Amazon has thousands of warehouses. When you order a product, the system checks which warehouses have it in stock, selects the optimal fulfillment location, and reserves that unit. Multi-location inventory is different from seat inventory consistency: seats are identical and interchangeable, but warehouse location matters for delivery time and shipping cost. The same product at a warehouse 2,000 miles away is not the same as one 50 miles away. The Data Model Inventory is per-SKU per-warehouse: (sku_id, warehouse_id, quantity_available, quantity_reserved).

Amazon’s left sidebar: “Brand: Nike, Adidas, New Balance. Price: Under $50, $50-$100. Size: 8, 9, 10, 11.” Each filter is a facet. Selecting one narrows results and updates the counts on all other facets. Faceted search is not just filtering — it’s computing counts for all possible filter values simultaneously, on the filtered result set. The Count Problem The hard part is not filtering: “show me Nikes under $50” is a simple index query.

A Tinder match happens when two users both swipe right on each other. The moment user B swipes right on user A, the system must detect that A has already swiped right on B and trigger the match notification. This is mutual match detection: an efficient, low-latency check for bidirectional intent. The Naive Approach When user B swipes right on user A: query the swipes table for swiper_id=A AND swiped_id=B AND direction=right.

Tinder processes 1.6 billion swipes per day. Each swipe is a binary decision: left (pass) or right (like). This data drives match detection, recommendation quality, and abuse prevention. Swipe storage is a write-heavy, eventually consistent problem with specific read patterns: “has user A already swiped on user B?” and “who has liked user A?” Write Path At 1.6 billion swipes per day, that’s 18,500 writes per second on average, with heavy peaks during evenings.

Tinder has 50 million users. When you open the app, you see a deck of profiles. Not 50 million profiles — maybe 100. Candidate generation is the step that narrows from the full user population to the small set of candidates worth scoring and presenting. It runs before the expensive ranking step, and its job is to be fast and to recall the right candidates, not to rank them perfectly.

Gmail filters billions of spam messages per day. The filters must run before message delivery, add minimal latency, and maintain low false-positive rates: legitimate email landing in spam is a worse outcome than spam landing in inbox. Spam filtering is a layered classification pipeline with each layer trading latency for accuracy. Layer 1: Connection and Sender Reputation Before reading a single byte of message content, the receiving mail server checks the sender:

Web search builds one massive index shared by all users. Email search is different: every query is private to one user, and results depend on that user’s labels, read status, and folder structure. Gmail’s search index is not one index — it’s one index per user. Per-user search indexes are the right architecture when search results are private, personalized, and must reflect user-specific metadata. Why Not a Shared Index A shared inverted index for email would store every user’s email together: word “invoice” points to all documents across all users containing “invoice.

Gmail groups related emails into conversations. Reply to a thread from three different email clients, forward it twice, and all seven messages collapse into one conversation. This is email threading: the problem of grouping messages that belong to the same conversation, despite arriving out of order, from different clients, with inconsistent subject lines. Threading Identifiers Email headers carry threading signals: Message-ID: a unique identifier for each message, set by the sending client.

An advertiser wants to reach “women aged 25-34 in San Francisco who have browsed running shoes in the last 7 days.” When a bid request arrives for a user, the DSP must evaluate whether this user matches the targeting criteria in under 5ms. The ad targeting pipeline is the system that makes this lookup fast. Audience Segments as Bit Arrays Each targeting criterion defines an audience segment: a set of user IDs.

An advertiser sets a $1,000 daily budget. Without pacing, the DSP would spend the entire budget in the first hour of the day, when bid prices happen to be low, and the ad would show to nobody for the remaining 23 hours. Budget pacing distributes ad spend evenly across the day (or according to a target schedule) so the advertiser gets coverage throughout the day rather than a burst. The Pacing Problem Pacing is a rate control problem with a stochastic input.

When you load a web page, an auction happens in under 100 milliseconds to decide which ad you see. A publisher sends a bid request to an ad exchange. The exchange fans it out to dozens of demand-side platforms (DSPs). Each DSP decides whether to bid and at what price. The exchange picks the winner. All of this completes before the page finishes loading. Real-time bidding (RTB) is one of the most latency-sensitive distributed systems in production.

A payment system that approves every transaction is useless. One that declines too many legitimate transactions loses customers. Fraud detection is a latency-sensitive classification problem: decide in under 100ms whether a transaction is fraudulent, using signals that fraudsters will actively try to evade. The Signal Stack Fraud signals come from multiple layers: Velocity checks: has this card been used more than 5 times in the last minute? Has this device initiated more than $500 in the last hour?

When your location changes, every friend who has you in their Nearby Friends list needs to know. At 10 million active users each with 200 friends, a single location update could trigger 200 downstream notifications. At 333,000 updates per second, that’s 66 million fan-out operations per second. This is location fan-out: propagating position changes to all interested subscribers efficiently. Why This Is Different From Chat Fan-Out Chat fan-out sends one message to N subscribers.

Nearby Friends shows you that a friend is “0.3 miles away.” It does not show you their exact coordinates. This is not just a UI choice: the system should never expose precise location data even if the client asks for it. Privacy-aware proximity means computing distance and revealing enough for the feature to be useful, while structurally preventing exact location exposure. Bucketed Distance The simplest approach: compute exact distance server-side, then bucket the result before returning it.

A Nearby Friends feature needs to know where each of your friends is right now. Not where they were an hour ago. The data has a very short useful life: a location update older than 30 seconds is essentially stale. This is ephemeral location storage: high-frequency writes, short TTL, and no need for historical persistence. The Write Pattern Each mobile client sends location updates every 30 seconds while the app is active.

You pick a seat, enter your credit card details, and for those 10 minutes the seat is yours. Nobody else can take it. If you don’t complete payment in time, the seat releases back to inventory. This is the hold-and-confirm pattern: temporarily reserve a resource, give the user time to complete a multi-step transaction, then either confirm the reservation or release it. Why You Need It Without holds, the user experience is broken.

When 500,000 people hit your site at 10:00:00 AM, your backend doesn’t scale to 500,000 concurrent sessions in milliseconds. A virtual waiting room absorbs the spike: users enter a queue, receive a position, and are admitted to the actual purchase flow in controlled batches. Without it, the site goes down for everyone. With it, users wait but the experience is orderly. The Queue Token Pattern When a user arrives during high load, issue them a signed queue token containing their position and timestamp.

Write amplification is when a single logical write triggers multiple physical writes to storage. It’s one of those problems that’s invisible at small scale and becomes a serious bottleneck as data grows. LSM trees were designed to minimize it. Replication inherently causes it. Understanding where it comes from helps you predict system behavior under load. Sources of Write Amplification Storage layer: writing 4KB of data to a filesystem that uses 512-byte sectors requires updating multiple sectors plus the inode, the directory entry, and potentially metadata blocks.

System design interviews always start with “estimate the scale.” Engineers without production experience guess numbers and move on. Engineers who’ve been paged at 3am because they guessed wrong take this seriously. Capacity planning is the discipline of estimating resource requirements before you run out of them. The Estimation Framework Start with requests per second. Then translate into resource requirements: CPU, memory, network bandwidth, storage. Example: URL shortener serving 100 million active URLs.

You add a new derived field to your user profiles: “days since last purchase.” You can compute it for new events going forward. But existing users already have purchase history. You need to compute this field for all existing users retroactively. That’s a backfill. Backfills happen constantly in production systems. They’re more disruptive than they look. The Basic Problem You have a dataset of 500 million user records. You need to reprocess all of them to compute the new field.

The saga pattern coordinates multi-step distributed transactions by breaking them into a sequence of local transactions with compensating actions for rollback. What the saga post didn’t cover: there are two fundamentally different ways to implement coordination, and choosing between them shapes your entire system architecture. Choreography: Services React to Events In choreography, there is no central coordinator. Each service listens for events and reacts by doing its part and publishing its own events.

A model trained offline achieves 94% accuracy. In production, accuracy is 78%. The model didn’t change. The data did, in subtle ways you didn’t notice. This is training-serving skew: the distribution of features the model sees during training doesn’t match what it sees during serving. Where Skew Comes From Feature preprocessing differences: the training pipeline normalized “amount” by dividing by the maximum amount in the training dataset. The serving pipeline didn’t.

A model update ships. Fraud scores change. Conversion rates drop. You need to roll back to the previous model in minutes, not hours. If your model deployment is treated differently from code deployment, rollback is painful. If models are versioned and deployed with the same tooling as code, rollback is a one-command operation. What Model Versioning Means A model is a file (or set of files) produced by a training run.

You have a new fraud detection model. It performs better in offline evaluation: higher precision, higher recall on historical data. But offline metrics don’t always translate to production. A model can look great on historical data and behave unexpectedly when it sees live traffic with its latency constraints, real-time features, and edge cases that didn’t appear in the training set. Shadow mode lets you run the new model in production without affecting users.

Training a machine learning model is a batch job that runs for hours or days. Serving predictions from that model to users requires sub-100ms latency. These two requirements produce completely different infrastructure. The model serving layer is where the offline ML world meets the online serving world. The Prediction Service A model serving service exposes an API: input features in, prediction out. POST /predict { "model": "fraud_detection_v3", "features": { "amount": 1250.

You have a statistically significant result: the new checkout flow increases conversion by 2%. Now what? You don’t immediately flip it on for everyone. You ramp it: gradually increase the percentage of traffic seeing the new experience, watching for problems that didn’t show up in the experiment. Why Ramp After a Successful Experiment The experiment ran at 10% traffic for two weeks. It didn’t show problems. But some issues only appear at scale:

You have experiment assignments: which user got which variant. You have metric events: user completed checkout, user added to cart, user bounced. To compute experiment results, you need to join assignments to events. At millions of events per day across dozens of experiments, this join is the core engineering challenge of an experimentation platform. The Data Model Three streams of data: Assignment events: user 1001 was assigned to treatment in experiment “checkout_v2” at 14:32:00.

Treatment group conversion rate: 4.3%. Control group: 4.1%. Difference: 0.2%. Is that real, or random noise? You need statistics to answer that question, and most engineers implement A/B testing without really understanding what their stats are telling them. The Core Problem If you flip a fair coin 10 times and get 6 heads, you don’t conclude the coin is biased. You know 6 out of 10 is within the range of normal random variation.

You want to test two versions of a checkout flow. Half of users should see version A, half should see version B. The same user should always see the same version across sessions. The assignment must happen in milliseconds. And you need to run 50 experiments simultaneously without them interfering with each other. Deterministic Hashing The simplest assignment mechanism: hash the user ID, take the result modulo 100, assign to treatment or control based on the bucket.

A search index with a billion documents returns thousands of candidates matching a query. Showing all of them ranked by TF-IDF doesn’t produce great results. The ranking signals needed for good results (machine-learned models, user personalization, freshness, domain authority) are expensive to compute. You can’t compute them for a thousand candidates in under 100ms. The solution: a funnel. Use cheap signals to eliminate most candidates fast, then apply expensive signals to the small remainder.