Tinder has 50 million users. When you open the app, you see a deck of profiles. Not 50 million profiles — maybe 100. Candidate generation is the step that narrows from the full user population to the small set of candidates worth scoring and presenting. It runs before the expensive ranking step, and its job is to be fast and to recall the right candidates, not to rank them perfectly.

Gmail filters billions of spam messages per day. The filters must run before message delivery, add minimal latency, and maintain low false-positive rates: legitimate email landing in spam is a worse outcome than spam landing in inbox. Spam filtering is a layered classification pipeline with each layer trading latency for accuracy. Layer 1: Connection and Sender Reputation Before reading a single byte of message content, the receiving mail server checks the sender:

A model trained offline achieves 94% accuracy. In production, accuracy is 78%. The model didn’t change. The data did, in subtle ways you didn’t notice. This is training-serving skew: the distribution of features the model sees during training doesn’t match what it sees during serving. Where Skew Comes From Feature preprocessing differences: the training pipeline normalized “amount” by dividing by the maximum amount in the training dataset. The serving pipeline didn’t.

A model update ships. Fraud scores change. Conversion rates drop. You need to roll back to the previous model in minutes, not hours. If your model deployment is treated differently from code deployment, rollback is painful. If models are versioned and deployed with the same tooling as code, rollback is a one-command operation. What Model Versioning Means A model is a file (or set of files) produced by a training run.

You have a new fraud detection model. It performs better in offline evaluation: higher precision, higher recall on historical data. But offline metrics don’t always translate to production. A model can look great on historical data and behave unexpectedly when it sees live traffic with its latency constraints, real-time features, and edge cases that didn’t appear in the training set. Shadow mode lets you run the new model in production without affecting users.

Training a machine learning model is a batch job that runs for hours or days. Serving predictions from that model to users requires sub-100ms latency. These two requirements produce completely different infrastructure. The model serving layer is where the offline ML world meets the online serving world. The Prediction Service A model serving service exposes an API: input features in, prediction out. POST /predict { "model": "fraud_detection_v3", "features": { "amount": 1250.

A route is a sequence of road segments, each with a travel time estimate. Add them up, you have an ETA. That’s the naive version. It’s also wrong often enough that Google spent years making it less wrong. The Simple Version and Its Failures Sum the edge weights on the shortest path. If the router says the trip is 45 minutes, tell the user 45 minutes. This works fine when traffic weights are accurate and conditions are stable.

You train a model using yesterday’s data. You serve it using today’s data. The feature computation logic is slightly different between the two. The model degrades silently and you spend a week figuring out why. The Training-Serving Skew Problem ML models are trained on offline batches: historical data, features computed via Spark jobs, labels aggregated over time. At serving time, features are computed online: live data, lower latency budget, different code path.

“Find the 10 most similar items to this one” sounds simple. With millions of items represented as 256-dimensional vectors, exact search is too slow to be useful in production. What Embeddings Are An ML model maps an item (a product, a document, a user’s history) to a dense numeric vector. The geometry of that vector space encodes semantic similarity: similar items land close together. You train the model on interaction data and the embeddings learn to represent “things that users treat similarly.

You don’t know what a user wants. But you know what people like them have wanted. That’s the intuition behind collaborative filtering. The Two Approaches User-based CF finds users similar to you, then recommends what they liked. Item-based CF finds items similar to what you’ve already liked. Item-based is generally more stable because user behavior shifts rapidly (you might buy a couch once), while item similarity changes slowly (a couch is similar to other furniture regardless of who buys it).