You have a statistically significant result: the new checkout flow increases conversion by 2%. Now what? You don’t immediately flip it on for everyone. You ramp it: gradually increase the percentage of traffic seeing the new experience, watching for problems that didn’t show up in the experiment. Why Ramp After a Successful Experiment The experiment ran at 10% traffic for two weeks. It didn’t show problems. But some issues only appear at scale:

You have experiment assignments: which user got which variant. You have metric events: user completed checkout, user added to cart, user bounced. To compute experiment results, you need to join assignments to events. At millions of events per day across dozens of experiments, this join is the core engineering challenge of an experimentation platform. The Data Model Three streams of data: Assignment events: user 1001 was assigned to treatment in experiment “checkout_v2” at 14:32:00.

Treatment group conversion rate: 4.3%. Control group: 4.1%. Difference: 0.2%. Is that real, or random noise? You need statistics to answer that question, and most engineers implement A/B testing without really understanding what their stats are telling them. The Core Problem If you flip a fair coin 10 times and get 6 heads, you don’t conclude the coin is biased. You know 6 out of 10 is within the range of normal random variation.

You want to test two versions of a checkout flow. Half of users should see version A, half should see version B. The same user should always see the same version across sessions. The assignment must happen in milliseconds. And you need to run 50 experiments simultaneously without them interfering with each other. Deterministic Hashing The simplest assignment mechanism: hash the user ID, take the result modulo 100, assign to treatment or control based on the bucket.