Experimentation

Holdout Groups and Ramping: How to Split Your Traffic

Teams use the word holdout for 3 different jobs, and mixing them up costs either a production bug or a month of blurry data. Here is when to ramp, why 50/50 beats 90/10, and how to read a test when the decision to ship is already made.

Price Elasticity 101: How to Test Prices Without Wrecking Your Data or Your Customers

Price is the fastest lever you have, but list prices don't behave like a button color test. Vista ran 95% of its price experiments outside the standard A/B model entirely. This post covers what elasticity actually measures, why switchback testing beats user-level randomization for most catalogues, how to calculate breakeven before you launch, and the mistakes that wreck price test results.

How AI is Changing Who Builds Experiments and How

Nineteen senior experimentation practitioners tested AI-built A/B test variations on their own sites. Here's what changed for developers, and what didn't.