Research method
A/B Testing
Running a live, randomised experiment that splits traffic between a control and one or more variants to measure the causal effect of a change on a target metric.
When to reach for it
Confirming that a change actually improves the outcome it targets before rolling it out fully. Settling disagreements about a design or flow with real customer behaviour rather than opinion.
Strengths and limits
Pros
Gives a direct, causal read on impact rather than a proxy signal. Removes bias from the decision since real customers self-select into each path.
Cons
Needs enough traffic and time to reach a reliable result. Only tests the specific variants built, so it depends on earlier research to shape a strong hypothesis.
Practical considerations
Best run once a hypothesis has already been shaped by exploratory and focus-stage research, since a test is only as good as the idea behind it.
Common tools
Go deeper
Recipes that use this method
A method earns its value when its signal corroborates others. Here is where this one does its work.
Last updated
.svg)
