Research method
Synthetic Shopper Simulation
- Explore
- Focus
AI agents acting as shoppers browse or judge a control and one or more variants, and the tool predicts which version would perform better. Some tools drive real browsers on live or staged pages, others compare screenshots. It is a way to pre-screen ideas, not a substitute for a live experiment.
When to reach for it
Pre-screen a test backlog and drop the weakest variants before spending traffic.
Sanity-check a variant for obvious friction before launch.
Compare two themes or page versions when traffic is too low to test both live.
Often paired with A/B Testing.
Pros
Returns a prediction in hours rather than weeks of live traffic.
Session replays and agent reasoning can surface friction worth investigating.
Useful for ordering a backlog when traffic is the constraint.
Cons
Independent studies of predicting A/B winners land well below vendor headlines, and one study of 330 CRO tests found model judges close to chance.
Predicted effect sizes are often shrunk or inflated, so they cannot feed revenue forecasts or sample size planning.
Headline match rates can mislead, because many 'winners' in test archives come from non-significant tests.
Agents don't behave like real shoppers with real money, time pressure and habits.
Practical considerations: sample sizes, tools, logistics
Practical considerations
Pair it
Anything worth shipping still goes through an A/B Test. Use synthetic results to decide what to test, never what to ship.
Tool scope
Shopify SimGym compares two themes only, and bot protection can block its shoppers. Squoosh and VWO's synthetic test suit backlog screening better.
Vendor claims
Ask how a match rate was measured, how many tests it covers and whether non-significant tests count as winners.
Never in isolation
Recipes that use Synthetic Shopper Simulation
A method earns its value when its signal corroborates others. Here is where this one does its work.
Last updated
.svg)
