Guide
Using synthetic users in research
Synthetic output is a hypothesis, never evidence. Use it to shape what you ask real people and what you spend traffic on, never in place of either.
Where it fits
Use
Explore
- Pilot an interview guide or survey
- Orient in an unfamiliar audience or category
- Generate hypotheses to probe with real customers
- Spot unclear claims before a real copy test
Use with care
Focus
- Drop the weakest variants or concepts before spending traffic
- Check visual hierarchy before a 5 second test
- Boost a thin segment using your own real survey data (specialists only)
Don't
Validate, and any stage for these
- Pick a winner or measure lift
- Size effects, forecasts or samples
- Measure intent, usability or NPS levels
- Supply customer language for copy or keywords
- Speak for niche, minority or non-Western audiences
- Judge something genuinely new
Why: synthetic users give you the most common answer
Studies keep finding the same pattern. Synthetic answers land near the typical response and lose the spread of real people, including the unusual words, edge cases and subgroups that research is usually looking for. Researchers call this mode collapse. Rankings tend to hold up better than sizes, which is why synthetic methods can help you prioritise but can't tell you how much something will move.
Five questions before you run one
- What decision will this change, and what does a wrong answer cost?
- Is this about real behaviour or what people say they think?
- Is the audience well represented in public data?
- Do we need a ranking, a number or the full range of answers?
- Which real-people method follows it, and when?
Rules every time
Pair it with real people
Every synthetic step is followed by a method that could prove it wrong.
Keep it upstream
Synthetic methods carry a low strength of signal and never sit in a Validate step.
Read the order, not the size
Trust which option ranks higher more than how big the gap looks.
Expect flattery
Discount agreement, and treat a narrow spread of answers as a trait of the model.
Ground it in your data
Personas built from your own interviews beat demographic prompts.
Label it
Reports say which findings came from people and which came from models.
Synthetic and hybrid methods in the hub
Sources: AAPOR Task Force on AI in survey research (2026), Nielsen Norman Group (2024 and 2026), Insights Association, MRS, and published studies on LLM survey and A/B prediction. Last reviewed 16 November 2026.