Editorial update — October 15, 2026. This article was originally published on September 15, 2017. We have replaced the Universal Analytics and Optimizely Classic setup instructions with a GA4-focused analysis workflow. The original publication date is retained.
Use Google Analytics 4 to investigate behaviour around an A/B test and check measurement. Make the experiment decision using the agreed analysis plan and a statistical method appropriate to the randomization. A difference between two GA4 report rows does not, by itself, establish that a treatment caused a change.
What changed since the original guide?
Universal Analytics is retired. Its custom-dimension slots, Classic integration screenshots and window.ga event snippets in the former version of this article are not instructions for GA4. Google documents the retirement in Google Analytics 4 has replaced Universal Analytics. Use the current integration for your experimentation product and analytics installation.
1. Define the decision before collecting data
Specify the hypothesis, eligible population, randomization unit, primary outcome, guardrails and analysis window. Set the smallest effect that would change the business decision and plan the sample size or sequential decision rule accordingly. Do not end a fixed-horizon experiment simply because a dashboard briefly crosses a significance threshold.
Agree which system supplies the primary outcome and how differences between systems will be investigated. GA4 can support diagnosis; it does not have to be the source of truth for every experiment.
2. Connect assignment and exposure to GA4
Use your vendor’s supported GA4 integration. Check whether your Google tag is installed directly or through Google Tag Manager before choosing a setup route. For example, VWO provides a specific GA4 integration guide for Google Tag Manager. Optimizely documents selecting integrations in its Web Experimentation setup guide. Follow the documentation for your product rather than copying event names from another tool.
Check that the integration identifies the experiment and variant consistently. Document when assignment happens, when exposure is recorded and how the outcome is associated with the assigned unit. A repeated exposure event is not a new independent participant. Avoid mixing users, sessions and events as if they were interchangeable denominators.
3. Validate measurement before interpreting a result
- Check both control and treatment journeys, including repeat visits and the relevant outcome events.
- Look for missing or duplicate exposure and outcome records, inconsistent variant identifiers and unexpected variant switching.
- Check the observed allocation against the intended split. Investigate a sample-ratio mismatch before trusting the outcome.
- Reconcile date ranges, time zones, eligibility, metric definitions and identity rules across systems.
- Investigate consent choices, blockers, tag timing and cross-domain tracking as possible reasons for missing observations. Do not assume a fixed discrepancy percentage is acceptable.
If recording an outcome depends on behaviour changed by the treatment, the measured groups may no longer be comparable. Resolve that measurement problem before explaining the apparent lift.
4. Use GA4 to investigate behaviour
Once the integration is validated, examine relevant journeys and outcomes for the experiment groups. Start with questions such as where people leave the flow, whether an intermediate action changes and whether revenue and transaction patterns are consistent with the primary result. Check the scope of each report or segment: including users who were ever exposed is not necessarily the same as counting only eligible outcomes after assignment.
Record any sampling, thresholding, missing data or reporting limitations shown by the system. Use a validated export and an appropriate analysis workflow when the question cannot be answered reliably in the report. For a structured starting point, see our analytics data analysis method.
5. Separate planned analysis from exploration
Analyse the primary outcome using the planned method, including an effect estimate and its uncertainty. A non-significant result does not prove that the variants are equivalent or that the change is harmless. Ask whether the interval still includes effects large enough to matter.
Pre-specified segments can answer planned questions when the design supports them. Searching many segments after seeing an inconclusive result increases the risk of finding a chance pattern. Treat those findings as hypotheses for follow-up, not automatic grounds for personalization. A significant result in one segment and a non-significant result in another does not itself establish that their treatment effects differ.
6. Record the decision and what remains uncertain
Document the observed effect, uncertainty, guardrail results, data-quality issues and decision. If you choose a design for usability or brand reasons without demonstrated lift, say so. If the remaining uncertainty is too large, decide whether to gather more evidence, redesign the test or stop pursuing the idea.
When traffic cannot support a useful experiment, use the low-traffic research recipe to investigate usability and comprehension. Those findings can guide design decisions; they do not replace a causal estimate of conversion impact.
.svg)
