Experimentation / CRO

Experimentation and Testing Programs acknowledge that the future is uncertain. These programs focus on getting better data to product and marketing teams to make better decisions.

Research & Strategy

We believe that research is an integral part of experimentation. Our research projects aim to identify optimization opportunities by uncovering what really matters to your website users and customers.

Data and Analytics

90% of the analytics setups we’ve seen are critically flawed. Our data analytics audit services give you the confidence to make better decisions with data you can trust.

Hypothesis Testing vs “Do No Harm” Treatments in A/B Testing

"Hypothesis Testing" and "Do No Harm" treatments represent different experimental goals, hence it is important to differentiate them to ensure correct statistical analysis and interpretation of results, as well as increase your testing velocity. Hypothesis testing is used to test whether one variation is significantly better than another, while a "Do No Harm" treatment is used to test whether one variant is not significantly worse than another by a predetermined margin.

Use Cases:

- Estimating the duration of an experiment
- Balancing a portfolio of experiments in your program
- Increasing your testing velocity

Type:
Artifact
Pillar:
Test & Learn
Get this Blueprint PDF
Get the Figma Toolkit
Toolkit includes PDF of the blueprint, links to related assets and a loom video on using the blueprint.

No spam - just the good stuff

Get this blueprint toolkit and the best of experimentation and CRO news delivered right into your mailbox, every week!
Thank you! Your submission has been received!
Oops! Something went wrong while submitting the form.
Open Figma Toolkit Now
Toolkit includes PDF of the blueprint, and if available links to related assets and a loom video on using the blueprint.