Process & Methodology

When to Run an A/A Test, and When to Check Instrumentation

A single A/A test cannot prove your testing tool is sound, and it fails on its own about 1 time in 20 when everything is working. The sample ratio check on the same data is roughly 6x more sensitive. Here is a decision tree for when to run an A/A test, when an instrumentation check is enough, and when to do neither.

Sideshow EP4: How does the structure of our organization influence how well we experiment?

Explore four experimentation org structures - product-operating and feature-operating, centralized and decentralized - and how they help you experiment better.

Sideshow EP3: Strategic Finance and Experimentation - Are You Wasting Money on Wrong Experiments?

Explore why experiment is a forward-looking investment, how finance can help you translate results into ROI language, and how to align tests with top-down goals

CoE Interviews Part 10: Marty Cagan - Get Your Head Out of the Sand

Explore why Marty Cagan says you need to get your head out of the sand and stop running low-risk optimization, if you want real innovation

How to create an optimized experimentation strategy using the ResearchXL model

Why Ladbrokes is decentralizing their CRO operations

CRO vs. CXO vs. Experimentation