Speero Experimentation Blueprints

Experimentation Operating System (XOS) Blueprints help visualize organizational processes in order to optimize how a business delivers an experimentation program.

They have two parts: 1. they are decision support tools that are built on top of and customized, and 2. are connected with some program or business metric such as research velocity, or decision quality, speed, etc.

We present them as downloadable 'tools' (Figma, Miro, Decks, Docs, Sheets) for you to take, customize, and optimize your program with.

Filter blueprints by pillars of the XOS:

Hypothesis Testing vs “Do No Harm” Treatments in A/B Testing

"Hypothesis Testing" and "Do No Harm" treatments represent different experimental goals, hence it is important to differentiate them to ensure correct statistical analysis and interpretation of results.

View blueprint »

Where and how should I test to make the most money?

Based on my website's traffic and conversion numbers, how many experiments can I run at each step of the user journey?

View blueprint »

XOS Tool Stack

Get to know the most common software categories and tools that professional experimentation teams use to perform their job.

View blueprint »

User research methods framework; which method is best suited to your research objective?

This is a framework to support you in planning which research/validation method is best suited to your business needs.

View blueprint »

User- or Session-based Test Analysis?

The metric you choose for doing a post test analysis matters a lot, while most experiments use session counts as the default metric for test analysis, this blueprint helps you clearly understand the different scenarios i

View blueprint »

Client vs server side testing tools - pros and cons

There are pros and cons to both methods of experimenting, client- and server-side. Ideally, you'd be able to experiment with both.

View blueprint »

How to engage community and create culture around experimentation?

What works to change people and ways or work, is different depending on the org. We need to experiment with ways to engage and train our teams.

View blueprint »

Should I run an AB test?

Should you actually run that A/B test? Tests are expensive, make sure you know.

View blueprint »

What's in a strategic testing roadmap?

A strategic testing roadmap is the culmination of research and the basis of a great OKR-style action plan for a testing program.

View blueprint »

Cadence for experimentation meetings?

Experimentation is a process, so there needs to be periodic meetings for different reasons.

View blueprint »

Results vs Action

It's important to separate the concepts of 'result' from 'action' so you can independently track how your program is doing

View blueprint »

How to balance test velocity vs complexity?

How do you balance test velocity and test quality? This relationship is at the core for how effective your testing program is running. It's the eternal dance of speed vs certainty.

View blueprint »

3 Methods for Confirming Test Effects

Is the result you found in a test repeatable. THIS is the gold standard of scientific research.

View blueprint »

Problem-Statement Focused Hypothesis

This is a framework to help ground experiment ideas (or solutions) in research, utilizing 'problem statements' as the bridge.

View blueprint »

Research XL

Research XL Our quantitative and qualitative conversion research methods identify the fears, frustrations, and motivations people experience throughout your website journey.

View blueprint »

Multi Armed Vs A/B Testing

This framework is a guiding tool on when to run a multi-armed bandit vs. running a true A/B test.

View blueprint »