‍Author: Martin P.
Title: Content Marketer

EP 14: Problem-statement Hypothesis Blueprint

How to ground experiment ideas or solutions in research.
Experimentation for life.

Yo yo, Martin P. here.

A warm welcome to all new subscribers.

Every week, we blast what’s worth reading, listening, and visiting in the experimentation and CXO industries. 

And of course, Speero Blueprints™ for optimizing experimentation programs.

If you got a worthwhile blog post or a podcast you want to share, just reply to this email, and I’ll share it in the next episode.

Here is This Week in Experimentation:

Problem-statement Focused Hypothesis. Link.

There isn’t one “right way” to using A/B testing tools. Link.

How Hulu scaled experimentation through decentralization. Link.

How Financial Times turbocharged the value of their app for users. Link.

Wayfair created ML platform, Demeter, to speed up A/B testing. Link.

Customer Service Revolution event. Link

Blueprint of the Week: Problem-Statement Focused Hypothesis

Problem-Statement Focused Hypothesis Blueprint helps you ground experiment ideas or solutions in research, connecting them with ‘problem-statements’. 

This enables you to ensure your tests focus on problem-statements grounded in research, and allows for alternate 'solutions' to be proposed as long as they are both grounded in the same hypothesis (and problem-statement)

The best use case for this blueprint is that you use it as a guiding principle that ensures you’re running a program based on identifying insights found in research. 

You can also use this framework to keep yourself grounded when shifting to alternative designs (as long as they are grounded in the same hypothesis).

Link.

Talk of the Week: There is no one “right” way to approach A/B test tools

Chad Sanderson is the Head of Product for Convoy’s Data Platform team, a unique infrastructure team that takes experimentation from end-to-end. 

That includes everything from collecting and storing data to using it. 

Chad’s team owns both the internal machine learning system and the internal experimentation system, meaning they have built their own toolset from the ground up versus relying on third-party tools. 

Yet there is no one “right” way to approach A/B tools, and Chad has seen every scenario imaginable in action at his time at companies like Microsoft, Sephora, and Subway. 

In this episode of Testing Insights, Chad breaks down what tooling approach makes sense, and when.

Link.

Reads of the Week:

Hulu: How we scaled experimentation. Read how Hulu developed a Center of Excellence and the key actions they made to take their experimentation to the next level. 

See how decentralization allowed their marketers and product managers to scale experimentation, make decision-making into a scientific and objective process, and evolve A/B experimentation into a habit.

Link.

6 Lessons from rapid experimentation at the Financial Times. Here, Simi Agbaje explains how they turbocharged the value of the Financial Times app (which had 3X user engagement of their site) by making it into an even ‘stickier’ product for the app’s subscribers.

Link.

Speeding up A/B Tests with Delayed Reward Forecasting: Some experiments at Wayfair last two months and more. They wanted to speed up learnings from these experiments while they kept optimizing for long-term rewards. 

So they created a data science platform “Demeter”. Demeter uses ML models to forecast long-term KPIs based on customer activity.

Read their overview of Demeter and theoretical foundation in causal inference.

Link.