‍Author: Martin P.
Title: Content Marketer

The Experimental Revolution aims to inform, educate and elevate the industry by sharing the latest in Experimentation learnings, and how these translate to better, faster businesses decision-making.

EP 3: How to Cook eXperiments With Most Impact

Yo.

Martin P. here.

Your frameworks host for the week.

Hot and sunny at my place.

Some would say too hot.

You could cook something on concrete.

That much hot.

While talking about cooking.

What interesting experiments are you cooking lately?

Feel free to share results in the replies.

Interesting enough, and I’ll share them in the next email.

By the way, you should be cooking wining or learning experiments all the time.

But what to cook?

What should you prioritize first?

Framework of the Week—PXL

‍

Prioritizing experiments means nothing if you prioritize based on hunches, guesses, and opinions. Speero wanted a framework based on being objective. 

To deliver maximum objectivity, we created our own PXL framework for prioritizing experiments. Grab your free template here.

With objectivity at its core, this model requires data to impact the scoring. It breaks down what it means to be 'easily implemented', and instead of taking a shot in the dark regarding the potential impact of your idea, this model quantifies the potential.

The PXL prioritisation framework asks you: 

• Is the change you're implementing above the fold? Such changes make a bigger impact, as they are more noticeable. 
• Will the idea run on a high traffic page? If it's a high traffic page, winning changes have the potential to make a bigger impact on your bottom line.
• Can people notice the change in less than 5 seconds? Ask your colleagues if they see the change. If it takes longer than 5 seconds it's likely to have less of an impact and if it’s not noticed let alone changing user behavior it’ll likely end up as an inconclusive test. 

Ideas that are derived from opinions have less chance of success - so it's well worth doing the research. That’s why our framework adds weight to ideas derived from data.

• Did you discover the issue through user testing?
• Did you discover the issue through qualitative feedback, like a survey?
• Is your hypothesis supported by heatmaps and mouse tracking?
• Are you addressing findings from digital analytics?

As these are all backed by data, you won't waste any time discussing ideas that are unfounded or based purely on opinion. We also put constraints on the ‘ease of implementation’ by setting brackets for different estimates. 

If you want to run a successful A/B testing program it's essential to prioritize your ideas in a way that's objective and based on facts, not opinions. Grab your free GSheet template.

Talk of the Week—Standard Deviations Podcast EP7: Flicker Mechanisms with Ben Labay

Ben Labay, my CEO, and Juliana Jackson, senior product manager at CXL, talk on evolutionary behavior, decision support, and how evolution impacts behavior in every way possible.

Why do we learn more about history than the present? Where human evolution and making mental models out of data intersect? What are flickers mechanisms you use every day without even noticing? Listen to Standard Deviation EP7 and find out.

Reads of the Week

Short list:

• 8 things I’ve learned building a UX knowledge repository
• When experimentation starts as a solution to raise ROI: The pitfall of not having the right scope for experimentation ROI
• Is it legal to run pricing tests?

‍

8 things I’ve learned building a UX knowledge repository talks on what the authors of Atomix UX Research, a self-managing, scalable networked system of knowledge learned in the last 4 years with it. For instance, how context is 99% of research repository, and how too much context makes it hard for you to zoom out. Or if you trust your data (and if you’re using it, you should), it is not your opinion, it is a fact.

When experimentation starts as a solution to raise ROI: The pitfall of not having the right scope for experimentation ROI is a Dutch study on how most experimentation starts in marketing departments looking to increase ROI, but also why most later broader experimentation scaling turns sour and actually lowers ROI. Explore how to avoid this Catch-22 and why you need more KPIs than just ROI to scale your experimentation.

Is it legal to run pricing tests? Is a part of Speero’s blog post on challenge of finding the optimal product price. Pricing is a forgotten skill/knowledge marketers used to have, but now rarely even talk about. For a reason. It’s a delicate subject. Especially online. You gotta be careful to not ruin your brand or customer trust. Even worse, break a law.

Event of the Week—Traffic & Conversion Summit

When: Tuesday, September 27 to Thursday, September 29 in San Diego, CA (USA)

Traffic & Conversion Summit has been running for the last 12 years and continues in 2022. In 2021, they added the option to attend virtually. It returns as the same hybrid event—hosting the same actionable content and fun networking as always. Expect to learn about:

• New traffic channels like Telegram, Discord, and Meta
• New selling models that actually make an impact
• New conversion breakthroughs from people doing thousands of landing pages
• Plus ultra-current breakout sessions with 8 different content jam sessions

Martin out. Like I said, give me a reply if you want me to cover something.

‍

FEEDBACK & COLLABORATION
Hope you have enjoyed this month's The Experimental Revolution. If you have any feedback on the newsletter or would like to collaborate with Speero on thought-leadership content, please contact Martin P. See you next month.     ‍

‍
Join the monthly conversation with the leading minds in Experimentation by signing up to The Experimental Revolution.‍