‍Author: Martin P.
Title: Content Marketer

EP 50: Multi-Armed VS A/B Testing Blueprint

No amount of experimentation can ever prove me right; a single experiment can prove me wrong. — Albert Einstein

Yo.

Martin P. from Speero here.

Nothing much to say except to add our newsletter to your primary tab if it keeps ending in your promotional tab. 

This Week in Experimentation:

Blueprint of the week: Problem-Statement Focused Hypothesis Blueprint — ground your experiment ideas (or solutions) in research, utilizing 'problem statements' as the bridge.

Talk of the week: It Depends EP 2, Product Videos

Everywhere you look, video content is popping up on product pages and content channels. But is time and money-intensive video production truly necessary? On “It Depends,” Paul and Shiva weigh in by explaining that while there’s no clear answer as to whether you should add product videos to your product pages, you can make an informed decision by considering your audience, your budget, and your product itself.

Read of the week: Hit Big Hairy Audacious Goals by Getting Aligned

“BHAG,” which stands for Big Hairy Audacious Goals, are very ambitious, compelling, and long-term goals (think ten+ years), but still create urgency in your company today. Examples include:

— Best Buy’s BHAG is to move from an in-person shopping experience to becoming a digital leader in technology. 

— Nike’s original BHAG was to crush Adidas.

But how do such goals relate to the work of the experimentation team? Let’s dive into this topic to understand what’s going on. 

Event of the week: Nonprofit Innovation and Optimization Summit 2023

NIO Summit is a deep dive into online fundraising and how to build and optimize a digital-first fundraising program for your organization. In short, how to acquire more supporters, convert more donors, and unleash the generosity of your audience. 

At NIO Summit, you’ll discover how to write better copy, design better landing pages, run more effective social campaigns, track and analyze results, and much more. You’ll also learn a lot about testing and optimizing and the impact that can have on your cause. 

Speero Story of the week: Testing XV — Gating content is cool. But have you ever tried to give your users a peek at it to increase downloads? Here’s how Speero tested this and increased downloads by 40%. (Read below)

Blueprint of the Week: Problem-Statement Focused Hypothesis Blueprint 

Problem-Statement Focused Hypothesis Blueprint helps you ground experiment ideas (or solutions) in research, utilizing 'problem statements' as the bridge. 

This enables you to ensure your tests focus on problem statements, which are grounded in research and allow for alternate 'solutions' to be proposed as long as they are both grounded in the same hypothesis (and problem statement).

Let’s say you have a concrete, tactical test idea. 

With this framework, you can put this idea into the solution part, and then find your hypo and if statement (from that idea). What do you believe will happen if you implement your idea? 

Now, it’s time to take a step back. What is your problem statement? Where is the evidence that your test idea is really a problem? Most of the time, you can back this in research.

Use Cases:

— Prioritize your tests based on business needs.

— Connect your solutions to business problems.

— Get buy-in for experimentation.

— Focus on the most important user problems.

You can also link these problem statements to the business. Use them as an opportunity to understand what your business is trying to prioritize. 

This way, when you present in front of the leadership, you all can collectively agree on which three problems should be addressed first, instead of having a bunch of solutions backed by hypotheses.

Speero Story of the Week: Testing XV, A Peek to Motivate Them

Gating content is cool. But have you ever tried to give your users a peek at it to increase downloads? Here’s how Speero tested this and increased downloads by 40%.

Whitepapers/Resources are common among B2B sites.

They ask a lot without the audience knowing if there's enough value inside.

Speero’s client was a B2B SaaS brand.

The client's resource pages had a mixed performance, so Speero analyzed them to identify possible causes and solutions.

During the analysis, we noticed that the best-performing 'report' pages showed a page of the gated report in the hero. Other lesser-performing pages showed no content.

This insight inspired the following hypo.

“If we display some of the content of the resource it will increase motivation and value of the downloadable resource and persuade more visitors to provide their details.”

The results were outstanding.

Variation resulted in:
+40.66% increase of Gated Content Downloads at 97% CTBC. 
Uplift: +30 downloads/month for just these three pages.

Displaying part of the report motivated users to provide their details and download the report.

Even more, the form wasn’t visible on mobile immediately, but downloads improved.

We decided to implement the test and test the same hypo on other pages to see if we can replicate the results.