‍Author: Martin P.
Title: Content Marketer

EP 50: Multi-Armed VS A/B Testing Blueprint

If you want to increase your success rate, double your failure rate — Thomas J. Watson, Former Chairman and CEO of IBM

Yo.

Martin P. from Speero here.

If you’re new, welcome.

This is the place where we share the hottest blogs, podcasts, events, and blueprints from the world of experimentation. Now, let’s see:

This Week in Experimentation:

Blueprint of the week: Multi-Armed VS A/B Testing Blueprint — the first is great for seasonality and low-on-time testing, the latter for deeper learning.

Talk of the week: The Magic Of Ancillary Revenue with Jason Keough

Some industries, like airlines, get most of their profits from ancillary revenue. Jason Keough joined Lorenzo Carreri and Rishi Rawat and talked about the types of ancillary revenue he tested and which you can add to your Ecommerce biz, fear-based messaging, sales pitches for upsell and cross-sell pages, and more.

Read of the week: The Ultimate Playbook to Experimentation for SaaS by VWO

Where is SaaS without great UX and optimized LTV and ARPU? The Ultimate Playbook to Experimentation for SaaS is a book loaded with inspiring, real-life SaaS examples that will teach you about the experimentation strategy and KPIs you need for each stage of your user journey — Acquisition > Activation > Monetization > Retention > Upsell > Referral. 

Event of the week: Opticon by Optimizely

The combined power of data-driven precision and the capacity for creative execution is what makes great things happen fast. Opticon 2023 will explore the intersection of art and science – how the two work in tandem to drive better digital experiences.

From cutting-edge technologies to cheat sheets for success, their diverse community of industry leaders will showcase the latest trends and best practices that blend creativity with analytics to drive growth. Happening on Oct 10-12, in San Diego, CA, at Marriott Marquis San Diego Marina.

Speero Story of the week: 5 Interviews, 20+ Insights — We interviewed the sales team to explore what troubles the customers and them, and uncovered a cornucopia of insights (Read the whole story below)

Blueprint of the Week: Multi-Armed VS A/B Testing Blueprint

Multi-Armed VS A/B Testing Blueprint is a guiding tool on when to run a multi-armed or a true A/B test. 

A/B testing allows for a more statistically controlled learning environment, while MAB is more focused on generating a win as quickly as possible (at the sacrifice of understanding 'why'). 

MABs are good for holidays, low on time, and seasonality testing, while the A/B test provides a deeper insight into what went good or bad in your tests.

Use Cases:

— Decide if and when to use MAB or AB.

Speero Story of the Week: Research II — 5 Interviews, 20+ Insights

Speero’s client was losing sales, revenue, and sales resources.

We interviewed the sales team through 5 interviews to explore what troubles the customers and them, and uncovered a cornucopia of insights.

These were our key takeaways:

1 — Tool consolidation, Pricing, and features/functions are the core motivating topics for the client’s customers. Customers want a tool to consolidate their options into one tool, with features focused on their and the industry’s needs.

2 — Features/functions are a major area for product improvement since they’re the main reason the client is losing customers.

3 — Some users require technical assistance or help with plan selection, but contact sales via the website. This wastes sales resources. A better CTA and support service can help avoid this and improve customer experience.

4 — It’s not clear how the client is better/different than their competition. Many leads ask about this from sales.

5 — Experiment with more form fields to improve lead quality and enhance your sales call experience with info appropriate to their industry and needs.