Author: Martin P.
Title: Content Marketer
EP 35: How to Balance Test Velocity Vs Complexity?
I believe in innovation and that the way you get innovation is you fund research and you learn the basic facts — Bill Gates
Yo yo,
Martin P. from Speero here.
To all new folk reading this, welcome.
This is your place to get the top experimentation news.
The older emails were rather too long, weren’t they?
Let’s try a new format:
One read.
One talk.
One event.
One blueprint — but an in-depth explanation of it.
Time to show…
This Week in Experimentation:
Blueprint of the week: How to Balance Test Velocity Vs Complexity? Speero’s fav blueprint to benchmark, understand the progress and optimize experimentation programs. Link.
Talk of the week: Next level A/B Test Planning & Prioritization — to run an effective program, you need more than just ad-hoc picking the best tests. Link.
Read of the week: The CXL Guide to Customer Churn: What Is It and How To Reduce It — Including how to measure, define, and 10 ways to reduce churn. Link.
Event of the week: The Road to Mastery — or how to win in B2B and rule the content game, even though generative AI is trying to stop you. Link.
Blueprint of the Week: How to Balance Test Velocity Vs Complexity?
Test Complexity vs Velocity Blueprint is probably one of Speero’s favorite frameworks to benchmark and understand the progress and state of our clients’ experimentation or CRO programs.
This blueprint tells the story of two opposing forces:
Better tests with more impact
VS
Running more tests
This is a neverending battle.
You can’t have the cake and eat it.
Every program manager, CRO expert, or experimenter is constantly hitting themselves over the head with hitting the right balance between complexity and velocity.
Should I push more tests, regardless of their quality?
Or should I focus on bigger, bolder tests?
The key here is, unsurprisingly, balance.
You should be constantly moving toward the upper right side of the graph.
So, you run more tests. But…
As you run more tests, you want to slowly build the capability to run more complex tests and to run the more efficiently every time you jump a certain ‘hurdle’ on this graph.
As you add server-side testing or dynamic personalization, you learn and develop the process to do this easier and faster next time.
This way, you’re building velocity AND complexity simultaneously.
What is also cool about this blueprint is that it's connected to the Results Vs Action Blueprint: a blueprint that deals with the test results.
You have the test result — loss, win, flat, error — what now?
Usually, when there's an error it's because you pushed on complexity, which is a good thing. So that error should map explicitly to the y-axis of the test complexity.
You should also list WHY you got that error.
Maybe you’ve added metrics you didn’t understand enough or account for, and they failed within the tests’ setup. Yes, you have a QA, but you didn’t account for this weird browser or audience segment.
As you add complexity, account for that complexity.
Learn from the process, even when you fail.
Failing is learning and all.
Talk of the Week: Next level A/B Test Planning & Prioritization
Planning and prioritizing experiments requires more than just thinking about which is the best test to run.
For an effective testing program, you need a system you can measure, monitor, and show accountability for your testing roadmap.
In this 15-min interview, Shawn David, Operations engineer from Speero, will share how Speero researches, test ideas, turns insights into actions, and run experiments.
Check out the video to:
— Get an inside glimpse into Speero's Experiments, Insights, Action planning and prioritization methodology.
— Understand why "receipts," or keeping a track record of test ideas is crucial for managing client relationships.
— See how this tool, built can be used to develop or enhance your own test planning and prioritization model.
Read of the Week: The CXL Guide to Customer Churn: What Is It and How To Reduce It
You’ve acquired a ton of customers lately for your company. On the surface, this is awesome. More customers, more money. So you throw your energy into your customer funnel.
However, soon after sign-up, they seem to fly right out the back door.
Why is this happening? Why do customers leave—or use the service less—often without saying anything? What you’re experiencing is customer churn.
Explore:
What is customer churn?
What is the churn rate?
Lagging vs leading indicators.
How to detect and measure churn.
10 ways to reduce churn.
Event of the Week: The Road to Mastery
How will B2B marketers keep up with the ever-evolving landscape of content marketing?
How will ChatGPT change your job or business?
How can you invoke creativity to create content no one can replicate, let alone compete with?
We’re excited to sponsor The Road to Mastery, a virtual summit presented by Omniscient Digital.
They’re bringing in marketing experts to explore the current trends in content marketing and how they're drastically changing the way businesses create engaging (and revenue driving) experiences with clients on an ever-evolving landscape.
This one-day virtual event will take place on March 23rd, and include:
— A panel on The Modern Approach to B2B Content Marketing with Devin Bramhall, Ali Orlando Wert & Anum Hussain
— A roundtable discussion (open to attendees) on How to Win in B2B with Kevin Indig, Peep Laja, Ashley Faus, and Allie Decker.
— An AMA with Jay Acunzo on How to be Remarkable in the Age of AI
The future of content marketing is here.
Event link.
.svg)
