Author: Martin P.
Title: Content Marketer
EP 22: Should I Run Experiments Simultaneously?
Experiment your way to growth.
Hi there. Martin P. here.
Welcome to another edition of Experimental Revolution.
Your place for the latest trends, news, podcasts, articles, and blueprints in experimentation.
Here’s what this week has in store.
This Week in Experimentation:
Blueprint of the week: Should I Run Experiments Simultaneously? Yes. Here’s a blueprint for this, alongside how to define and then break strategic KPIs into smaller metrics that are easier to improve. Link.
Talk of the week: Stewart Ehoff and Shiva Manjunath explain why developers are often ignored in experimentation, even though they are one of the most vital experimentation resources. Also, how to get engineering resources on your side. Link.
Read #1: Show me the data. Members of a design team will often argue for different solutions to a UI problem. That's when to say "show me the data" to resolve the debate by empirical insights, rather than who argues the best. Link.
Read #2: CUPED. CUPED helps you with one of the biggest challenges in experimentation: measuring statistically significant results with less sample size. Link.
Read #3: The untapped profit in customer retention. Or how to experiment your way into customer retention and increased customer loyalty. Link.
Opinion of the week: How to triple your digit growth in 2023. Build a positive ROI CRO program in 90 days, use research and experimentation to uncover marketing positioning and more. Link.
Event of the week: Agency Resource Management Built for People & Profitability. Speero works with ClickUp's growth team in testing and shaping the program for how to use ClickUp itself and measure testing and CRO efforts on the website. This power-user partnership leads us to organize the upcoming webinar. Link.
Blueprint of the Week: Should I run experiments simultaneously?
One of the most asked questions ever: should I run experiments simultaneously? Generally, the answer is yes, you should.
If you don't, you face multiple worse 'evils'. But otherwise, there is always this decision tree.
Use Cases:
— Define the strategic KPIs that fuel optimization efforts
— Identify metrics crucial to the company's success
— Break down KPIs into smaller metrics where you can see improvements faster
Talk of the Week: Complex Test and Getting Engineering Buy-In
Stewart Ehoff, head of experimentation at RS Group, and Shiva Manjunath joined forces to explain why developers are often ignored in experimentation, even though there are zero mature experimentation companies that don't use them. Explore:
— How to Un-silo the value of experimentation.
— Why you should de-prioritize complex tests.
— To add or not to add test variations?
— Is the feature worth the risk?
— How to grow your experimentation program when engineering blocks you.
And of course, how to get buy-in for engineering resources.
Reads of the Week:
Read #1: Show me the data. Link.
Members of a design team will often argue for different solutions to a UI problem. That's when to say "show me the data" to resolve the debate by empirical insights, rather than who argues the best. Link.
Read #2: CUPED. Link.
CUPED helps you with one of the biggest challenges in experimentation: measuring statistically significant results with less sample size.
In an experiment, the standard error or “noise” goes down with the square root of your sample size. But the sample size is an expensive resource.
Even titans like Amazon and Facebook feel the pain of experiments to enroll users and mature because they’re usually looking for relatively small effects.
CUPED is an extremely popular implementation that uses pre-experiment data to explain away some of the variances in the result data.
Read #3: The untapped profit in customer retention. Or how to experiment your way into customer retention and increased customer loyalty. Link.
— How many customers buying from you today did you acquire last year?
— What value do loyal customers represent to your business?
— What experiments are you running to improve customer retention?
If the answers to these questions aren’t filling you with confidence, this article will help.
Opinion of the Week:
“Do you want to achieve triple-digit growth in 2023? Here is what you need to focus on:
– Build an ROI positive Conversion Rate Optimization program in 90 days
– Isolate and break down internal business silos and agency silos
– Conduct market research to understand direct and substitute competitors
– Use research and iterative testing to uncover your market positioning
– Leverage effective practices around data collection, data management, and data activation
– Take a holistic approach to optimize the end to end customer journey
– Focus your strategy by setting 12-month growth goals and 90 day KPIs
– Utilize step by step proven frameworks for analytics, research, and experimentation
– Create processes, workflows, documentation, and communication for scaling
– Share insights across your business to break down fragmented customer journeys
– Develop playbooks for creating a culture of growth
– Implement methods for becoming data driven and customer focused
– Draw expertise from case studies, business examples, and stories that will help you apply these in your startup
These are some of the most powerful levers that used to be a luxury, but are now becoming table stakes for 2023. Unfortunately, there are very few people focused on providing all of this so you can successfully use it for yourself
If any or all of these topics seem relevant, then you should start attending my new Triple Digit Growth LIVE training sessions each week. We will cover all of these topics in depth.
We will also be reserving at least 15 minutes per week for questions or topics from attendees, which we will thoughtfully and specifically answer to personally help you,”
Event of the Week: Agency Resource Management Built for People & Profitability
Speero isn’t only a power user of ClickUp (2+ years of usage plus hitting automation limit every month), but ClickUp is also a power user of Speero’s experimentation.
Speero works with Clickup’s growth team in testing and shaping the program for how to use ClickUp itself to set up, track, coordinate, and measure testing and CRO efforts on the website.
This power-user partnership leads us to organize the upcoming webinar:
— Agency Resource Management Built for People & Profitability —
This event is for you if time is crunching your agency all the time.
Or if you have trouble building the right systems that enable your teams to do their best work, without burnout.
Learn:
— How resource management fits into agency operations
— Why resource management is key to profitability and scale
— Actionable strategies to keep your employees (and clients) happy
You'll walk away with a blueprint to better manage team resources, streamline workflows, and make resource management more predictable (and profitable).
We hope you'll join us, register today to save your spot!
PS: Think someone else could use Speero’s CRO and Experimentation Blueprints™?
Share the newsletter with them.
.svg)
