Author: Martin P.
Title: Content Marketer
EP 34: Experimentation Meeting Cadence
Creativity requires input, and that's what research is. You're gathering material with which to build — Gene Luen Yang
Hi there.
Martin P. from Speero here.
To all new people reading this, welcome.
This is your place to get the latest in research and experimentation.
From interaction effects and impact of experimentation on conversion rate, to how to sell your A/B testing program and UX research, here’s:
This Week in Experimentation:
Blueprint of the week: Experimentation Meeting Cadence. A clever little blueprint that suggests a ritual pairing of content and timing. Link.
Talk of the week: The Impact of Experimentation on Conversion Rate — Jonny Longden is here to explain what’s the issue with the impact of your experiments and what to do about it. Link.
Read #1: 3 Ways to Sell Your Org on Experimentation — start by finding someone who believes in experimentation or can benefit from it. Link.
Read #2: Interaction Effects in Online Experimentation — explore two types of interaction effects and their potential impact. Link.
Read #3: How to Find Optimization Opportunities in Analytics Data — Experience and intuition can only take you so far, so here’s our guide to using Analytics to find optimization opportunities for your website. Link.
Opinion of the week: How to get buy-in for UX research by Hannah Shamji. Link.
Event of the week: inOrbit — 30 speakers, 30+ sessions, 2 workshops on UX, data, and experimentation. Link.
Blueprint of the Week: Experimentation Meeting Cadence
This is a clever little blueprint that suggests a ritual pairing of content and timing.
Weekly — The Test and Learn Ritual.
— What roadblocks have you got this week?
— How can you move past blockers and influence test velocity?
— How to ensure everyone is on track?
— All on a quite tactical level.
Monthly — The Planning and Process Ritual
— This is your audit of the planning and process, and the flywheel itself.
— Do all parts of the flywheel work properly?
— How can we address them?
— Document issues?
Quarterly — The Assessment and Integration Ritual
— Here you consider the input and output sides of the process.
— What are your OKR-style big initiatives and rocks?
— What customer research activities you wanna do to build a better testing program?
— What kind of research do you want to do?
— What kind of big themes around customer problems or customer opportunities do you want to address?
— How does it integrate with the business's growth model and strategy?
Quarterly — The Decision and Action Ritual
— Here you consider the quality of decision-making.
— Who is making the decisions and how?
— In what priority?
— How quickly are decisions being made?
— What are the cumulative results of our decisions in the last quarter?
Experimentation Meeting Cadence Blueprint is a nice combination of how and when you need to have touchpoints with your team, and which topics you need to cover.
Use Cases:
— Create systems of feedback with your team
— Impact the culture of your program or org
— Train and educate your team
— Improve accountability systems
Talk of the Week: Why Don’t We See Experimentation Program Impact on Bottom-line?
Jonny Longden is here to explain what’s the issue with the impact of your experiments and what to do about it.
There are 3 reasons why you aren’t seeing the results:
1 — You’re doing experimentation wrong (Check how you do statistical analysis)
2 — You can’t see incremental changes to volatile trends
3 — The UX of the website is not the primary force impacting conversion
4 — Experimentation is about probability and confidence
5 — Experimentation is still the best possible to make decisions and de-risk investment
90% of all website changes are a waste of time and money.
But that 10%, oh man.
Reads of the Week:
Read #1: 3 Ways to Sell Your Org on Experimentation
When trying to influence a big cultural shift in your company, don’t go alone.
Find those who believe in your vision of experimentation and you won’t stand alone in front of the executives.
But maybe your company hasn’t even started going online. Now what?
You’ll have to appeal to how experimentation solves other people’s problems. How can experimentation make their lives easier?
Once you have that understanding, you can focus on making three distinct appeals:
— Experimentation is proof of value
— Experimentation is a learning opportunity
— Experimentation is hygiene
Read #2: Interaction Effects in Online Experimentation
As push more tests and bring more scale, you get more and more risk of interaction between tests.
In some situations, the interaction can happen from a functional conflict between two changes. They aren’t compatible, causing a variation in effect.
In other situations, the changes can be functionally compatible but interact nonetheless, changing data or user behavior.
Read on to find out two kinds of interaction effects and their potential impact.
Read #3: How to Find Optimization Opportunities in Analytics Data
When it comes to experimentation, nothing’s more important than data. Experience and intuition can only take you so far, so here’s our guide to using Analytics to find optimization opportunities for your website.
Read on and find out how to discover your high-impact pages, identify potential technical or usability issues, fix broken links, compare Landing Page metrics, and more.
Opinion of the Week:
By Hannah Shamji.
“When I was head of research at Copyhackers agency, all I heard was No No No to customer research:
-We don’t have time for research.
-We just want the copy.
-We already have that information.
-We already know our customers.
If clients didn’t want research.... was I going to be out of a job?
It’s been 3 years since then.
I’m no longer at Copyhackers (I was never a 9-5 kind of girl), and I’ve been successfully selling 5-figure customer research packages for the last 2 years.
Don’t get me wrong, the resistance to research is alive and well.
Lots of businesses I talk to still see it as a time suck.
But I'm still able to sell research rather consistently.
....How?
I’m not special.
I learned the hard way. Most of us do.
But the only reason I persisted to figure it out was thanks to a little moment of serendipity.
One day on the call with a Copyhackers client, I asked if they’d done research before.
The client responded:
“Yes, we just paid a couple of guys $10K to run a survey… I’ll send you the results”
I was shocked.
I’d never heard anyone pay that much for research, let alone pay for only research.
But if one company was willing to pay for research AND pay a separate company for copy….. why not others?
What was I missing?
Turns out I was missing a few things, which I’ve since figured out... and I want to share them with you.
This month I'm opening up my playbook on how I pitch customer research to get the yes.
Selling customer research isn’t a topic that gets much spotlight, but it should”
Link to original LinkedIn post.
Event of the Week: inOrbit
16th-17th March 2023, Grand Hotel Bernardin, Portorož, Slovenia.
inOrbit is more than just a conference on digital marketing - it's a gathering of experts with one goal in mind - growth.
Two tracks - Strategy and Deep dive - are combined with unconference events and panels, with sessions on personal and business growth, some focusing explicitly on agencies.
A large chunk of time is dedicated to networking, mingling and once again enjoying a live event.
Enjoy topics such as:
— How you can make your marketing more effective in tough economic times by harnessing behavioral science.
— Turning CX data into business intelligence — Automated Verbatim Analysis in Digital Transformation.
— The Decision Economy.
Event link.
.svg)
