Author: Martin P.
Title: Content Marketer
EP 40: Test Implementation Checklist
Maturity, one discovers, has everything to do with the acceptance of ‘not knowing. ― Mark Z. Danielewski
Hi and hello.
This is Martin from Speero.If you’re new, welcome.
This is the place where we share the latest talks, reads, blueprints, and events in experimentation.
If you want me to share your podcast or blog post, just reply to this email or reach out to at Martin@speero.com.
Before we jump into this week, there’s something new me and Ben Labay, CEO, decided to test.
Speero wants to give you more valuable content in our newsletter.
More value, more education, more of anything for you, really.
Something you can’t find in the first Google Search.
Since Speero released Experimentation Program Maturity Benchmark Report for 2023, why don’t we start the give-value series with how to mature your experimentation program?So, in the next dozen of emails, besides regular content, we’ll explore how you can mature your experimentation program across all four key pillars:
- Strategy and Culture
- Data and Tools
- Process and Governance
- People and Skills
Now let’s explore:
This Week in Experimentation:
Maturing of the week: [Strategy & Culture > Culture > Leadership Buy-in] How much support you got for experimentation and how to drive it further? Read bellow.
Blueprint of the week: Test Implementation Checklist — Your plan of attack when you have to change ‘business as usual’. A flicker machine to ask yourself if there’s the tech and buy-in you need even before you run the test. Link.
Talk of the week: What’s a Good CRO Budget and 5 Ways to Reduce Bounce Rate — strike a balance between investing for strong results, avoiding overtesting, optimizing layout, and more. Link.
Read of the week: Bridging the Gap Between Google Analytics UI and BigQuery Export — why don’t the numbers match with the UI? And where does it say that? Link.
Event of the week: Data Dawn after the Google Optimize Sunset: How to Store, Analyze & Query Better — You’ll have to transfer to a paid platform. Treat this as a switch as a fresh chapter. Link.
Speero Story of the week: Add Search to Cart? When we add a search bar in the cart and focus on a gain or loss perspective. will this lead to more add-to-cart and AOV? Read bellow.
Maturing of the Week: Leadership Buy-in
Pillar: Strategy and Culture
Subpillar: Culture
Activity: Leadership buy-in
How much support from Leadership do you have for experimentation?
— At Beginner Maturity Level, HiPPO is rampant, and “Just do it” is the typical reply you get. Also HiPPO’s might be scared to step away from decision-making without their approval or opinion.
— At Aspiring Maturity Level, those few employees interested in experimentation aren’t in positions to drive an efficient program. There is no budget for resources, tools or education to do better Experimentation. Management and the 'powers that be' tend to want to work faster, or not have opinions challenged. Leadership does think about experimentation, but only as an afterthought.— At Progressive Maturity Level, management supports and drives the growth of the experimentation program.
— At Strategic Maturity Level, data is championed at every level of the company, from the CEO to customer service representatives. Management supports and drives the growth of the experimentation program.
— While on the highest, Transformational Maturity Level, leadership sees the value in experimentation and supports the efforts. You have a documented customer experience vision that is shared across the whole organization. Experimentation is publicly discussed, and the value of experimentation is praised at every level of the organization.
How Can You Level Up?
Internally:
— Develop and/or nurture a data-driven culture. For example, standardize SOPs and processes, data collection, data transparency, choose your metrics carefully, etc.
— Empower managers to support and grow the experimentation program. For example test one of their ideas, show how it works, the process, show the benefits, the risk assessment.— Empower management and leadership to find a shared customer experience that the whole organization is unified in optimizing.
Externally:
— Use experimentation maturity audits and benchmark reports to show issues, bottlenecks, and places to improve. For example, in our latest benchmark report we found out that 60% of companies don’t have clear incentives for experimentation.
— Champion data-driven culture with an external agency. External agencies already have data-driven culture and experience in developing the quickly and efficiently.
— Use external help to establish processes to document shared customer experience vision.
Blueprint of the Week: Test Implementatation Checklist
So you have a winning test.
Should you implement it?
Don’t just straight to yes.
Not all changes are good.
Especially if they involve changing the process for several teams or serious stakeholder buy-in.
Test Implementation Speero Blueprint gives you a plan for attack when you have to change ‘business as usual’.
More than anything, this is your ‘flicker’ mechanism.
Something you check on every once in a while and ask yourself a series of questions about the technical feasibility and buy-in you need even before the test is scoped.
Use Cases:
— As a checklist when you’re setting up SOPs and processes for test implementation in your program.
— Change is not always easy, especially if it involves a change in process for multiple people or teams. This blueprint provides a plan of attack for changing 'business as usual'.
Feel free to use it and tweak it based on your specific needs. There are no one-size fits all models.
Talk of the Week: What’s a Good CRO Budget and 5 Ways to Reduce Bounce Rate
In EP 30 of CRO Live Hour, Simbar Dube and Khalid Saleh discussed 5 effective ways to lower bounce rates:
- Improve site speed
- Optimize design and layout
- Provide clear and relevant content
- Optimize CTAs
- Target the right audience
Then they discussed what’s a reasonable budget for testing and optimization efforts — striking a balance between investing enough to create meaningful results and not overspending on non-essential testing.
Read of the Week: Bridging the Gap Between Google Analytics UI and BigQuery Export
"But why won't the numbers match with the UI?"
If you have worked with the BigQuery event export data for your GA4 property, you definitely have asked this question at some point.
Or worse - someone else asked you this.
And while trying to answer it, you probably have been asked the dreaded follow-up question:
"And where does it say that?"
With this article, Minhaz Kazi, Dev Advocate at Google, will try to shed light on both.
Speero Story of the Week: Add Search to Cart?
Scorecard:
Learning — Overall, adding a search bar and using the psychological principal of loss aversion in the cart led to more searches and additions to carts.
But fewer visitors finalized their purchase which led to a drop in transactions.
The test performed the worst on mobile devices (significant drop in RPV and transactions).
Next steps — Don’t implement. The search bar was used but might have been distractive in the cart, plus it pushed the call-to-action to checkout down.
An option here to iterate and test in the opposite direction: remove distractions like the menu and search bar.
Or to improve the search experience and try again.
Event of the Week: Data Dawn after the Google Optimize Sunset: How to Store, Analyze & Query Better
Deborah O'Malley of GuessTheTest, Glenn Schmelzle of the Funnel Reboot podcast, and Jeroen Wiersma who specializes in developing A/B tests (and measuring their impact) will shed light on what brands can do to clean up their data house during and post the GO migration.
Link to event.
.svg)
