Experimentation
Program
Maturity Audit

The experimentation program audit is a classic program management tool, meant to ask great questions to help find gaps in the efficiency or effectiveness of, in this case, experimentation programs.

Our experimentation program audit covers four main pillars or areas that are crucial for a successful experimentation program.

Fill in the audit for free and you'll get your results at the end.
"Before we did the audit it was already clear that we have a long way to go, but we didn't have enough clarity to decide on what to do next. Doing the audit gave us the map we needed to bring our CRO efforts to the next level in the coming year."
Bas Boland,Vitaminstore

What does my experimentation maturity audit score mean?

The overall experimentation score corresponds to different levels of maturity, set using our 10+ years of experience in testing and optimization. The score corresponds to the following levels of maturity. If you want to discover how to increase your experimentation program maturity, check out our maturity audit service which helps you define the plan to get you there.

Level 1: Beginner

An overall score of 0-20%
Businesses at the start of their experimentation journey. Few of the fundamental building blocks needed to run an effective experimentation program are implemented.

Level 2: Aspiring

An overall score of 21-40%
Businesses that have established some of the important elements needed in preparation for running an effective experimentation program. These businesses typically have many internal hurdles to overcome and practices to implement to run a successful experimentation program.

Level 3: Progressive

An overall score of 41-60%
Characterized by businesses that are starting to recognize the importance of insight-driven experiments and the need to improve their processes to increase the performance of their work. They have the necessary foundational elements in place to run a basic experimentation program.

Level 4: Strategic

An overall score of 61-80%
Businesses that have most of the foundational and some advanced practices in place, employing a strategic approach to experimentation. They are likely to have wider company buy-in for experimentation as a core business growth driver due to the results from their work.

Level 5: Transformative

An overall score of 81-100%
These businesses are the industry elite. They are outperforming their competition through a well-oiled experimentation program that is consistently delivering results.

What are the four pillars of an experimentation program?

Strategy
& Culture

Strategy & culture is a hugely important area for businesses, but an area that is often not addressed until later on in the development of your experimentation program.

We see this a lot: individuals often start out experimenting in small ways, with staff members (often in marketing or product roles) using some of their working week to allocate to this practice. It’s usually not until later on, once fundamental experimentation processes and tools are in place, and results are beginning to be seen that the wider business gets on board, and experimentation is more widely considered across all functions.

The problem with this approach is that it often leads to failure. Without senior buy-in which provides:
Funding for tools and research
A dedicated team and the right team structure to assist the rest of the business with testings.
The authority to test strategic elements of the business and a culture that accepts failure and learnings, leading to bigger wins and more insights and learnings.

Without the above, many programs fall by the wayside due to competing work commitments or a lack of specialist expertise needed to run a program that delivers impactful results.

Resources to help you with strategy & culture

How to Get Your Company to Really Care about Your Customers
6 Clever Nudges To Build a Culture of Experimentation
How to Nail Your First 90 Days as a CX Director

More about our maturity audit service

People
& Skills

This area of our assessment relates not only to the expertise needed to run effective experimentation programs but the structure of your teams.

While non-specialist individuals often start testing programs, it soon becomes apparent that specialist skills are needed to push beyond basic testing. This is one of the harder areas to nail down because:

● It can be hard to find people with the right skill sets.
● It’s hard to hire some of the needed roles if you don’t fully understand what skills are needed or how to test candidates for such skills.
● You need budget and buy-in for managers and leaders.

Even once you have conquered the skills gap, how you structure your team also plays an important part in how well your experimentation program will work. There are many approaches to this such as creating centers of excellence that can act as a central skills hub and resource for teams across businesses to run experiments - one of the ultimate goals for high maturity businesses.

Resources to help you with people & skills

How to Structure Your Optimization and Experimentation Teams
Cisco’s Experimentation Journey: From “Outsourced” to “Center of Excellence”

Full-service experimentation programs

Process &
Methodology

The foundation of experimentation programs, this area relates to how you;
● Conduct research to come up with hypotheses.
● Prioritize your ideas and conduct ideation sessions to create variants.
● Your testing workflow–how you manage ideas through to development, QA, calling tests, and post-test analysis.
● The metrics you set to evaluate the program itself.

It doesn’t matter how much money, skills, buy-in, data, tools, or strategy you throw at your experimentation program, without the right processes you’ll end up doing nothing more than spaghetti testing (throwing anything at the wall and hoping something sticks).

This tends to be how most businesses start out, running ad hoc tests based on little more than hunches. But after running your first few tests it often becomes apparent that this approach doesn’t yield results, especially not consistently and more formal processes are needed.

Resources to help you with process & methodology

A/B test calculator
How to Prioritize Your A/B Test Ideas with our free PXL template
Experimentation Metrics That Matter

Learn more about our research services

Data
& Tools

This is where most businesses start. Before they get a strategy in place, or processes, before they hire dedicated staff, they usually think about what A/B testing tool they need and sign up. But there’s a lot more to this than just getting an A/B testing tool, you need to;

● Select tools with the right level of functionality for your needs now and in the future, as well as understanding their limitations.
● Think about your tool stack as a whole, and the interoperability of data and integration between them.
● Think about how you’ll combat siloed data around your business.
● Consider the level of self-service vs. specialists expertise needed to use the tools.
● Plan how you will maintain the tools and keep them up to date.
● Set out how you will regularly check the validity of tracking and data from tools.
● Think about what and how will you make data accessible and usable for different people. Whether that’s giving data analysts access to raw data and BI tools, to dashboards for senior managers.

Having tools, and data you can use and trust impacts what you test, the validity of your findings, and whether your experimentation program is working. Without it, you’re flying blind.

Resources to help with data & tools

Pitching a Data Strategy? Here's How to Ensure the C-Suite Says “Yes.”
Data Discrepancies in Google Analytics: What Can Go Wrong, Why, & How to Fix It
Client-Side Vs. Server-Side A/B Testing Tools: The Difference

Learn more about our data services

Experimentation Program  Benchmark Report
2021

Our experimentation program maturity audit findings are out.

Over 210 brand-side experimenters shared where and how their businesses were performing in the 4 key areas needed to run an effective testing program.

The report gives you key findings across:
●  Strategy & Culture
●  People & Skills
●  Process & Methodology
● Data & Tools

Download the report to access:

  • What you can do to get ahead of your competitors with a list of easy-to-implement experimentation practices that most businesses aren't doing.
  • What the highest maturity level businesses are doing to get ahead, with advice on which areas you should focus on too if you want to increase the effectiveness of your testing program.
  • Your own free experimentation audit. Use this to compare your performance against other company's in the the benchmark report.

Key findings:

  • 14% of respondents strongly agreed that they share experimentation learnings across their company
  • 54% of businesses don't consider customer data or insights in decision making
  • Only 18% of respondents strongly agree that they prioritize hypotheses objectively
    ux experimentation program benchmark report 2021

    Get your free copy of the benchmarking report

    Your copy of the Benchmark Report is on its way to your email address right now.
    If you don't receive the report via email, please get in touch with katie@speero.com.
    Oops! Something went wrong while submitting the form.