Experimentation Program UK, US & AU

Who’s the experimentation program for?

What industry?

We can apply our scientific methods to any industry. What’s important is that you want to run better experiments, understand your customers better and make better business decisions.

What size?

Our experimentation program is for medium to large enterprises. Most of our clients are already running experimentation programs and looking to scale, or want to start one using the right processes. You’ll need at least 100,000 unique visitors per month to your website for us to run our program.

If you’re a smaller company, check out our conversion research service.

What’s the commitment?

Our full-service experimentation programs start from 10,000 USD per month on a 12-month program. This includes everything from analytics audits and experimentation program maturity assessments to user research and test execution. You’ll work alongside our team of experimentation strategists, designers, developers, QA, and project managers to execute your program.

What will my experimentation program look like?

Discovery

We’ll audit your testing and analytics tools to ensure we have reliable data. We also learn more about your business and the current process for running tests.

Kick-off

We bring together people from across your business, sharing knowledge about the program, and aligning the experimentation objectives with wider business goals.

Experimentation Maturity Audit

We evaluate your experimentation program in terms of;
- Strategy & culture
- People & skills
- Process & methodology
- Data & tools.
‍
From this, we provide guidance on where and how you can improve your experimentation practices.

Training with CXL

We give your team access to CXL training courses to help fill any gaps in their optimization know-how.

Research

A range of research methods are used based on our research aims. Insights drawn from the research are used to create hypotheses for testing.

Prioritization

Using the PXL framework we objectively prioritize the hypotheses and create your testing roadmap.

Ideation sessions

We develop creative test treatments applying UX design principles and consumer psychology to give your experiments the best chances of success.

Ongoing experimentation

We get down to testing straight away in the first month of working together. Going through cycles of implementing tests, QA-ing, and post-test analysis. We’ll share the results in experiment scorecards, or an experiment knowledge base, that we help you create.

Experimentation Case Studies

$1.5 million in additional revenue

For Procter & Gamble brand, Native, by experimenting and improving their customer experience.

0-100 experiments

We helped MongoDB embed an experimentation culture that took them from no testing to running 100 tests per year.

$2M additional revenue

A single research-backed recommendation for RevZilla delivered $2M in annualized revenue.

“I’d gone on your site and read everything. I thought these guys know what they are talking about. We had previously engaged with some of the biggest UX and CRO agencies in the country but it felt like they hadn’t even been on our website. Speaking to you guys, you’d already identified some issues on our very first call. We knew we were talking to the right people.”

Alexander Robinson, Director, PMO at RevZilla

Experimentation Program FAQs

How long does an experimentation program take?

Experimentation should be an ongoing process core to your business, ideally used across teams and departments. Our full-service experimentation programs are a 12-month commitment over which time we not only help you run tests but help to embed the function within your business.

When is the best time to start an experimentation program?

This question is best answered by understanding how much it’s costing your business while you aren’t validating ideas. The costs come from the time and resources spent working to implement ideas that at best have no impact, or at worse negatively impact your business metrics.
‍
Additionally, there’s an opportunity cost. While you aren’t experimenting it’s likely your business is more cautious to try new ideas, and therefore the opportunity cost arises from not finding new ways to operate online. Your website traffic cannot be retroactively used to validate ideas so the sooner you start the more ideas you can validate and uplifts you can gain.
‍
While there are many ways you can experiment and validate ideas that don’t require website traffic, our website experimentation program primarily uses A/B testing. Therefore the best time to start is once your business has the necessary amount of traffic (100,000+ monthly unique users) to run statistically valid tests.

What results can I expect to see from experimentation?

The answer to this depends on the types of tests you are running and the focus of your experimentation program. If you are running basic UX tests you will likely see uplifts in conversion rates. However, you may find you reach the “local maximum” fairly quickly depending on how optimized your website is.
‍
If you are running strategic tests which are tied to business metrics you are likely to see larger uplifts that go beyond your local maximum, that impact areas such as profit and retention. Alongside validating business ideas and learning about your customers or users.

How often should I run experiments?

Ideally, all of the time in an ongoing fashion. The velocity of tests you can run depends on several factors, such as;
‍
● The volume of traffic you have to your website.
● Any blackout periods your business may impose where testing must be paused.
● Any technical work that may impact your ability to test.
● The resources you have available.
‍
Mature experimentation programs look to utilize as much traffic as possible, aided by refined processes and automations. However, the types of tests you are running and the journeys they impact need to be kept in mind to avoid issues with conflicting experiences for the user and mudding your results. Setting up swimlanes for your testing roadmap can help orchestrate high-velocity experimentation programs.

What is experimentation culture?

Experimentation culture, sometimes also called a “test and learn” culture refers to the mindset and subsequent practices within a business. Businesses who have a good experimentation culture;

● Use data to inform decisions.
● View failed experiments as an opportunity to learn rather than “a failure.”
● Make everyone feel safe to call out decisions that are solely based on opinions especially from HiPPOs (Highest paid persons opinion.)
● Have teams across the business involved in running experiments.
● Have leaders who frequently and repeatedly champion the experimentation mindset.
● Dedicate the necessary resources to support experimentation functions.

Full-service experimentation program