Experimentation and Testing Programs acknowledge that the future is uncertain. These programs focus on getting better data to product and marketing teams to make better decisions.
We believe that research is an integral part of experimentation. Our research projects aim to identify optimization opportunities by uncovering what really matters to your website users and customers.
90% of the analytics setups we’ve seen are critically flawed. Our data analytics audit services give you the confidence to make better decisions with data you can trust.
From initial audits and assessment to user research and test execution. We not only run the full experimentation process, we also help you embed the experimentation function within your business.
“We knew we should be shipping a large volume of experiments but we didn’t know how. The first few experiments we tried missed the mark. But then we started working with Speero who came in and helped us with tooling, research, statistics, and the processes needed to run experimentation at scale.” Dan Layfield, Growth Product Manager at Codecademy
Who’s the experimentation program for?
What industry?
We can apply our scientific methods to any industry. What’s important is that you want to run better experiments, understand your customers better and make better business decisions.
What Size?
Our experimentation program is for medium to large enterprises. Most of our clients are already running experimentation programs and looking to scale, or want to start one using the right processes. You’ll need at least 100,000 unique visitors per month to your website for us to run our program.
Our full-service experimentation programs start from 10,000 USD per month on a 12-month program. This includes everything from analytics audits and experimentation program maturity assessments to user research and test execution. You’ll work alongside our team of experimentation strategists, designers, developers, QA, and project managers to execute your program.
What will my experimentation program look like?
Kick-off
We bring together people from across your business, sharing knowledge about the program, and aligning the testing objectives with wider business goals.
Discovery
We’ll audit your testing and analytics tools to ensure we have reliable data. We also learn more about your business and the current process for running tests.
Maturity Audit
We evaluate your performance in: ● CX strategy & culture ● People & skills ● Process & methodology ● Data & tools From this, we provide guidance on where and how you can improve your experimentation practices.
Training with CXL
We give your team access to CXL training courses to help fill any gaps in their optimization know-how.
Research
We kick off our first research sprint. These happen throughout your program. Different methods are used based on our research aims, including.
Insights drawn from the research are used to create hypotheses for testing, and inform the persona and customer journey mapping sessions
Ongoing customer research used to draw fresh insights and hypotheses.
Ideation sessions
We develop creative test treatments using UX design principles and consumer psychology to give your experiments the best chances of success.
Prioritization
Using the PXL framework we prioritize the hypotheses we developed in month one based on potential impact.
Optimization roadmap
Using the findings from the CX maturity audit, prioritized hypotheses, and the CX vision.
Ongoing Experimentation
We get down to testing straight away in the first month of working together. Going through cycles of implementing tests, QA-ing, and post-test analysis. We’ll share the results in experiment scorecards, or an experiment knowledge base, that we help you create.
Take the first step in starting and scaling your experimentation program
We helped MongoDB embed an experimentation culture that scaled from 0 to 100 tests per year within 6 months of working together. Read more »
$1.5 million in additional revenue
For Procter & Gamble, by experimenting and improving their customer experience. Read more »
“I’d gone on your site and read everything. I thought these guys know what they are talking about. We had previously engaged with some of the biggest UX and CRO agencies in the country but it felt like they hadn’t even been on our website. Speaking to you guys, you’d already identified some issues on our very first call. We knew we were talking to the right people.”Alexander Robinson, Director, PMO at RevZilla
Who will you be working with?
Work with Ben Labay, Managing Director
10+ Years in academic research, turned to experimentation program obsessive and host of the Testing Insights show.
Work with Emma Travis, CX Strategist
10 years running conversion rate optimization teams and programs. Specalising in behavioral research.
Meet the Director of Client Services, Annika Thompson
Annika has been doing digital marketing and optimization since 2012. She brings in-house experience from start-ups to FTSE 100 corporations.
Experimentation should be an ongoing process core to your business, ideally used across teams and departments. Our full-service experimentation programs are a 12-month commitment over which time we not only help you run tests but help to embed the function within your business.
When is the best time to start an experimentation program?
This question is best answered by understanding how much it’s costing your business while you aren’t validating ideas. The costs come from the time and resources spent working to implement ideas that at best have no impact, or at worse negatively impact your business metrics. Additionally, there’s an opportunity cost. While you aren’t experimenting it’s likely your business is more cautious to try new ideas, and therefore the opportunity cost arises from not finding new ways to operate online. Your website traffic cannot be retroactively used to validate ideas so the sooner you start the more ideas you can validate and uplifts you can gain. While there are many ways you can experiment and validate ideas that don’t require website traffic, our website experimentation program primarily uses A/B testing. Therefore the best time to start is once your business has the necessary amount of traffic (100,000+ monthly unique users) to run statistically valid tests.
What results can I expect to see from experimentation?
The answer to this depends on the types of tests you are running and the focus of your experimentation program. If you are running basic UX tests you will likely see uplifts in conversion rates. However, you may find you reach the “local maximum” fairly quickly depending on how optimized your website is. If you are running strategic tests which are tied to business metrics you are likely to see larger uplifts that go beyond your local maximum, that impact areas such as profit and retention. Alongside validating business ideas and learning about your customers or users.
How often should I run experiments?
Ideally, all of the time in an ongoing fashion. The velocity of tests you can run depends on several factors, such as; ● The volume of traffic you have to your website. ● Any blackout periods your business may impose where testing must be paused. ● Any technical work that may impact your ability to test. ● The resources you have available. Mature experimentation programs look to utilize as much traffic as possible, aided by refined processes and automations. However, the types of tests you are running and the journeys they impact need to be kept in mind to avoid issues with conflicting experiences for the user and mudding your results. Setting up swimlanes for your testing roadmap can help orchestrate high-velocity experimentation programs.
What is experimentation culture?
Experimentation culture, sometimes also called a “test and learn” culture refers to the mindset and subsequent practices within a business. Businesses who have a good experimentation culture;
● Use data to inform decisions. ● View failed experiments as an opportunity to learn rather than “a failure.” ● Make everyone feel safe to call out decisions that are solely based on opinions especially from HiPPOs (Highest paid persons opinion.) ● Have teams across the business involved in running experiments. ● Have leaders who frequently and repeatedly champion the experimentation mindset. ● Dedicate the necessary resources to support experimentation functions.
Let’s kick things off. Tell us what you’re looking for
Thank you! Your submission has been received!
Oops! Something went wrong while submitting the form.