Experimentation / CRO

Experimentation and Testing Programs acknowledge that the future is uncertain. These programs focus on getting better data to product and marketing teams to make better decisions.

Research & Strategy

We believe that research is an integral part of experimentation. Our research projects aim to identify optimization opportunities by uncovering what really matters to your website users and customers.

Data and Analytics

90% of the analytics setups we’ve seen are critically flawed. Our data analytics audit services give you the confidence to make better decisions with data you can trust.

How Gympass started and scaled experimentation across its business in just six months

The Challenge

Gympass recognized the need for experimentation as a way of driving data-driven decisions. However, there was no testing experience within the marketing team, so they faced challenges such as;

  • How do you actually ‘do’ experimentation programmatically?
  • How do you drive real strategic insights?
  • How do you design experiments? 

Shagun Aulakh was hired as the Director of Web Experience & Experimentation to build the experimentation program from the ground up with the vision of making experimentation part of the Gympass DNA, not just something done within marketing.

The Solution

Shagun hired us to help her achieve her goals and vision at Gympass. We worked alongside Shagun to;

  • Establish and help run the experimentation program.
  • Drive strategic insights and a testing roadmap through ResearchXL.
  • Create standards for reporting and knowledge sharing to align teams across the business. 

The evolution of goals over the initial six months tells a story of how rapidly this experimentation program was implemented. Shaguns’ initial goal was to get one experiment out the door. 

Once achieved, the goal shifted to increasing test velocity across the company. An important KPI at this stage was measuring the use of resources and templates. This helped to drive better governance and shared understanding, and by tracking this, we could assess the level of experimentation adoption across the business. 

We helped establish internal forums and monthly meetings open to any team across the business. Shagun tracked the number of participants in those meetings to ensure we engaged with diverse stakeholders from marketing, product, analytics, and engineering.

At this point, the focus shifted from increasing aggregate test velocity to increasing test velocity per team. This goal would move us closer to the vision of establishing experimentation across the whole business, not just in a few select teams.

We also started segmenting the metrics/goals by test type, e.g., are teams conducting simple or more substantial/disruptive tests? This helped us monitor if experimentation was becoming more sophisticated and identify any teams requiring more assistance or training.

"What I think transformed Gympass into having a strong experimentation program was a lot of time spent bringing stakeholders together, holding regular meetings, and coaching them…Before, we were handholding teams through a lot. Now they are pretty independent and self-sufficient. They will come to us with more complex questions, but generally, they can run experiments and make decisions without our constant support."

Shagun Aulakh, Director of Web Experience & Experimentation at Gympass

The Results

  • Helped Gympass move from no testing to all marketing teams across the business, running sophisticated experiments in just six months. This is the fastest we’ve seen a program move from zero to two. 
  • Achieved an 80% adoption rate of experimentation templates across the business, helping drive self-sufficiency and improved test governance. 
  • Helped create rituals and artifacts for training and enablement, such as office hours, workshops, reporting docs, etc.

Key Takeaways

1

Engage with your stakeholders regularly.

If you want to embed a culture of experimentation, you need to get people involved in your work. Check out our blueprint for making engagement accessible and enjoyable.

1

Consider how you will structure your experimentation team.

If you don’t plan to run experiments for your whole organization, you need to think strategically about how you structure any team or resources. Use our org chart to help you consider your options.

1

Use program metrics to measure and work.

You’ll only know how successful you are if you are measuring the right metrics. Program metrics help you see what areas need improvement and where to stretch your goals. Use our program metrics blueprint to help you measure what’s important. 

Let’s kick things off.
Tell us what you’re looking for

Thank you! Your submission has been received!
Oops! Something went wrong while submitting the form.