How Gympass started and scaled experimentation across its business in just six months

Gympass recognized the need for experimentation as a way of driving data-driven decisions. However, there was no testing experience within the marketing team, so they faced challenges such as;

• How do you actually ‘do’ experimentation programmatically?
• How do you drive real strategic insights?
• How do you design experiments? 

Shagun Aulakh was hired as the Director of Web Experience & Experimentation to build the experimentation program from the ground up with the vision of making experimentation part of the Gympass DNA, not just something done within marketing.

Shagun hired us to help her achieve her goals and vision at Gympass. We worked alongside Shagun to;

• Establish and help run the experimentation program.
• Drive strategic insights and a testing roadmap through ResearchXL.
• Create standards for reporting and knowledge sharing to align teams across the business. 

The evolution of goals over the initial six months tells a story of how rapidly this experimentation program was implemented. Shaguns’ initial goal was to get one experiment out the door. 

Once achieved, the goal shifted to increasing test velocity across the company. An important KPI at this stage was measuring the use of resources and templates. This helped to drive better governance and shared understanding, and by tracking this, we could assess the level of experimentation adoption across the business. 

We helped establish internal forums and monthly meetings open to any team across the business. Shagun tracked the number of participants in those meetings to ensure we engaged with diverse stakeholders from marketing, product, analytics, and engineering.

At this point, the focus shifted from increasing aggregate test velocity to increasing test velocity per team. This goal would move us closer to the vision of establishing experimentation across the whole business, not just in a few select teams.

We also started segmenting the metrics/goals by test type, e.g., are teams conducting simple or more substantial/disruptive tests? This helped us monitor if experimentation was becoming more sophisticated and identify any teams requiring more assistance or training.