Providing focus for Decathlon's testing program

Decathlon has over 2000 stores across 56 countries, launching its domestic presence in Switzerland in 2016. 

Covid-19 lockdowns were the catalyst for a renewed focus on the Swiss ecommerce site, and in 2021, Laurent Hayoz was hired as their first CRO specialist. Laurent established Decathlon Switzerland's first foray into experimentation, and as tests drove business value, pressure mounted to accelerate and scale the program. But as a lower-traffic domain, Laurent realized they needed to be more strategic to hit their goals.

Speero was brought in to help scale Decathlon Switzerland's fledgling testing program. 

We started from the ground up, introducing processes, strategy, documentation, and resources. Our first task was establishing the potential velocity and type of tests the website could support based on traffic levels and conversions. We used our test bandwidth calculator to understand how many tests we could run. This would help us create the strategy and prioritize tests. 

The next step was to produce a goal tree map to link company goals to program and test metrics. This helped us concentrate on the most impactful areas in a program where A/B testing opportunities were limited. 

Next, we needed to establish what and where to test, so we conducted a range of research as set out in the ResearchXL model. The research delivered a long list of insights, so we determined which represented the highest potential. We organized the findings into problem themes and prioritized four focus areas. They were; 

1. Category Navigation: users find it challenging to navigate to relevant products.
2. Information Gathering: users struggle to determine which product is best for them. 
3. Delivery and Returns: users struggle to understand the delivery and returns process.‍
4. Checkout: There is an opportunity to streamline the checkout process.

With our themes prioritized, the next step was to operationalize what work would be done and when adding yet another layer of prioritization to the process. We created a strategy map with three testing streams under each problem/opportunity statement. The three streams were; 

1. Just Do It (JDIs). Elements that didn’t require testing but needed to be changed or fixed. 
2. Tactical tests. Typically more straightforward UX tests. ‍
3. Strategic opportunities. Strategic opportunities aren’t necessarily tests. These could be business opportunities or recommendations for further investigations, which can lead to a test. For example, Decathlon tested a different payment method which led to a new business partnership

As there were limitations on how many A/B tests we could run, we used additional research to run pre-test validation, increasing the potential success rate of any eventual A/B test. Our PXL framework also prioritized individual tests within the streams based on impact, confidence, and effort. 

The three test streams were staggered, so JDIs and tactical testing started first. This is because the tactical tests are lower effort and don’t need much buy-in, therefore helping to oil a new testing process by running a higher velocity of tests early on. But also because the insights from tactical tests can inform bigger strategic tests.

‍

‍