Decathlon has over 2000 stores across 56 countries, launching its domestic presence in Switzerland in 2016.
Covid-19 lockdowns were the catalyst for a renewed focus on the Swiss ecommerce site, and in 2021, Laurent Hayoz was hired as their first CRO specialist. Laurent established Decathlon Switzerland's first foray into experimentation, and as tests drove business value, pressure mounted to accelerate and scale the program. But as a lower-traffic domain, Laurent realized they needed to be more strategic to hit their goals.
“At some point, you will want to scale more but hit a glass ceiling of traffic bandwidth. In that case, you need to improve the quality and impact of your tests.”
Laurent Hayoz, Conversion Optimization Specialist at Decathlon
Speero was brought in to help scale Decathlon Switzerland's fledgling testing program.
We started from the ground up, introducing processes, strategy, documentation, and resources. Our first task was establishing the potential velocity and type of tests the website could support based on traffic levels and conversions. We used our test bandwidth calculator to understand how many tests we could run. This would help us create the strategy and prioritize tests.
The next step was to produce a goal tree map to link company goals to program and test metrics. This helped us concentrate on the most impactful areas in a program where A/B testing opportunities were limited.
Next, we needed to establish what and where to test, so we conducted a range of research as set out in the ResearchXL model. The research delivered a long list of insights, so we determined which represented the highest potential. We organized the findings into problem themes and prioritized four focus areas. They were;
Category Navigation: users find it challenging to navigate to relevant products.
Information Gathering: users struggle to determine which product is best for them.
Delivery and Returns: users struggle to understand the delivery and returns process.
Checkout: There is an opportunity to streamline the checkout process.
With our themes prioritized, the next step was to operationalize what work would be done and when adding yet another layer of prioritization to the process. We created a strategy map with three testing streams under each problem/opportunity statement. The three streams were;
Just Do It (JDIs). Elements that didn’t require testing but needed to be changed or fixed.
Tactical tests. Typically more straightforward UX tests.
Strategic opportunities. Strategic opportunities aren’t necessarily tests. These could be business opportunities or recommendations for further investigations, which can lead to a test. For example, Decathlon tested a different payment method which led to a new business partnership
As there were limitations on how many A/B tests we could run, we used additional research to run pre-test validation, increasing the potential success rate of any eventual A/B test. Our PXL framework also prioritized individual tests within the streams based on impact, confidence, and effort.
The three test streams were staggered, so JDIs and tactical testing started first. This is because the tactical tests are lower effort and don’t need much buy-in, therefore helping to oil a new testing process by running a higher velocity of tests early on. But also because the insights from tactical tests can inform bigger strategic tests.
Established an end-to-end experimentation process at Decathlon Switzerland.
Increased the quality and impact of tests by ensuring the most significant opportunities and problems were addressed.
Set the testing strategy for the organization, and continue to run the experimentation program alongside the Decathlon team two years on.
Use A/B tests wisely.
A/B tests aren’t the only method to validate a hypothesis, and if your website has low traffic, it’s important to utilize other forms of validation too. Get our validation method flow chart to help you decide when to use an A/B test vs. other methods.
Prioritize the right experiments.
With limited test bandwidth or resources, every test you run must have the greatest potential impact. Use our PXL framework to prioritize your test backlog.