Delivering 5x growth for Codecademy with strategic testing

Codecademy is an interactive platform that offers coding classes in 12 programming languages to over 50M learners across 190 countries.

Operating on a subscription model with a massive user base, Codecademy realized that consistent conversion uplifts could earn millions in additional revenue. But their experimentation program lacked a rigorous process, and many tests were inconsequential changes driven by opinion or what their competitors were doing. There was also a lack of understanding around the key growth levers, meaning that time and resources were spent testing elements that didn’t move the needle.

Codecademy had the potential to drive significant growth and gain a much better understanding of its users through experimentation, but they needed help.

The first step was to identify the levers responsible for growth. As Codecademy had a strong brand and top-of-funnel traffic, we focused on growth levers at the bottom of the funnel, such as pricing, checkout, plan mix, and the trial model.

Once the growth levers were identified, we conducted a range of research to understand users' perceptions and behavior throughout the experience. This helped us to identify frustrations, uncertainties, and doubts impacting conversion rates.

With a ton of research-informed hypotheses and a prioritization framework that factored in the impact of growth levers, we moved Codecademy from ad-hoc spaghetti testing to something altogether more strategic.

Experimenting with price and plan mix is complex because they impact the business, product, customer experience, and multiple internal teams such as marketing, finance, customer support, and sales. This complexity often causes executives to call off such experiments. But by using the Speero test workflow, we were able to show the rigorous nature of our process and build confidence to run these complex tests.