From Zero to 100 Tests a Month
Since MongoDB wanted to experiment at a high velocity while starting from zero, Speero started with running experiments on website locations containing ample opportunities and a nimble infrastructure. This way, Speero made it easy to get things going quickly and with an impact.
These initial experiments let us build confidence in experimentation for the C-level and in the same time establish the necessary processes to execute tests.
By the way, this is our standard process for delivering our full-service experimentation program. It’s made of:
- Initial assessments
- User research and data analysis
- Test prioritization
- Design
- Execution
- Program management
This way, Speero is able to identify and leverage locations with lots of testing opportunities and agile structure. Then we can leverage these first wins to establish and refine experimentation processes and guardrails, win trust, and direct the program in a profitable direction.
Digestible Communication For The Win
We knew that test experience and understanding were different in different departments of MongoDB. So, we adapted our communication style and content to suit the audience. This helped to bring everyone along for the ride without alienating anyone. Everything was easy to digest and understand both for those unfamiliar or familiar with experimentation.
“You translate what experimentation means for different people in a very clear, digestible way. There's a lot of good statistical knowledge built into how Speero prioritizes things, and you are able to translate that to different people with different levels of knowledge.”
Laura Borghesi, Senior Director of Growth Marketing at MongoDB
How Speero Turned Experimentation Fears into Mantras
Speero didn’t only raise experimentation maturity for MongoDB but also established a Center of Excellence team sitting across the growth function. The job of the CoE team was not to own any part of the user experience but enable others to run experiments.
While running the program, we identified and worked on opportunities to embed experimentation across MongoDB. How? We shared processes and documentation, ran internal workshops, and provided access to CXL training.
This is something completely missing from the DIY approach. Since we already have processes, documentation, and a world-class experimentation training platform (CXL Institute), we can incorporate and teach experimentation way faster than a team doing all of this organically. Experience beats talent.
“In the beginning, people were scared of changing things so quickly. But then it became a mantra. It's like, no, we're not changing everything or disassembling the user experience. We’re collecting data to help you make better decisions. Everybody got so excited in the end, and now people are coming to the team to ask, can we do more experiments?”
Laura Borghesi, Senior Director of Growth Marketing at MongoDB
As confidence grew across the teams, the CoE role shifted to overseeing the integrity of experiments, setting the overarching roadmap, and increasing velocity by implementing different testing swim lanes.