Experimentation / CRO

Experimentation and Testing Programs acknowledge that the future is uncertain. These programs focus on getting better data to product and marketing teams to make better decisions.

Research & Strategy

We believe that research is an integral part of experimentation. Our research projects aim to identify optimization opportunities by uncovering what really matters to your website users and customers.

Data and Analytics

90% of the analytics setups we’ve seen are critically flawed. Our data analytics audit services give you the confidence to make better decisions with data you can trust.

From Zero to 100 tests a year in just 6 months through the power of Speero’s full-service experimentation program.

Laura Borghesi, Director of Growth at MongoDB needed a high velocity program yesterday. But with so many different priorities resting on her back, doing it alone was impossible.

She hired Speero because of our no BS, transparent communication. The result? Zero to 100 tests a month, 1 CoE, no internal fighting over experiments, and a transformed internal culture.

The Challenge

Laura Borghesi has just painted a pretty picture for her executive team. There was a throve of experimentation opportunities for their brand. She convinced the higher-ups that experimentation would allow MongoDB to grow way faster than it would do organically, without a process.

But somehow, this victory didn’t seem that big. If she and MongoDB wanted to leverage experiments, they will need to run a high-velocity testing program. The battle was won, but war was far from over.

And running a high-velocity program is a spartan-like project. Difficult and time-consuming. Optimization budgets are tight, meaning smaller teams and less prioritization. Optimization is everything but a small task, leading to being very busy and very tired optimizers. Not to mention that Laura and her team needed a flawless data set up so they could make confident decisions based on data they could trust.

“I needed people to help me run it because I was running the whole growth team at the time. I had multiple responsibilities and couldn’t focus 100% on the experimentation program. I needed someone to run the program, and me to guide it”

Laura Borghesi, Senior Director of Growth Marketing at MongoDB

In her current situation, ramping up a high-velocity experiment program from scratch seemed out of reach for Laura. Even worse, low testing velocity wouldn’t justify the expectation she created in the meeting.

So Laura put out an RFP and reviewed three experimentation agencies. Speero won for two reasons:

  1. Straightforward and transparent approach and communication
  2. We were able to clearly explain high-level experimentation and its benefits for executives unfamiliar with experimentation.

“The reasons why you guys won were that very much I loved your approach and I loved the fact that it was like straightforward, no BS right type of approach. And the communication was clear to my counterparts who had very little familiarity with experimentation, who needed a sort of like the high-level speech about experimentation.”

Laura Borghesi, Senior Director of Growth Marketing at MongoDB

The Solution

From Zero to 100 Tests a Month

Since MongoDB wanted to experiment at a high velocity while starting from zero, Speero started with running experiments on website locations containing ample opportunities and a nimble infrastructure. This way, Speero made it easy to get things going quickly and with an impact.  

These initial experiments let us build confidence in experimentation for the C-level and in the same time establish the necessary processes to execute tests. 

By the way, this is our standard process for delivering our full-service experimentation program. It’s made of:

  1. Initial assessments
  2. User research and data analysis
  3. Test prioritization
  4. Design
  5. Execution
  6. Program management

This way, Speero is able to identify and leverage locations with lots of testing opportunities and agile structure. Then we can leverage these first wins to establish and refine experimentation processes and guardrails, win trust, and direct the program in a profitable direction.

Digestible Communication For The Win

We knew that test experience and understanding were different in different departments of MongoDB. So, we adapted our communication style and content to suit the audience. This helped to bring everyone along for the ride without alienating anyone. Everything was easy to digest and understand both for those unfamiliar or familiar with experimentation.

“You translate what experimentation means for different people in a very clear, digestible way. There's a lot of good statistical knowledge built into how Speero prioritizes things, and you are able to translate that to different people with different levels of knowledge.”

Laura Borghesi, Senior Director of Growth Marketing at MongoDB

How Speero Turned Experimentation Fears into Mantras

Speero didn’t only raise experimentation maturity for MongoDB but also established a Center of Excellence team sitting across the growth function. The job of the CoE team was not to own any part of the user experience but enable others to run experiments. 

While running the program, we identified and worked on opportunities to embed experimentation across MongoDB. How? We shared processes and documentation, ran internal workshops, and provided access to CXL training. 

This is something completely missing from the DIY approach. Since we already have processes, documentation, and a world-class experimentation training platform (CXL Institute), we can incorporate and teach experimentation way faster than a team doing all of this organically. Experience beats talent.

“In the beginning, people were scared of changing things so quickly. But then it became a mantra. It's like, no, we're not changing everything or disassembling the user experience. We’re collecting data to help you make better decisions. Everybody got so excited in the end, and now people are coming to the team to ask, can we do more experiments?”

Laura Borghesi, Senior Director of Growth Marketing at MongoDB

As confidence grew across the teams, the CoE role shifted to overseeing the integrity of experiments, setting the overarching roadmap, and increasing velocity by implementing different testing swim lanes.

The Results

  • Moved from zero to 100 tests a year in just six months by leveraging locations with the most test opportunities and training the MongoDB team.
  • Build confidence and deterred internal fighting through digestible and transparent communication.
  • Helped transform internal culture from being scared of experimentation to teams leading the way.
  • Helped MongoDB’s growth team achieve their vision— create a center of excellence so people across the business could use experiments to make business decisions and drive growth. 

“Growth to me is a scientific approach to growing your business faster than what it would organically do on its own. And you do that through two pillars—data and experimentation.”

Laura Borghesi, Senior Director of Growth Marketing at MongoDB

Key Takeaways

1

Building a self-service CoE model helps increase test velocity.

But it’s not always the best team structure. Check out our blueprint on experimentation team structures to help you decide what’s best for your business.

1

Velocity is important, but you must ensure you are testing for the right reasons and delivering business results.

Download our strategic roadmap template to help you organize tactics and strategies against research and metrics.

1

You need to monitor other metrics alongside velocity to develop a successful experimentation program.

While increasing velocity might be your goal, you need to balance it with aspects such as test complexity. Use our velocity vs. complexity graph as a conversation starter on this topic.

Let’s kick things off.
Tell us what you’re looking for

Thank you! Your submission has been received!
Oops! Something went wrong while submitting the form.