Experimentation / CRO

Experimentation and Testing Programs acknowledge that the future is uncertain. These programs focus on getting better data to product and marketing teams to make better decisions.

Research & Strategy

We believe that research is an integral part of experimentation. Our research projects aim to identify optimization opportunities by uncovering what really matters to your website users and customers.

Data and Analytics

90% of the analytics setups we’ve seen are critically flawed. Our data analytics audit services give you the confidence to make better decisions with data you can trust.

Optimizing beyond conversion rate

One year ago, we decided to rebrand our agency ‘CXL Agency’ to Speero. The name itself is irrelevant. The important part of our branding rebirth was our new philosophy about optimization.

In the past, we were focused on conversions within the confines of a website. Conversion rate was our bread and butter. We were successful, and our clients were happy.

But we eventually realized that there was more to the story. There was a better way to connect businesses with customers, work with more client-side stakeholders, and ultimately make a larger business impact. 

Standing in the shoes of the customer

I recently chatted with my predecessor, outgoing Managing Director, and Founder Viljo Vabrit. Our conversation got to the heart of CXL’s rebrand, and the bigger picture of optimization that we were in search of. 

For Viljo, the rebrand was the product of a personal realization that the customer experience could be downright unpleasant yet our CRO work didn’t necessarily address that.  

“I did a lot of shopping last Christmas. And as you know, stuff happens at the last moment. I came across a lot of solid, well-optimized websites to do my purchases.” 

Viljo noted through his buying experience, tons of conversion rate tactics were being used by online retailers, such as;

  • Pop-ups
  • Surveys
  • Upsells
  • Cross-sells 
  • Urgency
  • Scarcity
“I started to think about my own experience as a customer. Those things, of course, help you convert to make your first purchase. But the overall experience for the customer can be pretty awful.” - Viljo

When it comes to buying online, everyone knows the pain points that can arise post purchase. Your items are delayed or missing. If you want to track your order, you have to log into a separate tracking website. When you email customer support, they take five days to respond. The list goes on. 

The experience can be far from perfect. And that’s exactly why we were determined to start zooming out and looking beyond the initial customer acquisition > purchase journey. 

Looking past Top of the Funnel

Most website optimization agencies only look at top of the funnel (ToFu) conversions. 

The driving questions become:

  • How do we get the customer to buy?
  • How do we get the customer to buy faster?
  • How do we get the customer to buy more? 

While the answers to these questions may result in short-term wins, the aggressive nature of the game may mean you lose out on long-term customer retention.

“That's a big problem because people don't have brand loyalty a lot of the time, I'm seeing more of those well-optimized online businesses who have nasty overall experiences. They’re doing a great job converting people into customers, but those customers don't stay.” - Viljo

If you want to understand why customers stay or churn, you need to look past the top of the funnel. You need to look at data in places like customer support logs and surveys.

True, these resources aren’t directly connected to the sales piece of the puzzle. But they are connected to the customer experience. And the customer experience goes way beyond the point of sale.

Gathering post-purchase data

There is a huge wealth of valuable information that can be gathered after a customer completes a purchase. That’s because the majority of the user experience and customers' opinion about a brand is shaped in the days that follow a purchase.

For example, we had a cosmetics retail client who had incredibly high customer loyalty, and we wanted to investigate what factors contributed to that. While they had a visible founder running the business and a great company story there was more going on. So we got in touch with customer service, and started asking questions.

We discovered that their delivery function was adding high-quality candy to the packages of cosmetics that were shipped out to their customers. And people really loved that. It was a small thing, but it resulted in clear and measurable customer retention, referrals, and positive testimonials.

“Knowing how to retain customers is not a tactic, but it’s something that you can figure out using the same processes we use”  - Viljo

Processes include things like:

  • Gathering and analyzing qualitative data
  • Gathering and analyzing quantitative data
  • Experimentation

When we looked at our most successful experimentation programs, we saw that those that went beyond the website and explored the broader customer experience were able to create the most value.

Experimentation can help you achieve so much more than conversions

Experimentation can ultimately give you three things: 

  1. More money
  2. Saving money
  3. More efficient processes 

If you’re only looking at conversions, you’re stopping after #1. You may be making more money, but you’re leaving things like AOV insight and increased retention on the table.

Number two gets at the idea that when you’re ready to make any change to your business model (such as a new product or brand repositioning), you don’t want to be too risky. Experimentation can help you decide how much to change, and quickly.

Number three encompasses the role of your internal teams. For instance, updating your digital experience will impact the role of customer service, and your marketing team’s re-marketing efforts. 

Creating a sustainable customer experience

Along the same timeline that we were coming to the above conclusions at CXL, the brands we work with were starting to realize it too. There was a subtle gathering of momentum around the desire to be more “ethical and sustainable" when it came to the overall customer experience.

Recently I talked to a CMO in the supplement space, who became a client of ours. In our initial conversation, he described the problem he was having.

He told me that he was working with a performance-based agency already that was doing a great job. He liked them, and they were making his company money. But the website was starting to look like an arcade game. He felt that all of the flashing bells and whistles were resulting in cheap conversions, eroding the reputation of the brand as a whole. The agency’s singular metric focus backfired when it came to brand loyalty.

In short, he was seeking a more ethical, long-term approach to optimization. The conversation got to the heart of our justification to rebrand to Speero, and the reasoning behind tweaking our services to fit the new direction. 

Measuring the customer experience

Chad Sanderson, Head of Product at Convoy, once shared a good analogy for the experimentation process with me. You can think of the experimentation process as a measurement tool. Your various channel owners - acquisition, marketing, product, etc. - all have different questions. Each one of them is looking for measurement yardsticks in areas like LTV and retention rate. The key to long-term, comprehensive optimization means measuring the customer experience from start to finish. 

In most cases, brands don’t do this. They get reports from 1-3 data sources regarding traffic and conversions, but exclude the experience itself. They don’t ask the question they should: “How was the shopping experience for you today, and how could we improve it?”


To be fair, there is simply not a lot of information out there about how to record, test, and optimize the full customer experience. But the short answer is to ask the question and pay attention to the answers.Some brands implement a pop-up that asks about the experience in the middle of the shopping process. Viljo explains why this is a no-no;

“It might create friction if you ask too early. I've seen a service that asks you in the middle of the shopping process. ‘So how do you like the shopping experience?’ I can’t say, I haven't shopped yet! So it's too early. But usually when I get the survey in the email with my purchase confirmation, I give feedback, I’ll l rate it.”

The purchase confirmation email is an excellent place to pose this question to customers. It’s not as aggressive as an on-site pop-up but will get you the information that you need. 

Defining the customer journey

Many brands also don’t understand what the customer experience is. Where does it start? Where does it end? And for different businesses/industries, those customer journeys can be very different. 

Regardless of your product, every brand can ask questions like:

  • How was the unboxing/onboarding experience? 
  • How easy was the return/refund process?
  • How was the customer support experience? 

As we mentioned earlier, most of the customer’s experience with a brand happens after the purchase. The customer may spend much more time communicating with you and forming opinions on your product and service after they receive it - not while surfing your website. That’s why it’s so important to conduct measurement and testing around these areas. 

We’re still optimizers

At the heart of our business, nothing has changed. We’re just thinking a lot more about the big-picture context - and measuring what’s needed to inform and improve the overall customer experience. 

We’re looking a lot closer at the customer journey, and doing things like sorting users into buckets based on their intent. What do users want to know? What do they want to accomplish? What do they want to buy? Analyzing these things means gathering Voice of Customer data, and doing things like encouraging all clients to add a survey on the Thank You page or purchase email. 

Other work includes:

Gathering downstream data

  • Identify average order value
  • Identify customer lifetime value 
  • Identify best selling products
  • Identify products that lead to the highest AOV per channel

Gathering downstream metrics 

  • What is the LTV? (what channel contribute to LTV?)
  • What is the retention rate?
  • What are the pipeline dollars? 
  • What's the MQL to SQL ratio? 
  • What's the SQLs?
  • What's the MQL? 
  • At what point should you sacrifice volume for quality? 

Tying all of this data into our experimentation framework is the biggest strategy piece that we are focused on with our clients right now. If you’re an eCommerce owner doing $10 million and you don’t know what your LTV is by channel, there’s no doubt you should be doing this too. 

It’s time to think about conversion rate as just one small piece of the bigger optimization puzzle. 

Did you like this article?

(Your feedback helps us write better!) worst 1 - 10 best

Did the article resonate with you?

What aspects did you enjoy or find lacking?
Were there elements you felt we should've covered?
Thank you.
Oops! Something went wrong while submitting the form.

Related Posts

Who's currently reading The Experimental Revolution?