Experimentation / CRO

Experimentation and Testing Programs acknowledge that the future is uncertain. These programs focus on getting better data to product and marketing teams to make better decisions.

Research & Strategy

We believe that research is an integral part of experimentation. Our research projects aim to identify optimization opportunities by uncovering what really matters to your website users and customers.

Data and Analytics

90% of the analytics setups we’ve seen are critically flawed. Our data analytics audit services give you the confidence to make better decisions with data you can trust.

Optimizing beyond conversion rate

One year ago, we decided to rebrand our agency ‘CXL Agency’ to Speero. The name itself is irrelevant. The important part of our branding rebirth was our new philosophy about optimization.

In the past, we were focused on conversions within the confines of a website. Conversion rate was our bread and butter. We were successful, and our clients were happy.

But we eventually realized that there was more to the story. There was a better way to connect businesses with customers, work with more client-side stakeholders, and ultimately make a larger business impact. 

The Evolution of CRO: Beyond the Website

I recently chatted with my predecessor, outgoing Managing Director, and Founder Viljo Vabrit. Our conversation got to the heart of CXL’s rebrand, and the bigger picture of optimization that we were in search of. 

For Viljo, the rebrand was the product of a personal realization that the customer experience could be downright unpleasant yet our CRO work didn’t necessarily address that.  

“I did a lot of shopping last Christmas. And as you know, stuff happens at the last moment. I came across a lot of solid, well-optimized websites to do my purchases.” 

Viljo noted through his buying experience, tons of conversion rate tactics were being used by online retailers, such as;

  • Pop-ups
  • Surveys
  • Upsells
  • Cross-sells 
  • Urgency
  • Scarcity
“I started to think about my own experience as a customer. Those things, of course, help you convert to make your first purchase. But the overall experience for the customer can be pretty awful.” - Viljo

Why Conversion Rate Alone Isn't Enough

When it comes to buying online, everyone knows the pain points that can arise post purchase. Your items are delayed or missing. If you want to track your order, you have to log into a separate tracking website. When you email customer support, they take five days to respond. The list goes on. 

The experience can be far from perfect. And that’s exactly why we were determined to start zooming out and looking beyond the initial customer acquisition > purchase journey. 

Looking past Top of the Funnel

Most website optimization agencies only look at top of the funnel (ToFu) conversions. 

The driving questions become:

  • How do we get the customer to buy?
  • How do we get the customer to buy faster?
  • How do we get the customer to buy more? 

While the answers to these questions may result in short-term wins, the aggressive nature of the game may mean you lose out on long-term customer retention.

“That's a big problem because people don't have brand loyalty a lot of the time, I'm seeing more of those well-optimized online businesses who have nasty overall experiences. They’re doing a great job converting people into customers, but those customers don't stay.” - Viljo

If you want to understand why customers stay or churn, you need to look past the top of the funnel. You need to look at data in places like customer support logs and surveys.

True, these resources aren’t directly connected to the sales piece of the puzzle. But they are connected to the customer experience. And the customer experience goes way beyond the point of sale.

Understanding the Full Customer Journey

For experimentation experts, the real optimization frontier starts after the conversion. The full customer journey includes post-purchase behaviors, support interactions, and long-term retention metrics like CLV. Understanding this end-to-end experience helps you test more than landing pages—it helps you test loyalty drivers, reduce churn, and improve operational efficiency.

This broader lens lets experimentation become a strategic engine, not just a CRO tool. By connecting qualitative and quantitative data, and tying experiments to every customer touchpoint, you build a compounding loop of insight, learning, and value creation.

Gathering post-purchase data

There is a huge wealth of valuable information that can be gathered after a customer completes a purchase. That’s because the majority of the user experience and customers' opinion about a brand is shaped in the days that follow a purchase.

For example, we had a cosmetics retail client who had incredibly high customer loyalty, and we wanted to investigate what factors contributed to that. While they had a visible founder running the business and a great company story there was more going on. So we got in touch with customer service, and started asking questions.

We discovered that their delivery function was adding high-quality candy to the packages of cosmetics that were shipped out to their customers. And people really loved that. It was a small thing, but it resulted in clear and measurable customer retention, referrals, and positive testimonials.

“Knowing how to retain customers is not a tactic, but it’s something that you can figure out using the same processes we use”  - Viljo

Processes include things like:

  • Gathering and analyzing qualitative data
  • Gathering and analyzing quantitative data
  • Experimentation

When we looked at our most successful experimentation programs, we saw that those that went beyond the website and explored the broader customer experience were able to create the most value.

Data-driven Decision Making

Most teams say they’re data-driven. Few actually are. The default playbook (pulling numbers from GA4, watching conversion rates fluctuate, and looking at dashboards) isn’t enough. 

That kind of data tells you what happened but not why. And optimizing without knowing why is just educated guessing in a lab coat.

Real optimization (the kind that moves metrics and changes orgs) requires more than surface-level numbers. It takes triangulation: pairing quantitative signals (like AOV, RPV, bounce rate) with qualitative depth—polls, heatmaps, session replays, cancellation reasons, user friction, all of it. Not because one is better than the other, but because together they build context.

You need to see the full picture:

  • How customers move through the funnel
  • What’s breaking trust or momentum
  • Where your experimentation program is leaking time, value, or insight

It's not just about measuring conversions. It’s about understanding behavior, diagnosing drop-offs, tracking performance at the program level—not just the test level.

The kicker? This isn’t just optimization hygiene. This is strategic leverage. A team grounded in real data doesn’t just avoid mistakes—it builds compounding insight. And that insight compounds into growth.

Gut feeling gets you started. But data’s what scales.

Importance of Analytics Beyond Conversion Rates

The most common analytics tools (like GA4) focus on quantitative data: how many people completed a specific action. But that only tells part of the story. To truly optimize the user journey, you need to see a wider picture.

This means adding qualitative sources like on-site polls and surveys, along with behavioral analytics tools such as heatmaps and session recordings. These research methods help you uncover why users behave the way they do, not just what they do.

No single source is inherently better. The real power comes from combining them to surface deeper insights and identify more tailored, high-impact optimization opportunities.

Key Metrics to Track For Holistic Optimization

We monitor a range of metrics to evaluate the performance of the experimentation program, focusing on both customer behavior and program efficiency.

First, we have customer funnel metrics. Here, we analyze the entire customer journey from initial website visits (bounce rate, content consumption, site progression) to final conversions (conversion rate, AOV, revenue per visitor, lifetime value). 

For subscription-based businesses, we also track subscription length, cancellation reasons, and more.

Next, we have program performance metrics. Here, we assess the speed and effectiveness of our programs by measuring metrics like the number of tests conducted, error rates, and flat rates.

We do look at the win rate as well, but our recommendation is to be very careful with it, we especially don't want to hinder velocity and innovation.

How to Use Data to Improve Overall Business Performance

Data (both qualitative and quantitative) is what allows you to understand the biggest optimization opportunities. Without data, all you have is instinct and gut feeling, which will only get you so far (humans are particularly bad at predicting the future).

Grounding your decisions in real data not only lowers risk, but also acts as the backbone of any successful optimization program. It turns guesswork into strategy, and strategy into measurable impact.

Experimentation can help you achieve so much more than conversions

Experimentation can ultimately give you three things: 

  1. More money
  2. Saving money
  3. More efficient processes 

If you’re only looking at conversions, you’re stopping after #1. You may be making more money, but you’re leaving things like AOV insight and increased retention on the table.

Number two gets at the idea that when you’re ready to make any change to your business model (such as a new product or brand repositioning), you don’t want to be too risky. Experimentation can help you decide how much to change, and quickly.

Number three encompasses the role of your internal teams. For instance, updating your digital experience will impact the role of customer service, and your marketing team’s re-marketing efforts. 

Creating a sustainable customer experience

Along the same timeline that we were coming to the above conclusions at CXL, the brands we work with were starting to realize it too. There was a subtle gathering of momentum around the desire to be more “ethical and sustainable" when it came to the overall customer experience.

Recently I talked to a CMO in the supplement space, who became a client of ours. In our initial conversation, he described the problem he was having.

He told me that he was working with a performance-based agency already that was doing a great job. He liked them, and they were making his company money. But the website was starting to look like an arcade game. He felt that all of the flashing bells and whistles were resulting in cheap conversions, eroding the reputation of the brand as a whole. The agency’s singular metric focus backfired when it came to brand loyalty.

In short, he was seeking a more ethical, long-term approach to optimization. The conversation got to the heart of our justification to rebrand to Speero, and the reasoning behind tweaking our services to fit the new direction. 

Improving Customer Support and Service

Your customer service can act as a key support optimization factor. You can pull customer data from them, analyze it, fix FUDs, and use it to improve the product, offering, or even the customer service itself.

 You can also improve customer service with AI and chatbots, but don’t forget that both aren’t ‘set and forget’. You’ll need to constantly improve them and feed them with better data.

Customer Support as a Key Optimization Factor

Customer support is such a crucial team to support website optimization. Why? They hold the keys to customer data. You can interview and survey them for so many insights into customer FUDs (fears, uncertainties, and doubts).

They provide specific insights into customer struggles, act as a vital feedback loop for website changes (alerting you to increased support tickets), and should even integrate experimentation into their daily work— something that is still surprisingly rare.

Best Practices for Responsive and Efficient Customer Service

Customer service isn’t just a support function—it’s a goldmine for experimentation insights. Responsive, efficient support teams don’t just solve issues quickly; they surface patterns, friction points, and language that can inform UX, messaging, and retention strategies. For experimentation leaders, optimizing service channels with data and iterative testing can reduce churn, improve NPS, and boost LTV.

Start by integrating support data into your experimentation backlog, automate where it adds value (not noise), and continuously test improvements. Efficiency isn't about doing less—it’s about learning faster and acting smarter.

Leveraging AI and Chatbots for Better Customer Interactions

This is a powerful opportunity more businesses should be leveraging... if they do it right.

Generative AI and chatbots can be tailored to individual customer interactions. The more quality data you feed them, the better they get at addressing user FUDs (fears, uncertainties, and doubts) in real time.

That said, these systems aren’t “set and forget.” Ongoing optimization is crucial. There's nothing more frustrating than a useless chatbot!

 Retention Strategies Over Acquisition

Acquisition costs are rising, and they’ll continue to rise. The golden age of the Internet is over. Everyone is making content and blasting ads. This means you need to focus on retention. Repeat purchases create a stable environment that you can boost with loyalty programs. Let’s find out how.

Why Retaining Customers is More Cost-effective

In a nutshell: acquisition costs keep rising fast. Now that everyone’s online or running ads, both content and paid are making a smaller impact, while the social media and internet gets more overcrowded, with more mediocre content.

Building a business that relies solely on acquiring new customers is not only expensive, it’s also unsustainable. It’s a treadmill that gets harder to keep up with.

Retention flips the script. Encouraging repeat purchases builds a more stable foundation, lowers your cost to serve, and puts you in a stronger position for long-term, sustainable growth.

Proven Retention Tactics

One of the most effective ways to boost purchase frequency is through subscription models. They simplify reordering for consumables, creating convenience and habit, two key drivers of repeat business.

Membership programs are another proven tactic, especially in the DTC space. By offering exclusive perks, discounts, and access, they create a sense of belonging that encourages ongoing engagement and repeat purchases.

Customer Loyalty Programs and Incentives

Testing loyalty programs isn’t about slapping on points and praying. You need to treat loyalty like any other funnel—as something measurable, optimizable, and testable.

First move: define what success even looks like. More repeat purchases? Higher AOV? Email opt-ins for the loyalty tier? Pick your metric and build from there.

Start simple. A/B test loyalty sign-up CTAs: “Get rewards” vs. “Earn points every time you shop.” Then test offer types—discounts, freebies, early access—to see what actually drives behavior.

Next: test visibility and placement. How many users even see the loyalty program? If it’s buried in your nav, that’s not a loyalty problem, it’s a UX problem.

Then test tiering. Does introducing a VIP level boost engagement, or just create drop-off from casual users? You won’t know until you test it.

Final pro move: personalize loyalty nudges based on user behavior. Test when and how you prompt based on purchase history, cart size, or engagement level.

Loyalty isn’t static. It’s a feedback loop.

Measuring the customer experience

Chad Sanderson, Head of Product at Convoy, once shared a good analogy for the experimentation process with me. You can think of the experimentation process as a measurement tool. Your various channel owners - acquisition, marketing, product, etc. - all have different questions. Each one of them is looking for measurement yardsticks in areas like LTV and retention rate. The key to long-term, comprehensive optimization means measuring the customer experience from start to finish. 

In most cases, brands don’t do this. They get reports from 1-3 data sources regarding traffic and conversions, but exclude the experience itself. They don’t ask the question they should: “How was the shopping experience for you today, and how could we improve it?”


To be fair, there is simply not a lot of information out there about how to record, test, and optimize the full customer experience. But the short answer is to ask the question and pay attention to the answers.Some brands implement a pop-up that asks about the experience in the middle of the shopping process. Viljo explains why this is a no-no;

“It might create friction if you ask too early. I've seen a service that asks you in the middle of the shopping process. ‘So how do you like the shopping experience?’ I can’t say, I haven't shopped yet! So it's too early. But usually when I get the survey in the email with my purchase confirmation, I give feedback, I’ll l rate it.”

The purchase confirmation email is an excellent place to pose this question to customers. It’s not as aggressive as an on-site pop-up but will get you the information that you need. 

Defining the customer journey

Many brands also don’t understand what the customer experience is. Where does it start? Where does it end? And for different businesses/industries, those customer journeys can be very different. 

Regardless of your product, every brand can ask questions like:

  • How was the unboxing/onboarding experience? 
  • How easy was the return/refund process?
  • How was the customer support experience? 

As we mentioned earlier, most of the customer’s experience with a brand happens after the purchase. The customer may spend much more time communicating with you and forming opinions on your product and service after they receive it - not while surfing your website. That’s why it’s so important to conduct measurement and testing around these areas. 

Customer Lifetime Value (CLV) Optimization

While conversion rate tells you who said “yes” today, Customer Lifetime Value tells you who’ll stick around tomorrow. For experimentation leaders, optimizing CLV means running tests that go beyond the initial purchase—across retention, upsells, onboarding, loyalty, and product experience. It’s not just about more customers, but better ones.

By connecting experiments to behaviors that increase long-term value, you unlock compounding growth. The goal? Build journeys that turn first-time buyers into high-value brand advocates—and make your tests pay off long after the win.

Understanding CLV and Why it Matters

Customer Lifetime Value (CLV, sometimes LTV) is a measure of how much revenue a customer generates over the entire duration of their relationship with your brand. It’s not just a metric, it’s a strategic lens through which smart ecommerce companies view their acquisition, retention, and optimization decisions.

At its core, CLV forces you to zoom out. Instead of asking, “Did this customer convert today?” you’re asking, “Is this the kind of customer who will stick around and be profitable over time?” That shift has enormous implications.

Strategies to Increase CLV

Many testing strategies let you increase CLV:

  • Promote subscriptions. Make reordering easy and automatic, especially for consumables.
  • Offer exclusive perks. Loyalty programs or gated member benefits drive repeat purchases.
  • Personalize the experience. Use first-party data to tailor recommendations and content.
  • Simplify reordering. Enable one-click reorders or reminders post-purchase.
  • Build onboarding flows. Encourage account creation and educate customers early on.
  • Bundle for retention. Pair high-margin items with essentials to encourage deeper engagement.

Customer Retention vs. Acquisition

Acquisition gets your customers through the door, but it’s often expensive and competitive. Retention keeps them coming back. It’s more cost-effective and directly impacts CLV.

Smart brands don’t choose one over the other, they align both. But if you’re not retaining customers, you’re stuck filling a leaky bucket.

We’re still optimizers

At the heart of our business, nothing has changed. We’re just thinking a lot more about the big-picture context - and measuring what’s needed to inform and improve the overall customer experience. 

We’re looking a lot closer at the customer journey, and doing things like sorting users into buckets based on their intent. What do users want to know? What do they want to accomplish? What do they want to buy? Analyzing these things means gathering Voice of Customer data, and doing things like encouraging all clients to add a survey on the Thank You page or purchase email. 

Other work includes:

Gathering downstream data

  • Identify average order value
  • Identify customer lifetime value 
  • Identify best selling products
  • Identify products that lead to the highest AOV per channel

Gathering downstream metrics 

  • What is the LTV? (what channel contribute to LTV?)
  • What is the retention rate?
  • What are the pipeline dollars? 
  • What's the MQL to SQL ratio? 
  • What's the SQLs?
  • What's the MQL? 
  • At what point should you sacrifice volume for quality? 

Tying all of this data into our experimentation framework is the biggest strategy piece that we are focused on with our clients right now. If you’re an eCommerce owner doing $10 million and you don’t know what your LTV is by channel, there’s no doubt you should be doing this too. 

It’s time to think about conversion rate as just one small piece of the bigger optimization puzzle. 

Speero runs “Growth Experimentation” programs for marketing and product growth leaders. Our promise is to 5X your testing velocity and impact.

We've helped hundreds of brand understand their customers, increase revenue, and grow from experimentation.

Want to see how Speero can help you experiment your way to long-term growth, not just short-term wins? Reach out.

FAQ

How Does Post-purchase Data Contribute to Better Optimization?

Post-purchase data reveals what happens after the conversion — the why behind repeat buys, returns, churn, and loyalty. It helps you:

  • Identify high-value customer segments.
  • Spot friction in fulfillment or product experience.
  • Refine offers, bundles, and retention flows based on actual behavior.

It’s not just about closing the sale, it’s about understanding what keeps customers coming back.

What Role Does Experimentation Play in Optimization Beyond Conversions?

Experimentation isn’t just about boosting conversion rates — it’s about validating decisions across the business. It helps you:

  • Increase order value through smarter upsells and bundling.
  • Improve retention by testing subscription flows or onboarding tactics.
  • Enhance profitability by merchandising higher-margin products.

When done right, experimentation becomes a company-wide decision-making engine, not just a CRO tool.

How can Brands Measure and Improve the Overall Customer Experience?

Measure it with tools like NPS, post-purchase surveys, and usability testing. Look for friction points across the entire journey — not just the purchase.

Improve it by acting on insights: clarify messaging, streamline navigation, fix checkout pain, and personalize interactions.

A great customer experience doesn’t just convert, it retains, delights, and drives long-term growth.

Did you like this article?

(Your feedback helps us write better!) worst 1 - 10 best

Did the article resonate with you?

What aspects did you enjoy or find lacking?
Were there elements you felt we should've covered?
Thank you.
Oops! Something went wrong while submitting the form.

Related Posts

Who's currently reading The Experimental Revolution?