Experimentation / CRO

Experimentation and Testing Programs acknowledge that the future is uncertain. These programs focus on getting better data to product and marketing teams to make better decisions.

Research & Strategy

We believe that research is an integral part of experimentation. Our research projects aim to identify optimization opportunities by uncovering what really matters to your website users and customers.

Data and Analytics

90% of the analytics setups we’ve seen are critically flawed. Our data analytics audit services give you the confidence to make better decisions with data you can trust.

Why you need to think about Customer Experience Optimization 

Conversion rate optimization (CRO) has always been a flawed term, and most people in the industry agree. 


Because the term frames your work in a way that focuses on the wrong outcomes - conversion rates. As well as limiting the scope of how you use experimentation to test and learn throughout your company (e.g thinking that if there’s not a measurable ‘conversion’ then experimentation shouldn’t be used). 

Here’s why that’s a bad thing;

I can increase your conversion rates and drive revenue instantly 

Just slash the cost of your products, or give away a free Rolex with every sale. Voilà, your conversion rates will soar. 

While this is a dumb example, it does show why conversions don't equal profit - sometimes they can equate to a loss. You might have increased conversion rates in the above example, but at the expense of your profit margins. And sales promotions could drive behaviour such as buying multiple products only to return them, costing your business even more. And that’s why focusing on conversion rates can be very damaging. 

We’re written previously about what metrics you should be tracking but the majority of testing programs we see still focus solely on revenue. And as the above illustrates this can lead you very much down the wrong path. 

So what should you be doing instead? 

Profitability driven by retention 

CRO focuses heavily on converting new customers to drive revenue, but we know the cost of acquiring a new customer is 5-25 times more expensive than retaining your existing ones. And that a 5% increase in customer retention leads to 25% - 95% more profit. So if your actual aim is to improve the metrics that matter the most to your business you’ll need to rethink what you do and how you measure your work. 

To impact on things like retention you need to optimize the whole customer journey, from discovery to fulfilment, after-sales and repeat purchase journeys. After all, your website is never just used in isolation. Your customers are primed before they arrive, based on your advertising, search results, what they heard about you from their friends etc. Whether they purchase again isn’t decided based on the website experience alone, but their experience of the product itself, the after sales care, the delivery. 

For example, a few areas to consider: 

  • Did the product live up to the promises on the website? 
  • Were orders delivered in the timeframe you promote online? Is it as quick as competitors? Was the delivery experience a good one? 
  • Did your team reply to any enquiries customers had in a responsive and helpful way?
  • Did you contact customers when they might need to repurchase with a reason to buy again? 

While conversion optimization can be key to getting a customer over the line initially, customer experience optimization makes sure you don’t leave money on the table at other points in the journey. And there's a lot of money left on the table; a Bain study found that 80% of CEOs believed their company delivered a superior customer experience but only 8% of their customers agreed.

Customer Experience Optimization 

So now you’re not just thinking about conversions and revenue but optimizing the whole customer experience to drive profit. There’s one more element you need to keep in mind. 

The underlying methodology used to perform conversion optimization can be applied anywhere, but ‘CRO’ has taken up root in the marketing/digital department and for many businesses that’s where it stays. 

This is despite the fact that experimentation can, and should be applied across your business, not only to improve key business metrics, but also to help in decision making and generating learnings about your customers or users.

Just because you’re thinking about different touchpoints doesn’t mean the same experimentation methodology can’t be applied. For example; 

  • You can split test adverts and emails to test messaging 
  • You can conduct research and then run an experiment to test new customer service scripts at help centres and measure the impact. 
  • You can run painted door experiments offering faster delivery and manually fulfil them to test the impact it has on the business before committing to a new delivery vendor. 


It’s time we changed our mindsets to something that truly serves the businesses we work with. Whether it's on your website or not, improving your customer experience through experimentation will ultimately play the biggest role in impacting the bottom line. 

Related Posts

Who's currently reading The Experimental Revolution?