The Diagnosis
Conversion rate is a business-wide signal shaped by pricing, brand, competitors, and market conditions, not a website KPI.
Treating it as a website problem drives unnecessary redesigns and gets experimentation programs killed for "not moving the needle."
Track program health (test velocity, decision quality, strategic alignment) instead of staring at a single volatile trendline.
Your conversion rate has almost nothing to do with your website.
That sounds wrong. Every dashboard ties it to web performance. Every CRO brief promises to improve it. Every redesign proposal justifies its budget by pointing at it.
I've said this flatly and repeatedly: "Improving your conversion rate is not a strategy. Yes, it's important and it could generate you a lot of revenue, but it is NOT a strategy." Put more bluntly: "The conversion rate of a website is not a purpose or vision, it's a PERCENTAGE."
But conversion rate is shaped by your pricing, your competitors, your brand, your promotions, your stock levels, your market conditions. It reflects how your customer support handles complaints and whether your logistics partner delivered on time last week. It's an indicator of literally everything about the entire business.
So why does every company treat it like a website problem? Because conversion rate is a symptom, and symptoms get blamed on whatever's closest, which is usually the website.
The concept that conversion rate equals website KPI is the root of some of the most expensive mistakes in digital. It's why companies buy new websites they don't need. It's why experimentation programs get killed for "not moving the needle." And it's why CRO teams get stuck defending a number they were never in a position to own alone.
The conversion rate is a business metric that happens to be visible on the website. Once you understand that distinction, it changes how you test, how you report, and how you talk to your leadership about what experimentation actually does.
The Human Brain Wants a Simple Story
Here's why this misunderstanding is so sticky: our brains are wired for linear cause and effect. Conversion rate is low. Conversion rate is measured on the website. Therefore, the website is the problem.
It's clean. It's intuitive. And it's dangerously wrong.
This is the exact logic behind most full website redesigns. Leadership sees a conversion rate they don't like. They map that number to the website. They conclude the website must be broken. So they spend six or twelve months and a significant budget building a new one.
But the website was probably the least significant thing impacting that metric. In fact, targeted, research-backed fixes can outperform a full rebuild: Baymard Institute's large-scale checkout research finds the average large ecommerce site can gain roughly 35% in conversion rate through better checkout design alone, with nearly one in five shoppers abandoning a cart over a too-long or complicated checkout process.
What actually moved the number? Maybe a competitor launched a better offer. Maybe a key promotion ended. Maybe the economy shifted, and customers tightened their wallets. Maybe the product itself stopped resonating with the market. None of these show up in your site analytics. All of them show up in your conversion rate.
The conversion rate doesn't care where the problem lives. It just absorbs everything (brand perception, market conditions, operational quality, pricing strategy) and spits out a single number. Then everyone points at the website team and asks why it's not higher.
The Discount Trap: When Metrics Lie
This gets more dangerous when you realize that improving the conversion rate can actually hurt the business.
Consider a common scenario: you run heavy promotions and discounts. Conversion rate goes up. Revenue might go up too. Even profit, in the short term.
But what did you just do to the customer perception of the brand? What about brand advocacy? You may have entered a race to the bottom with your competitors, one you'll never climb back out of. You attracted discount hunters instead of loyal customers. Research on the empirical relationship between discounting and brand loyalty backs this up: brands with weaker loyalty tend to rely on deeper, more frequent price promotions, a pattern that reinforces exactly the customer relationship you don't want. Your margins are thinner. Your LTV is worse.
None of these consequences show up as metrics in your A/B testing tool. And yet they're real. They're significant. And they're already doing damage.
This is the blind spot of treating conversion rate as the metric. Everything you change on your site is potentially causing wider, systemic effects that data alone can't capture. Some are positive. Some are negative. And you often have no way of knowing which until it's too late.
This is why mature experimentation programs track more than just conversion rate, using frameworks like Goal Tree Maps, a Results vs Actions framework, an Experimentation Decision Matrix, a Strategic Testing Roadmap, and Program Metrics, the blueprints we use internally to separate signal from noise. A dual reporting system is one piece of this: quantitative revenue impact alongside qualitative customer learnings.
The quantitative side tells you what happened to revenue. The qualitative side tells you what happened to the customer. You need both. Revenue without customer understanding is flying blind with one engine.
An A/B Test Is Not a Decision
A related trap compounds the problem. Most teams treat an A/B test result as a decision. Variant B increased conversion by 4%. Ship it.
But that test result is one piece of a puzzle, one input among many, some of which aren't data at all.
Did the winning variant align with the brand's long-term positioning? Does it create operational complexity for another team? Could it erode trust with a specific customer segment in a way that won't show up for months?
An A/B test tells you what happened in a controlled experiment over a limited window. It doesn't tell you what it means for the business. That requires judgment, context, and the kind of strategic thinking that no dashboard provides. This isn't a knock on testing, it's the reality of it: at Microsoft's Experimentation Platform, only about a third of tested ideas actually improved the metric they were designed to improve, even in a mature, well-resourced program.
The Results vs Actions framework exists precisely for this reason. It separates the test result (what did the data say?) from the action (what did you do about it?). A test can win and still not get implemented, because the implementation would disrupt three other teams, or because the lift came from a mechanic that degrades brand perception. Separating these two lets you track your program's agility and decision quality, not just its win rate.
The Experimentation Decision Matrix takes this further. Before a test even launches, you map out what you'll do for every possible outcome. What if the primary metric wins but the secondary is flat? What if you're running a non-inferiority test where "do no harm" is the goal, not a lift? Without this plan, you fall into the trap of torturing the data until the numbers say what you want.
If Conversion Rate Isn't a Strategy, What Is?
"Improve our conversion rate" sounds like a strategy. It isn't. It's a tactic without a direction.
A real business strategy looks more like this: shift our demographic to a younger audience. Or: sell the first product as a loss leader and generate LTV through CRM. Or: diversify into new audiences by creating new use cases.
Now ask: in what way does "improving your conversion rate" specifically achieve any of those? You're getting someone to buy, sure. But are they the right person? Are they buying the right thing? What does the purchase mean for the business beyond the transaction?
There is no reason experimentation can't serve real business strategy, but it requires connecting your testing program to strategic goals, not just pointing it at a conversion rate and hoping for the best.
Goal Tree Maps is built for exactly this. It lets you define strategic KPIs and break them down into smaller, testable metrics. Instead of "improve conversion rate," you get a tree that connects top-level business goals (35% more revenue this year) to tactical metrics (add-to-cart rate, plan selection rate) to engagement metrics (bounce rate, scroll depth). Now your tests target something specific and traceable, not a number shaped by forces outside your control.
A Strategic Testing Roadmap helps you organize those tests around themes and objectives. Rather than a backlog of random ideas, you end up with a quarterly roadmap built on "how might we" questions tied to research insights and business priorities. The whole team runs in the same direction because the direction is defined by strategy, not by a single volatile metric.
The Invisible 20%: Why You Can't "See" Your Impact
This is illustrative thought experiment, not real client dataset.
Imagine a client's conversion rate data plotted by day over twelve months. It's volatile: jagged peaks and valleys driven by seasonality, promotions, traffic mix, and dozens of other forces. Now imagine artificially inflating every value by 20% from a random point onward.
Can you spot where the 20% increase begins?
No. You can't. Nobody can. A 20% improvement disappears inside the natural volatility of the data.
This is why the most common question after running a CRO program for a while is: "Why hasn't our conversion rate increased?" Say the team delivered real, measurable lifts across dozens of tests. When leadership looks at the overall conversion rate trendline, it looks the same, because you can't see incremental improvements in a volatile metric shaped by the entire business.
Year-over-year comparisons don't save you either. They assume last year was identical to this year. It never is. Different competitive landscape. Different economy. Different promotions calendar. Different product mix. The comparison is fundamentally broken.
The brain wants a nice straight line with a visible uptick when good things happen. Reality doesn't work that way.
This is where Program Metrics becomes essential. Instead of staring at the overall conversion rate and wondering where your impact went, you track the health of the program itself: test velocity, quality of inputs, percentage of tests generating new insights, how many tests are backed by research. These metrics tell you whether your engine is running well, regardless of what the conversion rate trendline looks like on any given Tuesday.
Experimentation Is Not a Channel
This needs to be said plainly because the misunderstanding runs deep: experimentation is not a channel. It doesn't generate monthly revenue. It's not a marketing tactic.
Experimentation is a method for making better decisions about how to invest your web development spend. It's part of the overall cost of ownership and development of your digital real estate. It's a way of doing web development, a data-driven approach that prevents you from wasting money.
Think about it this way: how do you measure the ROI of the total cost of an eCommerce website? Most companies compare overall cost versus overall benefit. They don't demand month-on-month attribution of specific hosting costs to specific revenue outputs. Nobody asks "what's the ROI of hosting?"
Experimentation should sit inside that same ROI framework. It's a component of how you build and improve your digital product, not a standalone revenue line.
When you treat experimentation as a channel, you set it up to fail. You demand monthly revenue attribution from a method whose value is better decisions over time. You compare it against paid media, which has a completely different economic model. And when conversion rate doesn't visibly move (because it never visibly moves, as we just covered), you conclude the program isn't working.
The misunderstanding of this is the single biggest red herring that will doom an experimentation program. Not bad test design. Not low velocity. Not statistical errors. The framing.
What to Do About All This
Recognizing that conversion rate is a business metric, not a website metric, isn't just a philosophical shift. It changes how you operate.
- Stop promising conversion rate lifts. Promise better decisions. Promise reduced waste in web development. Promise incremental revenue impact measured at the test level, not the trendline level.
- Treat every A/B test result as one input, not a verdict. Layer in qualitative data, customer learnings, brand considerations, and strategic alignment before deciding to implement. Use frameworks like Results vs Actions and the Decision Matrix to make this process repeatable rather than ad hoc.
- Connect your testing program to business strategy. Use Goal Tree Maps to trace every test back to a strategic KPI. Use a Strategic Testing Roadmap to organize your work around themes, not random ideas.
- Measure your program with program metrics, not by staring at the overall conversion rate and hoping to see a bump. Track velocity, test quality, implementation rate, and the ratio of iterative to disruptive tests. These tell you whether the engine works.
- Learn about systems thinking. Your website doesn't exist in isolation. Every change ripples outward into brand perception, customer experience, operational load, and competitive positioning. The best experimentation programs account for this. They don't just measure what happened to conversion. They ask what happened to the customer.
The Metric Everyone Watches Is the Metric Nobody Owns
Conversion rate will always matter. It's a useful signal. But it's a signal shaped by the entire business, not by your website alone, and certainly not by your CRO team alone.
The moment you internalize this, you stop chasing a number and start building a system: one that makes better decisions, reduces waste, connects testing to strategy, and measures what it can actually control.
That's a program worth defending. It doesn't need to promise conversion rate lifts to prove its value.
Frequently Asked Questions
Is conversion rate a useless metric?
No. Conversion rate is a useful signal, but it's shaped by pricing, brand, competitors, promotions, and market conditions, not just your website. The mistake isn't tracking it. The mistake is treating it as the only metric that matters, or as something your website team can move on its own.
What should replace conversion rate as the primary KPI for an experimentation program?
Nothing replaces it outright. Pair it with program-level metrics like test velocity, the percentage of tests backed by research, implementation rate, and the ratio of iterative to disruptive tests. These tell you whether the program itself is healthy, independent of what the conversion rate trendline is doing on any given week.
Why can't leadership see the impact of a successful CRO program in the overall conversion rate?
Because a real, incremental lift, even a meaningful one, gets absorbed into the natural day-to-day volatility of a metric shaped by seasonality, promotions, traffic mix, and dozens of other forces outside the website's control. Year-over-year comparisons don't save you either, since no two years share identical conditions.
How do you connect A/B testing to business strategy instead of just conversion rate?
Trace every test back to a strategic KPI using a framework like Goal Tree Maps: connect top-level business goals to tactical metrics (add-to-cart rate, plan selection rate) and engagement metrics (bounce rate, scroll depth). Organize the roadmap around themes and research-backed "how might we" questions instead of a backlog of disconnected ideas.
About the author:Jonny Longden is Chief Growth Officer at Speero. He has spent nearly 20 years building experimentation and growth functions inside agencies, client-side teams, and management consultancies, leading product and optimization functions at Sky, Visa Europe, and Boohoo Group before joining Speero in January 2025. At R/GA, he led development of a £2 million ecommerce and optimization technology stack for Manchester United. Connect on LinkedIn.
.svg)
