Welcome to Sideshow episode 5. Sideshow is our new podcast where we interview leaders of growth, experimentation, and research about:
- What’s working these days, the latest techs and techniques.
- Where’s our industry going? How’s AI changing it?
- Experiments tested or insights learned
This time, our guest is Timi Olotu, chief revenue officer at CharlieHR, and he’s here to tell you why Chief Revenue Officers can be indispensable to experimenters: they can translate your results to leadership, help you balance rational and emotional comms, and more.
TLDR:
- Experimentation isn't just a technical activity for A/B tests or conversion rates. It's a fundamental business philosophy centered on understanding cause and effect to drive outcomes.
- The Chief Revenue Officer (CRO) is increasingly a strategic, cross-functional role. They are uniquely positioned to elevate experimentation from a tactical function to a central operating system.
- Many companies experiment poorly because they refuse to acknowledge that their business itself is an experiment. They try to "brute-force" solutions without truly understanding the problem.
- A key part of a CRO's job is to create a compelling, evidence-based "causal chain" that shows how what each team does connects to overall business outcomes.
- A successful leader must balance the data-driven, rational side of experimentation with the emotional pressures of business targets and stakeholder expectations. The goal is to show that a data-driven approach leads to the highest probability of success.
The Philosophy Behind Every Test
You've got a killer experimentation program. You're running A/B tests, optimizing conversion rates, and making incremental improvements. But are you truly moving the needle?
Or are you just messing around with the equivalent of a digital slot machine? The truth is, many businesses see experimentation as a tactical revenue driver—a specific activity you can pump money into and get money out of, like PPC or SEO. This mindset completely misses the bigger picture.
Experimentation, at its highest level, isn't about A/B tests or conversion rates. It's about a fundamental curiosity for cause and effect. It's about understanding what creates outcomes in a business and why things are happening the way they are.
This is where the Chief Revenue Officer (CRO) comes in. While the role of a CRO has historically been occupied by senior sales professionals, it's increasingly being filled by people with backgrounds in marketing, customer success, or even product.
What they all share is a view of the business from a high level, a cross-functional perspective that sees the entire commercial context. This blog post explores how a CRO, with this broader view, can elevate experimentation from a siloed activity to a strategic, central operating system.
Chief Revenue Officers Can Help You Translate Test Results
The role of a CRO is to understand the commercial context of the business and formulate cross-functional strategies to be successful within that context. “It's a role that is expansive and non-deterministic, meaning there isn't a specific, fixed set of things they look at.”
They analyze anything happening commercially to understand the terrain the company operates in and what strategy is needed to succeed in that terrain.
There’s a subtle but important difference between a CRO and a Chief Growth Officer. A Chief Growth Officer is often hired from a background in product or experimentation, with a bias toward technological mechanisms of growth.
A CRO, on the other hand, is often hired to lean into the people side of the business, using existing teams and skills to create growth. However, the philosophical goal of both roles is the same: to find the most high-leverage way to generate more revenue.
Timi, a CRO with a creative and writing background, offers a unique perspective. He views his role as being tethered to the phenomenon itself—the customer's experience—rather than just the methods of testing or understanding it.
“Writing forces you to clarify your own thoughts and simulate the experience of the person reading your words.” This is an invaluable skill in experimentation, which is fundamentally about altering a causal chain.
A writer thinks in terms of many different ways to express the same idea, simulating how someone might react and formulating hypotheses. This ability to simplify complex ideas into something meaningful and clear is a massive benefit for a role focused on driving strategy across a business.
Don’t Go Harder - Understand the Problem
Experimentation is inevitable; the only question is whether you do it well. The problem is that most businesses experiment poorly. “Company formation itself is an experiment, but many businesses refuse to acknowledge this due to hubris.” Instead, they try to brute-force their way to progress.
Those companies think they’ll find a solution to a problem by cycling through all the possible theories and solutions, rather than understanding why the issue is happening in the first place.
Timi gives an example: a company wants customers to buy more, so they send more aggressive emails, which often makes performance worse. They then double down, thinking they need to "go harder," which only aggravates the issue.
A big part of the CRO's job is to demonstrate the pain that exists in the business because it's not engaging with experimentation in a healthy way. You have to show them they've been banging their heads against a wall for years and not making progress, so they understand the value of insights and learning.
Another major issue is the widespread lack of skill in running experiments. The term "experimentation" has become incredibly confused. On one hand, it's seen as an A/B test, confined to a specific piece of software and user experience.
On the other hand, it's just "trying stuff to see what happens". The problem is that many people don't know how to run an experiment that isn't a statistically controlled A/B test.
But a true experiment doesn't need to be an A/B test. It just needs a bit of careful planning. The conceptual ingredients are key: a clear, falsifiable hypothesis, a way to measure the outcome, and a method for interpreting the data.
These are the core tenets of the A/B Testing Workflow Map, which helps you map out the rules and steps for setting up a test, from pre-test analysis to post-test analysis and decision flows.
People often treat A/B tests as if they provide absolute certainty, thinking that a positive result means one thing is definitively "better". But an A/B test only gives you a complex of probabilities to help you make a decision.
And even then, you (the human) have to interpret the results, especially when you're considering things like brand, customer experience, and competitive factors that can't be turned into simple metrics.
This is where the Experimentation Decision Matrix comes in, providing a reference on how to classify results based on the type of hypothesis and the action you plan to take. It helps you move past the idea that "winning" and "losing" are the only things that matter.
From Silos to Systems
The role of a senior leader, like a Chief Revenue Officer, is to create cross-functional initiatives. This is crucial because many businesses are incredibly siloed. The biggest challenge is finding ways to create coordination and collaboration across different areas like product, marketing, and operations.
One of the most important things to do is to construct a compelling causal chain (with evidence, data, and customer feedback) that spans multiple teams. It sounds wild, “but in many companies, no one can conceptualize the relationship between what they do and the business outcomes.”
Another key is to find the common denominator. Revenue is often a side effect of certain things happening with customers.
You have to re-center the conversation and show that everything everyone does is for the purpose of creating a combined net experience for the customer. For example, building a great product makes it easier for marketing to talk about it, which in turn makes it easier for sales to sell it.
This approach is closely related to the Goal Tree Mapping Blueprint, which helps you align your team's efforts with the company's strategic KPIs, breaking them down into smaller metrics and ensuring everyone is working toward the same goal.
This high-level coordination requires a system. The Apollo space mission, for instance, was a system-oriented program with a clear, complex goal (fly to the Moon) that was cascaded down and managed across different projects and teams.
A system-oriented program management enabled disparate teams to test completely different concepts (like rocket propulsion or spacesuits). This kind of coordination happens in big, centralized systems and ways of communicating.
This doesn’t happen in a lot of modern companies. Instead, they bring in resources, create teams, and try to develop a culture where people try to do the right thing.
There should be a software system that tracks strategic goals and initiatives, then cascades them into different initiatives that teams need to achieve.
In the absence of a software system to do this, a person—often a hyper-analytical, systemizing person—becomes the algorithm that propagates the system. This can be difficult because you're often swimming against the natural tendency toward entropy in a business.
These systems can’t just be lifted from one company to another, like a lean canvas framework. A system must be actively engaged with the empirical realities of the business to produce the right outcomes.
How to Balance Rational and Emotional Realities
A CRO has to balance the rational, data-driven approach of experimentation with the emotional realities of aggressive sales targets and stakeholder pressures. It's a complex communication job.
Experimentation professionals can have a purist mindset, wanting to design the most perfect program, while business leaders are focused on hitting targets.
The key is to help stakeholders understand that your goals are identical. The reason for a data-driven approach isn't to take longer or be perfectionistic; it's to have the highest possible probability of achieving the desired outcomes.
Don't talk to them about the intricacies of your testing method. Instead, recognize that you are navigating an emotional system, and use your tools to create alignment and solve problems in the right context.
Ultimately, a CRO is uniquely positioned to bridge the customer side of the business—which a lot of teams are exposed to—with the shareholder side, which has its own set of pressures and demands from finance.
This philosophical and systemic approach elevates experimentation from a tactical "toy" to a central nervous system for the entire business, ensuring that every effort, from copywriting to sales, is aligned and contributing to a shared vision of success.
.svg)
