This is the fourth edition of our State of Conversion Optimization report. The upward and downward trend data is increasingly interesting.
But it’s also easy—and equally useful—to spot the things that haven’t changed: the strategies that continue to work and the issues the industry hasn’t yet solved.
In partnership with AB Tasty, we gathered opinions from nearly 400 optimizers working in more than 50 countries.
We identified 7 key takeaways.
1. Process is vital—and difficult.
“In one sentence or phrase, what is your biggest CRO/optimization challenge?”
That was the sole open-ended question in the survey, and one challenge was top-of-mind for more than 1 in 4 optimizers: process.
“Better processes” included a desire to implement tests more quickly, correctly, and consistently. Three process-related issues showed up more than any others:
Optimizers are struggling to determine what they should test and the order to test it. Perhaps unsurprisingly, those roadblocks can also slow the rate of testing. (Limited access to developer resources doesn’t help either.)
Process challenges aren’t new: “Establishing a process” was the top issue cited in open-ended responses for the 2018 survey.
We also asked a direct question about process, which reinforced the open-ended finding. While most use some process, only 44.6% use a documented one. Nearly 1 in 5 optimizers doesn’t have a process.
Also, about one-third of optimizers had no process in place for user research or test prioritization.
For those with a prioritization framework, most have created their own:
The second most common challenge—winning buy-in from clients and C-suite decision-makers—mirrored last year’s report as well. It also highlights how soft skills can be critical to moving a program forward.
2. A lack of soft skills is holding back CRO programs.
If you want to feel pain, you need only browse responses from optimizers struggling to get buy-in from clients and in-house executives:
- “I'm not being given the time or resources to work on CRO the way I should be working on it (CXL model) even though a few small optimizations have brought in huge gains. It's more of a political challenge than a numeric one.”
- “CRO is used as a generic term to test something seen on a competitor’s website, so the majority of tests that have been run in the past don't start with data.”
- “In India, CRO is an afterthought. Challenging the status quo and educating the senior executives for buy-in and continuous iteration is a big challenge, even in Internet-only companies.”
- “As CRO becomes more mainstream, more people have a little knowledge. But a little knowledge is sometimes dangerous, and a lot of re-education is required.”
Skepticism or ignorance about conversion optimization affects more than the ability to prove ROI or retain clients. It also impacts implementation—executives are often anxious to end tests prematurely, if they’re willing to test at all.
That may be one reason why most optimizers still don’t have a standardized stopping point for A/B tests. The sub-50% figure has remained constant over the last three years.
Buy-in and budget, of course, are closely aligned. Developers aren’t cheap, and limited access to developer resources, as noted earlier, negatively impacts testing velocity.
Nearly two-thirds of all optimizers run no more than 1 test per week; around 2 in 5 are lucky to run 2 tests per month.
Nascent programs with limited buy-in also stick to the basics when it comes to experimentation.
3. Most optimizers (1) use analytics data to inform (2) A/B tests of (3) UX or design changes on (4) desktop and mobile websites.
For the third consecutive year, digital analytics was the most effective component of conversion optimization. It makes sense: Everyone has analytics data; not everyone can afford eye-tracking studies.
And what can you do with that analytics data? Run A/B tests on UX and design elements—the next two most useful components for “decision-making insights” in CRO programs.
A/B tests are the preferred method of experimentation by a longshot. More than 90% of optimizers run A/B tests; no other test type is run by a majority of practitioners.
Also not surprisingly, most optimizers are working on websites, not apps or other digital products.
Those details build a picture of the day-to-day work for most optimizers: Running A/B tests based on analytics data for UX or design changes to websites.
That simple formula may dominate, in part, because the industry remains young, in age and experience.
4. It's still a young industry—in multiple ways.
Nearly half of respondents were 30 or younger; some 87% were 40 or younger.
Given that breakdown, the median age (31) is slightly higher than you might expect (and is similar to the average age of 32.8). The demographics skew a bit younger compared to two years ago, when the median age was 33 and the average age 34.4.
The gender gap persists, even among new entrants. Some 27% of all respondents were female, similar to previous studies. The figure was the same for younger cohorts: 26% of those under 30 were female, and 28% of those under 25 were women.
Youth also means inexperience. More than 30% of optimizers have been at it for no more than 1 year, and most (60.6%) have worked in conversion optimization for no more than 2 years.
There’s a sharp drop-off past the 5-year mark:
Those with a decade or more of experience are a tiny minority—just 4.5%.
The trend toward youth may also hint at the increasing popularity of conversion optimization. More jobs means more entry-level hires, a challenge in itself for growing agencies and in-house teams.
5. It's tough (and expensive) to find and develop talent.
Human resources challenges go beyond access to developer talent. Many respondents were sole practitioners on their marketing teams or had CRO responsibilities bundled into other marketing activities.
That led to a common refrain: a lack of time, which plagued more than just solo “teams.” As one respondent noted: “We are only a four-person team, so having the time to actually do it is the biggest challenge.”
Certainly, hiring conversion optimization specialists isn’t cheap, even as average salaries have remained relatively consistent in recent years:
That data should be taken with a grain of salt—the variation hinges on the pool of respondents. More respondents from North America and Europe, as you might imagine, raises the average.
In fact, the average U.S. optimizer made $99,012 in 2019, about $4,000 more than in 2018, and more than the average optimizer anywhere else. Here’s how other countries stacked up (limited to countries with at least 5 responses):
Even for experienced practitioners, the need to learn more—or merely keep pace with changes—was urgent, especially when it came to statistical knowledge:
- “Lack of statistical knowledge in the team”;
- “Understanding statistical models”;
- “Figuring out exactly when a test is statistically relevant.”
(If only there were a course for that…)
Others cited a general anxiety about not knowing enough or falling behind. That concern is well-founded, as what you don’t know may indeed hurt you.
6. What you don't know may hurt you.
How’s your personalization program going? What challenges might server-side testing solve? For most optimizers, the answers are “What personalization program?” and “Not doing it, don’t know.”
Personalization was low on the list of effective conversion optimization efforts, but that may relate to implementation challenges, something several optimizers listed as their biggest CRO issue.
Those who use personalization (46.2%) appear to be having success. An overwhelming majority plan to expand those efforts in 2020.
That’s a surprisingly strong endorsement of personalization. If you’re struggling with implementation, it may be worth the effort.
A greater knowledge gap appears when it comes to server-side vs. client-side testing. We’ve written about the differences and potential advantages before. But those takeaways remain a mystery for most optimizers, who primarily run client-side tests.
Those who don’t run server-side tests are unaware of the potential benefits:
That lack of knowledge harkens back to a key struggle for optimizers—continually growing their knowledge base of conversion optimization strategies and how to execute on them.
You may not have the resources to run a server-side testing program, but you’d better know the potential value when you’re deciding how to scale your program.
Despite these challenges, there’s plenty of reason for optimism.
7. CRO continues to be more effective, a higher priority, and a bigger piece of the marketing budget.
Several key indicators confirm the growth of conversion optimization as a discipline. Since 2017, more than 90% of respondents have described conversion optimization efforts as at least as effective as the previous year; most cite year-over-year improvement.
Similarly positive trends show up for how conversion optimization is prioritized within organizations:
And what about the all-important budget question? That trend is positive, too:
The growing pains of hiring, scaling, and getting executive buy-in, in other words, need context.
Each year for the past several years, CRO has been more effective, a higher priority, and a better-funded component of companies’ marketing programs.
Conclusion
Each year, some things change. Others don’t. Both warrant attention. These seven takeaways offer a portrait of the CRO industry in 2019:
- Process is vital—and difficult.
- A lack of soft skills is holding back CRO programs.
- Most optimizers (1) use analytics data to inform (2) A/B tests of (3) UX or design changes on (4) desktop and mobile websites.
- It's still a young industry—in multiple ways.
- It's tough (and expensive) to find and develop talent.
- What you don't know may hurt you.
- CRO continues to be more effective, a higher priority, and a bigger piece of the marketing budget.
.svg)
