It’s one of our favorite parts of the year! This is our fifth annual State of Conversion Optimization report.
In partnership with Convert, we surveyed 399 optimizers from around the world to get a pulse on the state of the CRO industry today. What were some of the biggest challenges in 2020? What should you keep an eye out for in the new year?
Here are four takeaways.
Table of contents
COVID-19 has impacted virtually every facet of society. But, in response to our open-ended question about the “biggest CRO/optimization challenge” optimizers faced, only a handful mentioned COVID-19 by name.
And, it seemed, the biggest impact of the virus on optimizers was how lockdowns affected traffic. Imagine being in the middle of an experiment on a travel site when a country—or continent—shut down (or opened back up).
“We work exclusively with hotels,” reported one optimizer, “so this year’s biggest challenge has been demand/traffic patterns. We’re always dealing with highly seasonal businesses, but this year we have seen 90+% drops in traffic overnight at various stages throughout the year which are impossible to plan for and work with.”
Similar to last year, the two most common challenges for optimizers were the need for better processes and buy-in from decision-makers. For experienced optimizers looking to scale their efforts, they sought company-wide buy-in to create a Center of Excellence or align conversion optimization and brand teams.
Newer optimizers faced more basic challenges—convincing HiPPOs that data provided the best hypotheses or that failed experiments still delivered valuable insight. “The biggest challenge,” said one respondent, “is getting internal stakeholders to truly understand the value of failed tests and that a test failing doesn’t mean CRO isn’t working.”
Many of those “new optimizers” work on smaller sites, which bring their own challenges. “Not enough traffic” was mentioned as the biggest challenge by 10.9% of optimizers, compared to just 6.2% in 2019:
- “Our largest challenge is being highly specialized and we simply don’t get the traffic and conversions that larger, broader companies do because our market is so specialized.”
- “Insufficient website traffic, lack of alternatives during the pandemic.”
Digging deeper, another culprit may be responsible for the uptick in optimizers struggling with low traffic—free or low-cost CRO tools like Google Optimize that have made it easier than ever to enter the experimentation field.
Those low-cost tools continue to attract new entrants, as roughly 56% of the optimizers we surveyed had no more than three years of experience.
As experienced practitioners know, however, access to tools doesn’t ensure good processes—or money-making results.
This year, we asked optimizers two additional questions related to analytics and testing tools.
When it comes to analytics tools, it’s no surprise that Google Analytics is the most mentioned—multiples more than its nearest competitor, Adobe Analytics. Among mouse-tracking and heat-mapping tools, Hotjar was the clear favorite.
So if you want to know where most optimizers are getting their analytics data, it’s a combination of Google Analytics and Hotjar.
What was less expected was the dominance of Google Optimize among testing tools. In the short time since Google Optimize came out of beta (March 2017), it has quickly gained a stronghold in the testing tools space. (Optimizely and VWO earned the exact same number of mentions.)
With the recent launch of GA4, it’s clear that Google will continue to go after the market-share of popular paid analytics and testing tools—it’s difficult to compete against one of the world’s biggest brands and the price of free.
Almost 1 in 20 respondents (4.3%) mentioned using proprietary testing tools, a sign that the industry continues to mature.
Overall, the average salary of CROs remained similar to years’ past. We recorded a slight dip from $64,226 in 2019 to $62,845 in 2020.
However, that decline likely resulted from the high number of new entrants in this year’s survey, many of whom work in countries with lower average incomes. The average salary for U.S. based optimizers—a good bellwether given the quantity of respondents—rose slightly, to just under $103,000.
Some 50.1% of optimizers reported earning less than $50,000, while 17.8% earned $100,000 or more.
Of those with over 7+ years of experience, roughly half reported earning six figures per year. As one respondent shared: “It’s hard to find good CRO specialists with experience.” It pays to be good at what you do.
While in-house optimizers accounted for the bulk of those making less than $35,000 (presumably because of an increase in entry-level jobs in lower-income countries) they also represented the segment with the highest earning potential, consistently reporting higher salaries in the top income brackets.
This year, Australia overtook Switzerland as the country with the highest salary average, followed by the United States. The EU held most of the other top spots.
As we noted in last year’s report: “Since 2017, more than 90% of respondents have described conversion optimization efforts as at least as effective as the previous year; most cite year-over-year improvement.”
We can’t say that in 2020.
This year was the first time since we started publishing these reports in which CRO efforts were reported to be less effective and a lower priority compared to the previous year.
While COVID-19 most certainly played a role, this trend could also be the result of the industry continuing to mature.
Some experienced optimizers reported running out of low-hanging fruit to optimize. Others were working on more ambitious and challenging projects that required company-wide culture change or collaboration.
Reflecting on goals for 2021, here’s what just a few had to say:
- “Promote experimentation culture to management and other stakeholders, hippo culture, risk aversion, lack of training.”
- “Fostering mature cultures of experimentation within client orgs.”
- “Broaden the scope from the “conversion” optimization to optimization of the consumer experience.”
- “Taking the CRO mindset into the offline world.”
On the other side of things, tools like Google Optimize have welcomed new entrants who are struggling with the fundamentals of CRO, such as generating hypotheses, understanding analytics, and having enough traffic to test:
- “Gathering the necessary sample size to measure a certain uplift amount.”
- “Data analysis to create good hypotheses.”
- “Get our team on board and understand the necessity to document, structure and archive. Bluntly: nobody cares right now, and I’m fighting an uphill battle to get it done.”
We remain optimistic. Fewer respondents spoke about trying to develop a “culture of experimentation” in our first surveys, and the cost of enterprise tools prevented many from getting started with conversion optimization.
These are new, challenging problems, but they signal a bright future.
The year 2020 was abnormal in many ways, but they didn’t change everything. Optimizers’ frustration with HiPPO-driven experimentation is still there. A confident grasp on testing statistics is elusive. Everyone would love to develop more winning hypotheses—and roll those tests out with greater frequency.
Here are the things that stood out to us:
- COVID-19 impacted many businesses and CROs, directly and indirectly.
- Google continues to grow its presence in the analytics and conversion optimization space.
- Experienced CROs remain in high demand.
- While effectiveness and prioritization of CRO declined compared to previous years, it may be the result of a maturing industry.
.svg)
