Welcome to Briefly Experimental
Every two weeks we'll deliver the best experimentation content and commentary, curated by a member of the Speero team. We'll break things down into the four key pillars needed for any successful experimentation program.
This edition was written by Silver Ringvee, CTO at Speero. If you'd like to geek out with Silver on any topic mentioned in this newsletter, just hit reply and you'll reach him directly.
Edition 16, December
Strategy & Culture
🌱 Identifying growth KPIs
Focusing on metrics like total revenue and conversion rate can lead to quick wins but doing this alone can have a negative effect on the business in the long run. Many companies grow via repeat purchases but if you are only tracking revenue and conversion rates you won’t know if you are on track for growth or just throwing money hand over fist to acquire new customers who never return.
Jeff Beckham, head of product marketing at Mixpanel divides common KPIs into two categories;
1. Summary KPIs: these give you a snapshot of business health
2. Growth KPIs: these are the metrics that lead to long-term success
To get your growth metrics, you need to identify who your most valuable customers are and what behaviors or traits they have in common. For example, you might identify that customers who visit your website three times in a two-week period are likely to become valuable customers.
In this article, Jeff guides you on how to identity and measure such metrics and we’ve written about this too, with a specific experimentation focus.
Process & Methodology
🧙 A glimpse into the future? Yes, please
Earlier this month I took part in a webinar hosted by Kameleoon along with John Trimble from TalkTalk, on the topic of price experimentation. It's a topic that can get quite cloudy both in terms of the technicalities as well as the ethics.
As an advocate for doing right by the user, I have some reservations about price testing. Simply showing one price to one user and a different price to another seems like dangerous ground. That said, when I started unpicking this topic I realized that this is not necessarily dissimilar to the dynamic pricing algorithms used in industries such as travel.
For me, price testing is about much more than just testing price. There are so many opportunities to test around the topic of price, e.g. the order you show pricing, how you display discounts and how you communicate value.
Catch up on the discussion by watching the webinar here.
💯 How good is your experimentation program?
If you want to evaluate your experimentation efforts and find gaps and opportunities across strategy, process, people, and data, then head over to our free Experimentation Program Maturity Audit. You can also download the 2021 benchmark report and compare your scores with other businesses.
Free experimentation audit
🐕 Dogs & UX heuristics
I think it's pretty safe to assume most people reading this will either like dogs or like UX (maybe even both?). So what better way to learn about conducting heuristic website evaluations than to tune into this weeks’ episode of the Heuristic Show where the Speero team evaluates UK-based pet food brand, Lily’s Kitchen.
Watch the episode as the team walks through some of the findings, or check out the blog post.
If you are interested in our team conducting a heuristic review of your experience check out our heuristic review service.
People & Skills
📈 Data analytics talent
While the importance of data should be rather obvious these days, research from IDC shows only 33% of employees are comfortable using data analytics to support their decisions. And while 83% of CEOs want their organization to be more data-driven, only 25% are highly likely to rely on data to make major decisions and inform their overall approach to work.
The underlying cause of such statistics, I believe, is a lack of analytics expertise. Finding top analytics talent can be a challenge, especially if you’re looking for someone that knows what data to collect, how to turn raw data into actionable insights, and present it well enough to make an impact.
In his recent article, David Lou, senior product marketing manager at Tableau, recommends involving freelancers on specific projects and I couldn’t agree more. Involving external experts can help get projects off the ground, cover skills gaps or talent shortages as well as help your in-house team to develop their own expertise.
🎓 New course on experimentation program management
Well-executed experimentation programs focus on developing the culture, skills, processes, and structure to support the wider business in decreasing innovation risk, improving growth metrics, or gaining customer insights.
To do that, it's important to learn the frameworks, tools, and rituals needed to run a highly effective program. If you are in charge of an experimentation program and want to work on the program itself, this new course by Ben Labay might be just what you need.
👀 Job opportunities
Here are a few interesting roles that have been posted in the past week.
- Experimentation Manager at Amazon (New York, US)
- Experimentation Enablement Manager at DocuSign (Remote, US)
- Conversion Rate Optimization Lead at Talent Inc. (Remote, US)
- Conversion Rate Optimization Specialist at Dickey's Barbecue Restaurants, Inc. (Texas, US)
- Conversion Rate Optimization Strategist at Spinutech (Remote, US)
- Digital Marketing Ecommerce Manager at McAfee (Texas, US)
- Senior CRO Specialist at IKEA (Malmö, Sweden)
- Senior Experimentation Strategist at Speero (Remote)
- UX Optimisation Manager at The Body Shop (London, UK)
- UX Optimisation Manager at Matalan (Liverpool, UK)
- Optimization Manager at Optimizely (Remote, UK)
Data & Tools
Data & T
🧰 New open-source A/B testing tool
As a CTO, I’ve always been excited when new experimentation-related open-source projects pop up. So, naturally, I was happy to read Jeremy Dorns’ take on open source A/B testing and learn that he's working on a new tool in this space: GrowthBook.
Besides giving the tool itself a try, I’ve been following their blog and product updates and so far it looks really promising. Although almost all of the previous attempts by others have slowly faded away, I have a good feeling about this one and recommend anyone interested to give it a go. They have an active Slack channel for quick announcements and support too.
News & Events
📅 Event: Learn how leading financial services use data and experimentation for faster growth
This is a free virtual event on Friday 10th Dec, brought to you by Segment, Forrester, Speero, and Kameleoon. During the event you'll get;
- Exclusive research by Forrester into banking, data, and experimentation.
- Learn how to stay compliant while optimizing your digital products and marketing.
- Learn how to cross-sell and up-sell customers with relevant banking offers and services.
Reserve your spot and receive an exclusive digest of the key insights from the event and a copy of the Forrester report.
📅 Event: CRO 2022 Predictions: The Year of Customer Lifetime Value Strategy
In DecisionLink's 2nd Annual CRO Predictions Roundtable, they are bringing together people from across industries to share their perspectives on what major trends will impact the office, business, and KPIs of the CRO in 2022.
The webinar takes place on Thursday, January 13th. You can register here
📰 News: Amazon CloudWatch Evidently launches experiments
Amazon has launched new capabilities for developers to introduce experiments and feature management into their application code. CloudWatch Evidently may be used for two similar but distinct use cases: implementing dark launches (feature flags) and A/B testing.
Read more
.svg)
