Welcome to Briefly Experimental
This edition was written by Edgar Spongolts, COO at Speero
Every two weeks we'll deliver the best experimentation content and commentary, curated by a member of the Speero team. We'll break things down into the four key pillars needed for any successful experimentation program.
Strategy & Culture
📢 The blindspot in Customer Experience
"The human mind does not run on logic any more than a horse runs on petrol.”
- Rory Sutherland
Ben Labay spoke to Mary Drumond, host of the "Voices of CX" podcast on the topic of adding action to CX. They discuss the blind spot for most people working in the Customer Experience space, which Ben calls "actionable measurement." For example, explicit understanding of user perceptions is hard to account for with data alone.
The pair dig into how you can get the right data, contextualize it, and use it in the right way to make smart decisions.
People & Skills
🏆 2021 Experimentation Culture Award Nominees
The shortlist is out. Unfortunately, it's not a particularly diverse line-up this year, which the team behind the awards discuss as a concern and something they plan to tackle next year.
Congratulations from us to all the individuals and companies nominated. Find out who the winners are in a live virtual broadcast on July 8th, 6 pm UTC.
Register for your free ticket here
👀 Brand-side job opportunities
Here are a few interesting roles that have been posted in the last couple of weeks.
- Senior Data Scientist (Experimentation Platform) at Microsoft (Washington, US)
- Director, Experimentation Program & Digital Tools at Kaiser Permanente (Georgia, US)
- Senior Product Manager- Experimentation at Thomson Reuters (Minnesota, US)
- Product Optimization Manager at Seeking Alpha (Remote, US)
- CRO Specialist at Blenders Eyewear (California, US)
- Ecommerce Site Optimisation and Innovation Specialist at Levi Strauss & Co. (London, UK)
- Senior Optimisation Specialist at Dixons Carphone (London, UK)
Process & Methodology
🔨 Ecommerce brands that are nailing CX
This post contains 20 great examples of what brands are doing to extend their relationships with customers after the initial purchase. I particularly like that there are a bunch of examples from lesser-known brands.
This post is likely to give you some fresh ideas of how you could improve your own customer experience.
Read more
⚙️ Work on your process and systems
As dichotomies go, order vs. chaos must be one of the fundamentals. Too much order can be oppressive, but too much chaos will create, well, chaos. In experimentation terms, poor systems and processes lead to sporadic spaghetti testing that delivers little value. Yet the majority of discourse in the industry is about how the test, rather than how to create well-oiled processes and systems to run them.
Practicing what we preach, we've been documenting our operating procedures and standardizing our workflows. The main goals are;
1. Reduce the cognitive load on our project teams, standardizing and automating where possible, so their time can be spent on higher-value consulting, strategy, and ideation work.
2. Better measurement of the work we do. We can apply our own optimization knowledge to become more effective and profitable as a company.
Consider your own experimentation processes; how can you optimize the system itself?
One final tip is to remember that any new system or process will never be perfect. Instead, rely on continuous and candid feedback from your team.
Data & Tools
🥼 The never-ending Frequentist vs. Bayesian debate
Somehow we are still debating Frequentist vs. Bayesian statistics and which is "better". The truth is that either can be effective when you know how and when to use them. Cassie Kozyrkov, Head of Decision Intelligence at Google presents one of the best articles on this rabbit hole of a debate.
Maybe if everyone reads the article we can stop trying to argue over which is "better" and instead focus on which is right for different contexts.
🛠️ Build or buy your experimentation tech stack?
In the latest episode of the Testing Insights show, Ben Labay chats to Kevin Anderson, Analytics & Experimentation at ING.
Kevin talks about the need to consider both team structure and business goals when deciding on the build or buy approach. Kevin also suggests that a better question to ask is: what will be the solution that your teams can most easily adapt into their existing workflow?
At the end of the day, a solution that no one uses isn’t a good solution - or a good investment. Instead start by auditing what you already have, then, buy or build what’s missing.
Watch the full Testing Insights show here.
Kevin also runs his own newsletter called Behavioral Insights which is worth checking out.
🐒 SurveyMonkey is now part of Momentive
Momentive is a newly formed brand that represents its portfolio of tools, including SurveyMonkey. This new name comes with a new pitch too; "an agile experience management company" all very business-speak.
What is interesting is while they plan to maintain their "cost-effective self-serve solutions" (read – SurveyMonkey isn't going anywhere), their focus is shifting to "solving increasingly complex challenges for our enterprise customers." So expect to see some new pricier options rolling out in due course.
A hint to what direction these new tools might take can be found in the new corporate top-level domain choice, momentive.ai...