Welcome to Briefly Experimental
This edition was written by Paul Randall, CX Strategist at Speero
Every two weeks we'll deliver the best experimentation content and commentary, curated by a member of the Speero team. We'll break things down into the four key pillars needed for any successful experimentation program.
Edition 6, February
Strategy & Culture
🧠 Epistemic humility and why everyone who makes business decisions needs to develop it
An experiment in the early 1960s was conducted where participants' personal beliefs and philosophies were ripped to shreds using techniques developed during World War II to test intelligence officer's strength of character.
Most participants hated the experience, but a few actually liked it. One even found it fun. The reason for these outliers, suggests Psychologist Adam Grant in his book 'Think Again' is because they were open to rethinking their beliefs.
Our cognitive biases can make us dismissive of new ideas. But the ability to adjust your beliefs based on new facts or perspectives is crucial in business. Philosophers call this “epistemic humility”.
This Atlantic article lists ways in which you can let down your guard and be happy to admit when you are wrong.
Process & Methodology
⚠️ Prioritizing experiments from a problem perspective
John Ostrowski, Principal of Growth Experiments, runs, guides, and centralizes experimentation at iTech Media. He was the latest guest on the Testing Insights show with Speero's Ben Labay.
John explained that for him, prioritization happens at three different levels. The first level (which John focuses on) is the customer problem and business metric level. To prioritize at this level John considers three questions;
- Will solving the problem have a positive impact and generate growth?
- Will solving the problem enable market differentiation?
- Is there a natural order of operations that impacts solving this problem?
Watch the latest Testing Insights episode to hear more
🤔 An experimental mindset is for life, not just for websites
Some businesses think of experimentation as A/B tests on websites. Others recognize it can be used across pretty much all business functions, using a range of different methods. But Experience Director, James Chudley suggests we should apply the mindset to our everyday lives too.
For example, by creating ‘prototypes’ for ourselves, we can move forward on previously lofty goals or ambitions, learn quickly and try new things out.
The mindset shift can take your ambition of one day becoming a chef and owning your own restaurant to ‘prototyping’ it by starting a supper club at home. This test allows you to try out recipes and learn if your cooking skills are up to scratch with real paying customers. All useful knowledge to have before quitting your job and investing in a restaurant.
People & Skills
🤝 How to be more inclusive in your next meeting
The pandemic has changed how millions of people communicate on a daily basis, but just like in-person discussions, if the session isn’t planned well you won’t get the most out of it. At worst, you'll actively disadvantage participants if you don't consider how to create a level playing field. For example, how do you ensure those who are dialing into sessions remotely can participant equally as well as members who might be at the office?
For some people, remote meetings have made things better. Former F1 boss, Claire Williams said: "I actually get to talk in these meetings now, because everyone has to listen when you raise your hand when it's your turn. You can't keep talking or trying to get a word in edgeways like in a normal meeting."
Facebook's content designer, Janel Torkington has outlined a few recommendations on how to facilitate inclusive, effective discussion online. The article focuses on first understanding the specific outcome you want, gives you over 40 tools and tactics you can implement right away, and 3 concrete ways to reflect on your progress as a facilitator of digital discussions.
👀 Job opportunities
Here are a few interesting roles that have been posted in the past week;
- Digital Experimentation Manager at Optus (Sydney, Australia)
- Marketing Manager, Experimentation at News Corp Australia (Sydney, Australia)
- Digital Experience Manager at Foot Locker (Murarrie, Australia)
- CRO Manager at Snug (London, UK)
- CRO Consultant at Co-op (Manchester, UK)
- Experimentation Program Manager at HP (Texas, USA)
- Director Digital Optimization & Strategy at Offerpad (Remote, USA)
- Director, Experimentation Program & Digital Tools at Kaiser Permanente (Georgia, USA)
- Experimentation Strategist at Fanatics, Inc. (Remote, USA)
We're looking for someone to join the Speero team too. If you know any great Digital Project Managers looking for a remote UK or East Coast US role, send them our way.
Data & Tools
Data & Tools
📊 Is your data-driven approach holding you back?
Pablo Picasso said, “the problem with computers is all they can do is provide answers.” It’s a profound statement rooted in a binary concept.
While being data-driven seems like a common-sense approach, does it lead us to compute problems in a way that misses some of the picture? Think about the scenario of the World War II bomber planes. Only the ones that returned, covered in bullet holes were analyzed to understand the attacker's strategy and weaponry. But those who were shot down and never returned weren't included, as they didn't have the plane to analyze.
It's this thinking that led researchers Bart de Langhe and Stefano Puntoni to suggest we should instead focus on being decision-driven. It's a subtle difference but accounts for any unknowns, searching wide first before digging into data. It reminded me of the Double Diamond framework for innovation which starts the process with Discovery.
Read the seven ways that decision-driven thinking differs from a data-driven approach
⚒️ 10 testing tools reviewed + pricing
While this article is written by testing tool, Convert, like us they appreciate that different testing tools are suitable for different levels of experimentation maturity. For example, a business just starting out will likely have a low velocity of tests, which aren't very technical. Therefore one of the basic and cheaper testing tools will be best until limitations are felt and it's time to upgrade to a tool with more capabilities.
This is a good guide to 10 popular A/B testing tools, alongside up-to-date pricing (where it's available)
.svg)
