‍Welcome to Briefly Experimental

This edition was written by Gertrud Vahtra, Experimentation Strategist at Speero
‍
Every two weeks we'll deliver the best experimentation content and commentary, curated by a member of the Speero team. We'll break things down into the four key pillars needed for any successful experimentation program.
Edition 8, April

‍

Strategy & Culture

‍

How to build a culture of data-driven empathy 

Often we see business teams focusing on quantitative customer data that's easy to access. But, quantitative data alone doesn't give you the full context of what's going on. Getting a better understanding of your customer's wants and needs will help you empathize with their situation and lead to meaningful analysis.

Usually, you'll acquire this context through the addition of qualitative Voice of Customer data, by conducting user research, customer surveys, or social listening (such as analyzing social media posts or customer reviews). We've recently written about how to collect passive VOC data and conduct user testing, as well as how to conduct moderated user research if you need a refresher.

The lynchpin for all of this to work, however, is getting the underlying data architecture in place to give you a single unified view of the customer.

Once you have the data, these 8 steps to building data-driven empathy will help you implement the right people and processes to work out what to do with the data once you have it; whether the goal is to improve your services or products, personalizing customer experiences, creating better marketing messages, fixing customer service issues, or fuelling test hypotheses.

‍

‍

Process & Methodology

‍

🚧  Using a phase-gate approach to test the right problems

In experimentation, It’s easy to feel like there are problems that need testing lurking around every corner. But are they problems that will be worth your time to solve?

Ben Labay (Speero MD) chats to Natalia Contreras-Brown, VP of Product at The Bouqs about the phase-gate process used to ensure they are working on the right problems.
Natalia's phase-gate process is used to identify;

• What is the problem?
• Is this the right problem?
• Is this problem worth solving right now?

Catch this episode of Testing Insights, to hear how Natalia bridges product and growth functions, and what role (and skillset) she sees as the lynchpin for any internal testing program.

🤩  21 ways to retain your best customers

“Getting a new customer to try your business can be anywhere from 5 to 25x more expensive than keeping your existing customers, yet, retention programs are often an afterthought.”

But how do you encourage your best customers to continue buying from you?

Matt Shealy proposes 21 tactical ideas to improve retention. These include giving your super users early access to products, creating special help support, and meeting 'unreasonable' requests. If you are working on experiments that look to improve retention, it's important to get the right measurements in place so you can see if your work is, well working. Here's a quick guide on calculating CLTV and other useful metrics.

In order to implement any of the above ideas, you'll need to ensure that you're talking to your customers and collecting feedback from them, to improve their experience in ways that matter to them. There is no “one size fits all” tactic that works for all brands or industries.

‍

People & Skills
‍

‍🏡   Onboarding new remote team members

This year, our team started creating standard operating procedures (SOP’s) to document our processes making the onboarding easier for new team members as well as providing a resource that all employees can later dig into for reference.

Each SOP is baked into our task management tool so our new team members can go through a guided set of tasks with links to documents for knowledge transfer, meetings, courses, video guides and tick off their progress as they work through the onboarding flow in their first couple of months.

But the knowledge transfer and meetings aren't enough if you want new people to feel like part of the team. To tackle the lack of personal connection we created an automated "watercooler" within Slack. This channel uses an Airbase of questions, which automatically posts a new question or a conversation topic every day. If you want to set up your own slack watercooler here's how we did it.

👀   Job opportunities 

Here are a few interesting roles that have been posted in the past week. 

• Marketing Manager of CRO at ADP (New Jersey, US)
• AB Testing Program Manager at Adidas (Oregon, US)
• Senior Experimentation Manager at Zillow (Washington State, US)
• Director, Customer Experience & Experimentation at Malwarebyte (Florida, US)
• Test & Experimentation Manager at Sunrun (California, US)
• Director of Conversion Optimization at SplitBase (Canada, Remote)
• Senior Optimisation Manager at Tesco (Farringdon, UK)  
• Growth Optimisation Specialist at thinkmoney (UK, Remote)
• Digital Optimisation Manager at M&G (UK, Remote)
• Digital Optimisation Lead at Conversionry (Australia, Remote)
  

‍

Data & Tools

‍

🔥   Scaling experimentation velocity through automation

Medha Umarji, Director of Experimentation at Fanatics, has helped scale the company’s testing from 5-10 tests per month to 50-70. When scaling at this level, automation is key. Medha spoke to Speero MD, Ben Labay on the Testing Insight show and discussed;

• Making the most of website traffic. Coding business rules into automations to minimize decision-making. E.g creating business rules for automating the start or stop of an experiment. This means whenever there's an opening on a certain page or domain, the next test should slot into its spot without someone having to manually set the test live.
• Putting a system in place to iterate and learn from every test. At Fanatics, they make sure they have at least 2-3 iterations on each experiment. At this point, 50% of their wins come from these iterations.

‍
Watch the latest episode to learn how Medha implements this approach and tackles problems that can arise from it. Apologies from Ben for the poor audio (and crazy T-shirt) on this one.

😓   Experimentation vendor selection is, like, really really hard

Choosing an experimentation vendor is a difficult and time-consuming process. The question of “which platform is the best” gets asked often, but basing your choice on the number of features or the cheapest pricing isn't the best method.

“They're all very samey with samey features packaged up in a less-so-samey way. Some have more features than others, some better features than others. But the understanding of your business challenges is the most important thing to communicate when in a vendor selection process. It allows those vendors to respond with contextually-specific, persuasive answers on how their platform can support your journey.”

As David Manheim suggests above, the context of your business and experimentation program is crucial; your experimentation maturity and goals of the program will impact what you need for a testing tool.

Read more about the considerations to keep in mind when choosing an experimentation platform.