The term ‘cohort’ refers to a group of people that share a common experience or event during a certain time span. For example:
- Customers who purchased a particular product in the last 12 months
- People who signed up for the free trial in March
- Customers that have been acquired via direct mail during the Christmas period.
In research terms, an A/B test is a type of cross-sectional study, whereas cohort analysis is a longitudinal study. Both are useful tools in your arsenal when it comes to conversion optimization.
It’s also worth clarifying the difference between a cohort and a segment, as the two often get used interchangeably.
A cohort is a group that shares a common event and time period.
Whereas a segment can be grouped by any characteristic; an event, a time period, or neither e.g a segment could be “all women.”
Carrying out a cohort analysis without measuring results via an A/B test or multivariate test is similar to sequential testing–you make changes to a website and note how your conversion rate changes as time goes on. But remember the value of A/B testing is that it eliminates the confounding variable of time in analysis.
When used together, A/B testing and cohort analysis can tell you a lot of information about how your business is growing and what’s working or not.
What kind of questions can cohort analysis answer?
Someone who starts using your website this month will not have the same experience as someone who starts using it next quarter. That is if you are running ongoing experiments as part of a customer experience optimization program. We can use cohort analysis to investigate the impact different website changes have had on customer lifetime value and retention.
Here’s an example that helps to highlight the value of cohorts more clearly:
From the above information, it’s hard to see whether things are improving or getting worse for old customers vs. new customers. This is because all customers are lumped together.
However, if you laid out the data in a format similar to the table below, you could see that new website users in May are spending more (on average) than new users in January did:
By analyzing your data in this way you can learn more than you would by looking only at the aggregate data.
Cohorts can be used to answer a whole host of different questions. Here are a few examples:
- What was the retention rate of customers acquired from the Black Friday campaign compared to customers acquired at other times of the year? Was the Black Friday campaign profitable in the long term?
- How did different marketing campaigns impact a customer’s average order value?
- Are free trial users from Q1 who engaged with customer support more likely to convert?
- What impact did different product feature releases have on conversion rates?
Which Cohorts Should You Track?
Tracking every single cohort is borderline impossible. So, to preserve your time, resources, and sanity you’ll have to be selective. Here are a few questions you should ask yourself when deciding which cohorts to track:
- Will the data I get from these cohorts produce useful information that can help me to improve my marketing strategy?
- Will I get a clear idea of what’s working and what’s not working in relation to my marketing or A/B test efforts?
- Which targets do I need to hit? Will analyzing this data help me to achieve my goals?
Example: Cohort Analysis for SaaS
There are several ways you can dissect your data to get interesting and useful answers when it comes to cohort analysis in SAAS. Here’s my favorite.
- Free trials: You can look at prospects in the trial that got a specific email follow-up sequence or went through a certain onboarding process. From this, you can see which prospects turned into customers and their lifetime value.
- Purchases: The impact different holiday seasons have on time to purchase or payment type e.g 1 year upfront vs. monthly payments.
- Sales: conversion rate of leads to sales by different salespeople over time.
- Marketing: Which marketing campaigns over which period lead to the highest value customers?
Conclusion
At their core, cohorts are groups that share a common experience across a defined period. They are people that signed up in March or free trial users that began in December, etc. When used with a robust website experimentation program, the insights and results you can obtain are priceless.
.svg)
