Experimentation / CRO

Experimentation and Testing Programs acknowledge that the future is uncertain. These programs focus on getting better data to product and marketing teams to make better decisions.

UX / CX Research

Our research programs aim to characterize behaviors and perceptions of your core and adjacent customers.  Why? Because we want to change these behaviors and perceptions for the better.

Data and Analytics

90% of the analytics setups we’ve seen are critically flawed. Our data analytics audit services give you the confidence to make better decisions with data you can trust.

Sample Ratio Mismatch

Sample Ratio Mismatch (SRM)

Sample Ratio Mismatch Alert! What do I do?You have set the ratio for your test at 50/50 and your testing tool has reported that you have a 70/30 traffic split in your experiment. This can not be trusted and needs to be rectified if this experiment's data is to be analyzed.There are several ways that your experiment can have a sample size mismatch, but the most common will be technical issues with the segmentation itself.Use this framework as a guideline to determine what is the best next step once you have received a sample ratio mismatch error:

Use Cases


You have received a Sample Ratio Mismatch (SRM) error from your testing tool or analytics team.

Type:
Ritual
Pillar:
Test & Learn