Research method
Moderated Usability Studies
Use a sample pool of participants who match your target audience. Watch and listen as they walk through your site and perform queued tasks. Moderated usability studies involve one-to-one sessions conducted by a skilled moderator who will set the tasks and questions, allowing for a deeper level of insight. Moderated usability studies can be conducted in person or via Zoom, Teams, etc.
When to reach for it
A requirement to understand the wider user journey, e.g. how users shop / browse more generally. A need for more in-depth, qualitative insights, a longer / more detailed user journey / set of tasks (that would be difficult to explain in a remote setting).
Strengths and limits
Pros
Moderated usability studies give more control to the moderator, since they are running the session live. This means they can answer questions from the participant and / or bring the participant back on track if they get stuck or misunderstand the task.
The insights from moderated usability studies tend to be more in-depth than those from remote. Again, this is because the moderator is able to ask follow-up questions, ask for clarification, etc.
Cons
The success of moderated usability studies is dependent on the skills of the moderator. Sessions can be subject to bias if moderators ask leading questions or tasks.
Sessions can also be subject to bias such as the Hawthorne effect, since the participants are being observed live.
Practical considerations
Tool
There are many remote usability study tools, with slightly different capabilities and pricing models. Much like with AB testing tools, the success of your research is not necessarily determined by the tool you use, but by what you do with it (e.g. accurate recruitment, well-crafted tasks, and questions, etc). Key considerations when picking a tool include; the size and quality of their participant panel, the ability to add a recruitment screener, the analysis capabilities, what's included (e.g. participant incentives). Also double check how their pricing works, e.g. does it scale up based on the recruitment criteria and estimated length of session. Most tools now offer the option to set up and recruit for moderated usability studies in a similar way to remote; you will specify the moderator's availability and suitable participants will book in to the available slots.
Recruitment
For moderated usability studies, a tool like Userlytics or Usertesting.com is not always necessary. You can use another recruitment method and use any video call tool to run the sessions. Alternative recruitment methods include recruiting from existing customer database, or recruiting via an on-site poll. But be careful, as doing it this way increases the chance of no-shows, it also requires some admin from your side in terms of booking in sessions and paying incentives.
Sample Size
Nielsen Norman says "The answer is 5, except when it's not" regarding usability study sample size. We use this as a guide; but one that should be considered rather than followed blindly. For example, if you're testing on multiple devices or with different user types, then you need to apply this rule to all of those segments. You may also want to add 1-2 additional "buffer" sessions to account for poorer quality participants, tech issues, no shows, etc.
Common tools
Go deeper
Recipes that use this method
A method earns its value when its signal corroborates others. Here is where this one does its work.
Last updated
.svg)
