Research method
On-site Polls
On-site polls are very short surveys that appear on your website for website users to answer. They are typically made up of 1-2 questions (since it is important not to disrupt the user journey). These can be open or closed questions, depending on the research objective. On-site polls can be run site wide or on specific pages, and can be triggered on a range of behaviours such as time on site / page, scroll depth and exit intent (on desktop).
When to reach for it
Gather feedback as users are experiencing a website to help understand key barriers from users who haven’t converted yet. This makes them an effective method for gathering insight into UX friction and barriers to conversion. On-site polls can also be used to understand more about external factors, such as intent or motivation for visiting the website, or to understand more about the types of people visiting the website.
You can also be creative with the way you use on-site polls; for example you could use an on-site poll to recruit users for other research methods such as interviews or usability studies. This can be particularly useful when looking to target users who have a genuine need for your product or service, but who have not yet converted.
Strengths and limits
Pros
On-site polls provide insight from users who have naturally found their way to your website, meaning they have a genuine need for your product or service.
By their nature, on-site polls allow you to gather insight there and then while the user is interacting with your website. This means feedback is almost immediate and prevents the user having to recall their previous experiences (which can introduce a lot of bias).
Cons
The main downfall of on-site polls is the disruption they can cause to the user experience. By appearing potentially mid-way through their journey, a poll could frustrate or disrupt, which could negatively impact brand perception and likelihood to convert. This can be somewhat mitigated by using logical poll design, e.g. making the pop up relatively subtle, using colours that match the website look and feel, etc. You should also consider what other pop ups you may have on your website. If you already have other pop ups such as offers, live chat, etc. then adding another pop up could exacerbate users frustration.
The response rate is often relatively low. When planning your poll you should expect less than 10% response rate. You need to look at your current traffic levels and poll triggers to determine how long your poll will need to run. If you're going to have to run a poll any longer than 4 weeks to reach a minimum of 200 responses, then a poll may not be the most suitable research method for you.
The nature of on-site pop ups (and their propensity to cause frustration) can sometimes mean a high amount of "spam" responses. This means users who simply answer to get rid of the pop up, or answer saying "this pop up" is their main frustration (don't be surprised to see some swear words within the responses!) Review your responses and filter out any spam before ending a poll, ensuring you have reached your sample size with quality responses.
Practical considerations
Questions / Logic
There are different ways to approach on-site poll question design. Some schools of thought suggest that asking an "easy" question first, such as a closed Y/N question, can help "hook" users into answering more open questions. For example, "Can you find all the information you're looking for on this page?". However, in reality, there is often a large drop-off in response rate between the first and any subsequent questions. You need to consider the impact of this, e.g. you may come away with only insight on the % of users who answered Yes and No, which is not necessarily actionable.
As such, you may wish to start with a more open question. This may reduce overall response rate but increase the quality of the insight, e.g. "What information, if any, is missing from this page?". But again, by asking this, you won't gather the top-level data on no. of Yes or No responses.
An effective method is to run a series of polls. E.g. start with "Can you find all the information you're looking for on this page?". Then, once you have gathered sufficient data, run a second poll "What information, if any, is missing from this page?" It may be that a high percentage of respondents answered the first poll with "Yes", which could suggest that the information on the page is sufficient and efforts may be best used elsewhere.
Tools
Hotjar is one of the most popular tools for running on-site polls, due to its relatively cost-effective pricing model (although this has changed somewhat over recent months). There are many tools available with differing capabilities. When considering a tool the main factor to think about is the triggering capabilities; you want flexibility in who and how you target the poll to ensure you gather the most relevant insights.
Sample Size
We need approximately 200 open-ended responses to be confident in our strength of signal within the analysis (have a 95% confidence that the answer strength of the signal is valid within ~5-10% of what is reported).
Common tools
Go deeper
Recipes that use this method
A method earns its value when its signal corroborates others. Here is where this one does its work.
Last updated
.svg)
