Good questions are critical in order to collect relevant and useful data from surveys. If you get this wrong, you can end up with surveys that are annoying for respondents to answer and results that just don’t make sense.
Whether you're designing survey questions for paper and pen or mobile data collection, many of the principles of good question design are the same.
1. Questions should be short and simply worded
Your audience should be able to read and understand your questions without any trouble. If a question takes too long to read, they might skip it or give a nonsense answer. If it’s too hard to understand what you’re actually asking, their answer won’t mean anything.
People are busy and it’s unlikely anyone will expend precious time and mental resources on deciphering your difficult questions, make it easy for them otherwise you might end up with useless data.
2. Don’t use jargon or words not known by the general public
Do not fill your survey with technical terminology and difficult language. If there’s a chance someone in your audience won’t understand a word, then don’t use it.
If you do have to include a term that some people won’t know, make sure you include a definition or explanation too.
The only exception to this is using technical terminology, or organization-specific acronyms in surveys your staff will fill out – this can be appropriate.
3. Questions should be neutral – avoid leading or loaded language
Leading questions suggest to a respondent that a particular response is more desired than the alternatives. Loaded questions use emotive language, or include a presumption in the question that isn’t necessarily true.
e.g., “Do you think the narrow-minded employment policies in this organization should change?”
For more objective results, do not use language that suggests one response is more socially acceptable than another. Otherwise your data will be distorted.
4. Don’t use double-barrelled questions
A double-barrelled question asks two questions at the same time, but with only one response option available.
e.g., "Was this training session enjoyable, and was the teacher informative?"
If a participant in this training session found the teacher informative, but did not find the session enjoyable they would have difficulty providing an accurate answer to this question – particularly if the only response options provided are “yes” or “no”.
5. Ensure response options cover all possibilities and are evenly distributed
If you provide response options, make sure all possible responses are covered and there aren’t any overlapping categories.
e.g., Child’s age in years: "younger than 5", "6-10", "10-15" “16 or older”
If the child is 5 or 10 years old, it is impossible to pick the correct response option.
If you use a scale for the response options, it should be evenly distributed. If it is skewed one way or the other, the resulting data will be skewed to.
e.g., How would you rate your experience?
“OK”, “Good”, “Very Good”, “Excellent”
There is no choice to report a negative experience with these response options.
Try to include all the most common responses, and perhaps add an “other” option to capture anything that falls outside of the common answers.
6. Questions, and response options, should be specific
If you want to know about frequency of a behavior, or about behaviors within a specific time period, make sure you word your question appropriately.
e.g., “Are you hygienic?”
“Do you wash your hands?”
“Do you wash your hands with soap regularly?”
“How many times a day do you wash your hands with soap?”
These questions get progressively more specific, and therefore easier for the respondent to know what is being asked of them. Specific questions also result in more informative responses.
It should be clear to the respondent what each question means, and how they should respond. When writing questions, consider what you would like to know about the frequency, duration, intensity, or type of behavior, and what data you actually want to end up with.
Use these tips to create better questions, and collect better data. If you want to learn more about question design, check out this checklist for good survey questions.
Do you have anything to add to this list? Comment below.
photo credit: FEAST training in Pakistan (license)