Surveys are the easiest research method to run badly. Anyone can drop ten questions into a form tool and hit send, and that is exactly the problem: nothing in the tooling enforces good survey design, so a questionnaire that measures something real looks identical to one that produces confident nonsense, right up until the data arrives. The difference is made earlier, in choices most teams rush.
Good survey design is mostly the discipline of asking less: fewer questions, worded more carefully, placed in a deliberate order, and tested before launch. This guide walks through the practices that separate a survey you can defend from one that merely went out.
Survey design starts before the first question
The first best practice has nothing to do with wording: check that a survey is the right tool at all.
A survey is a quantitative, attitudinal instrument. It measures what people say, at scale. That makes it excellent for sizing things you already understand: how common a behaviour is, how satisfied different segments are, which of several known problems bites hardest. It is a poor tool for discovery. If you cannot yet name the answer options, you are not ready to write the question, and a handful of interviews will teach you more than a thousand responses. The classic sequence is qual then quant, and our guide to qualitative and quantitative research walks through it properly.
Be honest about the say-versus-do gap too. If the real question is "what do people actually do", analytics or observation will beat self-report every time. People answer in good faith and misremember all the same, and a survey inherits that unreliability wholesale. If you are unsure which method fits your question, our guide to choosing a research method covers the decision.
One more filter before you write anything: every question must map to a decision. If no decision changes based on the answer, cut the question. Shorter surveys get better completion and better attention, and respondent attention is the scarcest resource you have.
Writing questions people can answer honestly
Most survey bias is baked in at the wording stage, and most of it follows a few well known patterns:
- Leading questions. "How much did you enjoy the new dashboard?" presumes enjoyment. Neutral framing, "How would you rate…", gives people permission to be negative.
- Double-barrelled questions. "Was the product easy to use and good value?" is two questions wearing one question's clothes. Whichever box people tick, you cannot tell which half they answered.
- Loaded and jargon terms. Words like "innovative", or internal feature names nobody outside the building uses, smuggle assumptions into the question. Use the language respondents themselves use.
- Vague quantifiers. "Do you usually…" invites guesswork. Anchor recall to a concrete window: "In the past seven days, how many times…". Memory degrades fast, so keep recall windows short.
- Hypotheticals. "Would you pay for…" measures optimism, not intent. Ask about past behaviour wherever you can, because the best predictor of what people will do is what they have already done.
A useful habit: read every question aloud and ask what a bored, honest stranger would make of it. If it needs explaining, it needs rewriting.
Scales and answer options
The answer options do as much work as the question, and they carry their own traps.
For rating scales, keep them balanced, with as many positive options as negative ones. Label every point with words rather than numbers alone, because "4" means different things to different people while "fairly easy" does not. Keep the direction consistent through the whole survey: flipping midway is a reliable way to manufacture noise, since a portion of respondents stop reading carefully and answer on autopilot.
Be wary of long batteries of agree or disagree statements. They invite acquiescence bias, the well documented tendency to agree with statements regardless of content. Item-specific questions, such as "How easy was it to…" with an easy-to-difficult scale, are slightly harder to write and noticeably more reliable.
For multiple choice, options must be mutually exclusive and collectively exhaustive. Always offer an honest exit: "not applicable", "don't know", or "other". Forcing a choice does not create data, it creates fabrication, and you will never know which rows are made up.
Five or seven points on a scale is plenty. Beyond that, the extra precision is imaginary.
Order effects, and why you always pilot
Questions do not sit in isolation: each one primes the next. Ask detailed questions about frustrations and then ask for an overall satisfaction score, and the score will sag under the weight of everything you just made people remember. Ask the general question before the specific ones. Put sensitive questions and demographics at the end, once some trust and momentum exist. And where the order of answer options is arbitrary, randomise it, because early options get chosen more often simply for being early.
Then pilot, every time. Give the survey to a handful of people from your target audience, ideally on a call, and ask them to think aloud as they answer. You are checking whether the questions mean what you intended, how long completion really takes, and where people hesitate, misread, or give up. Then look at the pilot data itself and confirm you can actually run the analysis you planned on data shaped like this. A pilot costs a day. Discovering a broken question after two thousand responses costs the whole study.
The survey you can stand behind
A well designed survey is, frankly, a little boring: short, plainly worded, carefully ordered, tested before launch. Nothing about it shows off. What it buys you is the only thing that matters, results you can defend when someone asks how you know. Ask less, ask it neutrally, pilot it properly, and when a stakeholder wants to bolt on five more questions at the end, remember that every addition is paid for in the quality of every answer above it.
People of Research is a community and newsletter for UX researchers, market researchers, and everyone who asks people questions for a living. Join us and get the good stuff in your inbox.