We are often more certain than the evidence really allows.
More quotesBy the time a survey asks the “important” question, it may have spent ten minutes decorating the respondent’s mind. Then the researcher solemnly records the new décor.
List service failures, then ask about satisfaction. Explain every intended benefit, then ask whether the concept seems useful. March employees through workload and poor-management questions, then enquire about staying. Ask shoppers about sustainability before their preference and greener packaging develops a suspiciously helpful glow. The answer can be sincere and heavily shaped by the route used to reach it.
Question order is product design. So are the response options, wording and recruitment. Randomise where sensible, separate priming material from key measures and compare the result with behaviour or another method. Run two orders when a decision matters. If the conclusion changes with the shuffle, the questionnaire has found a steering wheel.
And show the full questionnaire beside the headline statistic. “Forty-two per cent said…” is missing its opening chapters: who volunteered, what they had just considered and which answers were available. A topline without the route is a destination photograph with the roads cropped out. Surveys do not passively collect opinions from jars. They create an experience and then measure the person standing at its end.