The problem with opinion polls

George Bishop asked people whether they favoured repealing the Public Affairs Act of 1975. There was no such act. Thirty per cent still offered an opinion, sometimes quite firmly.

The respondent has been placed in a peculiar social situation. A competent adult is apparently expected to know this issue. Saying “I don’t know” can feel lazy, so they borrow clues from the wording, retrieve a nearby political attitude and construct an answer while the pollster waits.

Education changes how comfortable people feel admitting ignorance, which can make the result look like a difference in opinion when part of it is a difference in response style. The questionnaire rarely includes a column labelled “wanted to be helpful”.

The published percentage gives that fresh improvisation the same status as a belief held for years.

Product interviews do this with imaginary features. Staff surveys ask for ratings of policies barely encountered. A forced scale converts ignorance into a midpoint and uncertainty into a decimal.

Knowledge and confidence can be asked separately. Has the person heard of the proposal? Could they describe it? How strongly do they lean? Does a short factual explanation alter the answer? These checks cost space; they distinguish a durable view from something assembled during the interview.

One extra answer would help: “I had never considered this until you asked.” It may be the most commercially useful result on the form. The subject has low salience, weak understanding or both.

How much of an opinion poll reflects durable belief, and how much reflects politeness under pressure?

Behavioural principles

Behavioural ideas at play in this post

Short, plain-English explanations of the principles behind this post, with links to related books and examples in the archive.