We focus on vivid specifics and ignore the wider statistical odds.
More quotesThe telephone survey found no households without a telephone.
Flawless work. Extraordinary result. Fetch the champagne.
The joke matters because bad samples rarely announce themselves so generously. More often they return precise percentages, narrow error margins and conclusions about people who never had any chance of appearing.
Ask former customers through the service they stopped using and the angriest departures may be absent. Study an audience through followers and indifference disappears. Survey staff during optional sessions and the people with least time or trust become statistically silent. The method has edited the population before question one.
Large numbers make this worse by improving confidence in the visible group. Ten thousand biased responses are an exquisitely measured blind spot. Weighting can correct known differences among people observed; it cannot manufacture the views of a group the method never reached.
Perfect representation is usually impossible. The honest task is to describe who could enter, who probably could not and why their answers might differ. That description belongs beside the headline percentage, where it can spoil overclaiming before the conclusion escapes.
There is a commercial temptation to call the reachable population “the audience”. It saves time and tends to produce friendlier results. It may also optimise a product around people already comfortable with it while systematically misreading everyone who struggles.
The missing zero in the telephone survey is comic. The missing customer in a serious decision looks exactly like clean data.