We focus on vivid specifics and ignore the wider statistical odds.
More quotesNinety-two per cent satisfied. Available answers: excellent, very good, good, fair, OK. A glorious result for the Department of No Other Options. Its annual bonus is presumably magnificent.
The percentage is accurate in the same way a rigged fruit machine accurately reports its payout. The mechanism selected the outcome before anyone responded. Accuracy inside a crooked frame remains crooked.
Employee surveys pull the same stunt with “good” or “excellent” leadership. Product teams interview active users and announce that churned customers adore the feature they never reached. Marketing tests stop when the preferred headline noses ahead. A restaurant asking only diners who finished dessert can also discover astonishing satisfaction. Precision adds a tie and clipboard to nonsense.
You need not become a statistician. Ask who answered, who vanished, what choices existed, how large the sample was and which comparison gives the number meaning. Ask who commissioned it and what decision the result was expected to justify. Those questions catch a remarkable amount of bullshit before breakfast.
If those basics are unavailable, downgrade the conclusion immediately.
Bad methodology is commercially expensive because flattering research protects the problem. A survey engineered for reassurance cannot expose dissatisfaction early enough to fix it. Better research may make a worse slide and a better product. If a finding cannot survive ordinary questions about how it was produced, it has no business steering money. The percent sign is punctuation, not a certificate of truth.