Why using claimed data in general to understand your audience can be misleading

Facebook said Katy Perry’s audience was overwhelmingly female. Spotify said men listened too. Nobody had discovered the One True Audience. They had measured two different acts. A public like is partly a badge. Private listening can be pleasure without the badge. That distinction may decide where the label spends its money because concert promotion and streaming promotion are different jobs.

I distrust the phrase “the data says”. Data is rarely that chatty. Somebody chooses the column, date range and flattering comparison, then wheels it out as an impartial witness. Claimed enthusiasm becomes sacred when it supports the launch. Till receipts become sacred when the launch flops. Miraculously, the best methodology always agrees with the person paying for it. Clicks contain interface design. Purchases exclude people who wanted the thing but could not afford or find it. Reviews over-represent those moved enough to type. Surveys capture language, aspiration and occasional fantasy. Each source records an action in a setting. Its limitations tell us which question it can answer without putting words in its mouth.

When two methods clash, averaging them produces porridge. Name the difference. Ask what Facebook membership signals that Spotify listening misses. Talk to people beside the transaction log. Observe what happens after the confident survey answer meets an actual price. Research resembles questioning unreliable witnesses who saw different parts of the event; declaring one the winner throws away half the scene. Agreement is comforting, but what if the disagreement is the finding?

Behavioural principles

Behavioural ideas at play in this post

Short, plain-English explanations of the principles behind this post, with links to related books and examples in the archive.