Six forecasters beat one star

Some economists are better performers than others. Indeed, the top forecaster was around 5 per cent more accurate than an average forecaster. But then Soll added a twist. Instead of looking at individual predictions, he took the average prediction of the top six economists. To stretch language a little, these forecasters were being placed into a team. The average of their predictions is what you might call their collective judgement. Soll then checked if this prediction was more accurate than the top-ranked economist. Now, in a simple task, the answer must be ‘no’. In a running race, the average time of six runners has to be slower than the time of the fastest runner. This is what Justice Scalia had in mind when he argued for a trade-off between diversity and excellence. But this analysis flips when we move from simple to complex problems. Indeed, when Soll compared the collective judgement of six economists with the judgement of the top economist, it was not less accurate, it was more accurate. And not just a little more accurate but 15 per cent more accurate. This is a staggering difference - so large, in fact, that it shocked the researchers.

The best economist in the group beat an average economist by about five per cent. The average of the top six beat that star by fifteen per cent. This is offensive to every institution that wants one clever adult at the front of the room.

Complex forecasts contain several ways to be wrong. One person overweights recent inflation. Another underestimates a policy delay. A third knows one sector unusually well and assumes the rest behaves like it. Average genuinely independent estimates and some errors cancel. The combined answer contains pieces no individual expert owns.

Independence is the expensive word. Six people around a table are not six judgements once the first confident number has been spoken. The senior economist says 3.2. Somebody holding 2.4 wonders what they missed and moves to 2.8. Another chooses 3.0 to avoid looking dramatic. The spreadsheet now contains six entries produced by one social event.

Collect the numbers before discussion. Keep the initial estimates visible. Ask what information drove the outliers. Combine answers by a declared rule rather than by whoever speaks with the best radio voice. Diversity in photographs contributes nothing; the variation has to exist in information, assumptions or method.

Strong forecasters supplied the initial estimates. Their expertise improved the individual judgements; aggregation reduced part of the error remaining across them. The organisation receives a central estimate and a distribution showing how far the informed judgements differed.