Why updating our beliefs when presented with new information matters

Decision-Making

My colleague Phil Tetlock finds that forecasting skill is less a matter of what we know than of how we think. When he and his collaborators studied a host of factors that predict excellence in forecasting, grit and ambition didn’t rise to the top. Neither did intelligence, which came in second. There was another factor that had roughly triple the predictive power of brainpower.

The single most important driver of forecasters’ success was how often they updated their beliefs. The best forecasters went through more rethinking cycles. They had the confident humility to doubt their judgments and the curiosity to discover new information that led them to revise their predictions.

The salesperson begins with a 70 per cent forecast.

Monday: the buyer delays procurement. 55.

Wednesday: the technical test passes. 63.

Friday: a competitor withdraws. 76.

The following week, a required approval stalls. 48.

The manager circles the five numbers and asks why the forecaster keeps changing her mind. She answers by placing the evidence beside each change. The earlier estimate remains visible, including the confident 76 that lasted four days.

Another salesperson has kept a deal at 70 throughout the quarter and calls this consistency. They have recorded three new risks in the file but excluded them from the probability. Stock and staffing were committed against the original figure. Everyone will discover the true number at the deadline, when it has become useless for changing either.

The team changes the review. Every update must show the previous estimate, new evidence and reason for the movement. Managers question large jumps and estimates kept fixed through material events. People twitch when guessing and hold remarkably steady when protecting themselves.

At month-end, the buyer closes the first deal, making the 48 look pessimistic. Nobody edits it after the fact. The buyer delays the second deal into the next quarter. Its owner updates the forecast from 70 to 10 in one move and describes the outcome as unprecedented.