The Planning Fallacy

The Planning Fallacy

The planning fallacy is the habit of producing plans and forecasts that sit unrealistically close to a best-case scenario, when statistics on comparable projects were available and were not consulted. Both halves belong to the definition. The term is Daniel Kahneman and Amos Tversky's, developed later with Dan Lovallo, and the phenomenon itself is among the better-supported claims in judgement research. The specific figures most often quoted from the popular account are a separate matter, and most of them are unattributed.

What it is

The characteristic shape is a forecast built by imagining the project going as intended and then adding a margin, rather than by asking how long projects of this kind have actually taken. The result describes the luckiest available future. It is not simply that plans are optimistic. It is that better information existed and went unused.

Kahneman's own case is the strongest illustration precisely because he is the subject rather than the observer. A team he belonged to, writing a school curriculum, estimated about two years. He then asked Seymour Fox, a member of the team and an expert on curriculum development, what had happened to comparable teams. Roughly two in five had never finished at all, and among those that did, none had taken less than seven years or more than ten. The team heard this, carried on, and the project took eight. The relevant statistics existed, were obtained, were understood, and changed nothing, because the inside story of their own project felt more informative than the record of everyone else's.

Kahneman flags his own evidence there, and the flag has to travel with the anecdote. He describes Fox's figures as an informal recollection rather than a compiled dataset, and as not up to scientific standards of evidence. The story illustrates the fallacy. It does not measure it.

In effect

The remedy is not to try harder at correcting for optimism. It is to change the class of information the forecast is built from, by using the statistics of similar completed projects instead of the specifics of this one. That is the outside view, and it is why the two ideas share a chapter. The organisational version, which decorrelates judgements and runs a premortem before commitment, is the part Kahneman argues actually works, since he is explicit elsewhere that individual debiasing barely does.

One public case in the popular account is independently documented and checkable: the Scottish Parliament building, estimated in July 1997 at up to forty million pounds and completed in 2004 at roughly four hundred and thirty-one million.

The other three illustrations in that chapter are not in the same position, and this publication treats them as untraced. A 2005 study of rail projects and a 2002 survey of kitchen remodelling are both quoted with no researcher, journal, sample size or sampling method named, so their figures are not reproduced here. The curriculum team's figures are disclaimed by the author himself. That is an apparatus problem rather than a problem with the concept, and it is worth naming because those numbers circulate widely without attribution.

What it does not say

It does not say that every overrun is a planning fallacy. Projects also run late because the world changes, and the fallacy names a specific failure, the neglect of an available reference class.

It does not license the quoted statistics. Cite the concept freely and cite the figures only after tracing them.

It does not say that awareness fixes it. The curriculum team knew the comparison figures and proceeded anyway, which is the whole point of the story. What changes the forecast is a procedure that replaces the inside view, not an intention to be more realistic.

Sources

  1. Kahneman, D., & Tversky, A. Coiners of the term, with the applied work developed later with Dan Lovallo. Reaching this publication through Thinking, Fast and Slow, ch. 23.
  2. Thinking, Fast and Slow, ch. 23. Source of the curriculum-team story, the Scottish Parliament figures (estimated at up to £40 million in July 1997, completed in 2004 at roughly £431 million), the rail-projects study and the kitchen-remodelling survey.
  3. Seymour Fox's curriculum figures: about 40 per cent of comparable teams never finished, and none of those that did took less than seven or more than ten years. Described by Kahneman as an informal recollection and "surely not up to scientific standards of evidence".
  4. The 2005 rail-projects study and the 2002 kitchen-remodelling survey are unattributed in the body text, with no researcher, journal or sample size given for either. Their figures are recorded as untraced and are not reproduced here.
  5. Evidence status robust as a phenomenon. Systematic optimism in project forecasts survives independently of any single figure above.