Why the Planning Fallacy Persists
Students, public works and a curriculum team led by Daniel Kahneman all forecast the best case and ran late. The planning fallacy is the habit of planning from the imagined path and not from the record of similar jobs, and the evidence on why it persists comes from desks, from 258 transport projects and from an argument about motive.
Answer to the question
Why do projects always take longer than planned?
Plans are built from the best-case path through the work, and little weight goes to how similar jobs ended. Students finished theses in 55.5 days against a mean best estimate of 33.9, and costs were underestimated in 86 per cent of 258 transport projects. Asking someone outside the job, or checking comparable cases, gives a later and likelier date.
Key takeaways
- Buehler and colleagues (1994) found that students' honours theses took 55.5 days on average against a mean best estimate of 33.9, and only about 30 per cent finished by their own best estimate.
- Flyvbjerg and colleagues (2002) found costs underestimated in 86 per cent of 258 transport projects, by 27.6 per cent on average, with no sign of improvement over 70 years.
- Observers in Buehler's experiments predicted later completion dates than the people doing the task, 8.5 days against 5.5, and cited the doers' past completion times far more often.
- The 2002 authors attribute the transport underestimation to strategic misrepresentation, a rival to the bias account, and a record of outcomes cannot separate the two causes.
- The United Kingdom Department for Transport introduced reference class forecasting in 2004, and its 2020 update puts the rail uplift at the full business case stage between 19 per cent (50th percentile) and 60 per cent (80th).
A team writing a school curriculum on judgment and decision making was asked to write down, privately, how long the job would take. The guesses centred on two years, the lowest at one and a half and the highest at two and a half. The man who asked, Daniel Kahneman, was one of the two psychologists who had named the planning fallacy.
He then put a second question to the team's curriculum expert: how long had comparable teams taken? About four in ten of them never finished, the expert said, and none of those that did had finished in under seven years or taken more than ten.
The team carried on. It finished in eight years.
Kahneman tells the story in the twenty-third chapter of Thinking, Fast and Slow, and he is careful about its weight. The expert's figures were a recollection, not a dataset, and he says as much. The episode illustrates the planning fallacy and measures nothing. What he took from it, besides the fallacy, was a second name, irrational perseverance, for hearing the odds and continuing anyway, a habit that [Why You Finish the Series You Stopped Enjoying](https://thewhyweproject.com/articles/why-you-finish-series-you-dont-enjoy) follows into everyday choices.
The planning fallacy has a two-part definition. Plans and forecasts sit close to the best-case scenario, and they could be improved by consulting the record of similar cases.
Kahneman and Amos Tversky named it in 1979.
The evidence since then comes from student desks, from public works and from a dispute over motive.
Students predicting their own theses
In 1994 Roger Buehler and colleagues asked psychology students when they would hand in their honours theses. The mean best estimate was 33.9 days. The mean best case, if everything went as well as it possibly could, was 27.4 days, and the mean worst case was 48.6. The theses took 55.5 days on average. Only about 30 per cent of the students finished by their own best estimate, and fewer than half finished by their worst-case date.
Think-aloud records showed what the students were doing. They attended mainly to the plan, the future steps in sequence, and hardly at all to how similar jobs had gone before. Where they did recall earlier overruns, they put them down to causes that were external, temporary and specific to that job. When participants in a later experiment were prompted to link their past experiences to the prediction, the optimism disappeared.
Observers behaved differently. Asked about the same kind of task, they gave later dates than the people doing it, 8.5 days against 5.5, and they cited the doers' past completion times far more often than the doers did.
Bridges, tunnels and railways over seventy years
The students were a small sample on one campus. The public record is larger. In 2002 Bent Flyvbjerg and two colleagues published an analysis of 258 transport infrastructure projects worth US$90 billion in total. Costs were underestimated in 86 per cent of them, nearly nine in ten.
The average cost escalation across all projects was 27.6 per cent. Rail averaged 44.7 per cent, bridges and tunnels 33.8 per cent and roads 20.4 per cent. The authors found no sign that underestimation had decreased over the past 70 years. Their test of whether the year of the decision made a difference found no statistically significant effect.
| Case | Forecast | Outcome |
|---|---|---|
| Curriculum team | About two years | Eight years |
| Honours theses, mean | 33.9 days (best estimate) | 55.5 days |
| 258 transport projects, mean | Estimated cost | 27.6 per cent higher |
Is it a mistake or a decision?
The 2002 paper carries the question in its title, Underestimating Costs in Public Works Projects: Error or Lie? Its authors conclude that the cost estimates were highly and systematically misleading, and they attribute the underestimation to strategic misrepresentation rather than to error. People who stand to gain from a project being approved have a reason to state its cost low, and the approval process rewards them for it.
That is a rival to the account Kahneman and Buehler give, in which nobody needs a motive. Flyvbjerg put the case to Kahneman's side directly in a 2003 comment on an article by Lovallo and Kahneman. He praised their account of overoptimism and the outside view and argued that they underweight forecasts that are deliberately manipulated to get projects started. Because forecasters had not improved over time, he wrote, political explanations fit his data better than optimism does.
The two are less opposed than they look. The students had nothing to gain by flattering a thesis deadline to themselves, and they still ran 22 days over their best estimate, which shows the bias exists without an incentive. The transport record shows that the gap survives in settings where incentives are present as well. A record of 258 outcomes cannot show what was in anyone's mind, so it cannot divide the two causes between them.
The accounts do suggest different repairs. A bias in how people imagine their own work can be countered by changing what they look at. A motive to understate has to be met by someone with no stake in the approval, who checks the number against what happened elsewhere.
Asking what happened to the others
Both repairs lead to the same step, and it is the one Kahneman's team skipped for a year. Kahneman and Dan Lovallo called the two ways of forecasting the inside view and the outside view in 1993. The inside view builds the estimate from the case in hand, its team, its plan and its obstacles. Running through a plan's steps in the mind, the exercise assessed in [Does Mental Rehearsal Help a Decision](https://thewhyweproject.com/articles/does-mental-rehearsal-help-a-decision), is an inside-view exercise by design. The outside view treats the case as one member of a class and starts from what happened to the rest.
Buehler's students, prompted to look at their own earlier jobs, made the move that the planning fallacy calls for, at the scale of a desk. What the move needs is a reference class, and nobody has to feel pessimistic to use it. It also protects the planner afterwards, since a forecast built from the record can be judged on its method and not only on how the project happened to turn out, a trap described in [What Outcome Bias Does to a Good Call](https://thewhyweproject.com/articles/how-outcome-bias-judges-a-decision).
Kahneman's team had the outside number in hand after its first year and went on writing, which makes the practical rule a procedural one: ask someone outside the job for a date, or list the last three similar jobs and what each of them took, and start from those numbers before looking at the plan.
Flyvbjerg turned the outside view into a procedure for institutions, reference class forecasting, which builds the estimate from the recorded outcomes of comparable completed projects. The United Kingdom Department for Transport introduced it into transport appraisal in 2004, as the standard method for adjusting capital cost estimates for optimism, and its rail reference class held 46 projects. A 2020 update recalculated the rail figures from international cost data. For a rail project at the full business case stage it recommends an uplift of 19 per cent at the 50th percentile and 60 per cent at the 80th, so that a cost estimate of 100 is carried forward as somewhere between 119 and 160.
Common questions
Why do projects always take longer than planned?
The planning fallacy is the cause: plans are built from the best-case path through the work, and people give little weight to how similar jobs ended. In a 1994 study students finished theses in 55.5 days on average against a mean best estimate of 33.9. Public works show the same gap, with costs underestimated in 86 per cent of 258 transport projects.
Who coined the planning fallacy?
Daniel Kahneman and Amos Tversky named it in 1979. Kahneman later developed the idea with Dan Lovallo, and in the book Thinking, Fast and Slow he uses a curriculum project of his own, which was expected to take about two years and took eight, as the main illustration.
Does the planning fallacy apply to other people's projects?
Less so. In Buehler's experiments, observers predicted later completion dates than the people doing the task, 8.5 days against 5.5, and referred to the doers' past completion times far more often than the doers did. The bias is strongest when people forecast their own work.
Is the planning fallacy deliberate?
Not necessarily. Students with no stake in the outcome showed it, which points to a bias in how plans are imagined. The authors of the 2002 transport study argue that in public works the underestimation is strategic misrepresentation, a deliberate choice, and the evidence does not rule out that both operate.
Sources
Buehler, R., Griffin, D., & Ross, M. (1994). 'Exploring the planning fallacy: Why people underestimate their task completion times.' Journal of Personality and Social Psychology, 67(3), 366-381. doi:10.1037/0022-3514.67.3.366
Flyvbjerg, B., Holm, M. K. S., & Buhl, S. L. (2002). 'Underestimating costs in public works projects: Error or lie?' Journal of the American Planning Association, 68(3), 279-295. doi:10.1080/01944360208976273
Flyvbjerg, B. (2003). 'Delusions of success: Comment on Dan Lovallo and Daniel Kahneman.' Harvard Business Review, December 2003, 121-122.
Kahneman, D., & Tversky, A. (1979). 'Intuitive prediction: Biases and corrective procedures.' TIMS Studies in Management Science, 12, 313-327. The link is to the 1982 reprint, Judgment under Uncertainty, Cambridge University Press, 414-421.
Kahneman, D., & Lovallo, D. (1993). 'Timid choices and bold forecasts: A cognitive perspective on risk taking.' Management Science, 39(1), 17-31. doi:10.1287/mnsc.39.1.17