Why Do Projects Always Take Longer Than Planned?

A kitchen renovation, a thesis, a software project, a house move: almost everyone has lived through one that took far longer than planned. Often far longer than the last one took, too, even though we swore we’d learned.

It’s so common it has a name: the planning fallacy. And it happens even when nobody is being careless or lazy. Add steps to the project in the toy at the top and see how often it finishes on time, even when each step is estimated fairly.

Late, even when you knew

In 1979, Daniel Kahneman and Amos Tversky described the planning fallacy: the tendency to underestimate how long a task will take, even when we know that similar tasks have taken longer before. The second half is the strange part. It isn’t only that we don’t know. We know, and plan as if we didn’t.

In a well-known 1994 study, the psychologists Roger Buehler, Dale Griffin and Michael Ross asked students when they’d finish their final-year theses. On average, the students predicted about 34 days. Asked for a worst case, “if everything goes as badly as it possibly can”, they said about 49 days. The average actually taken was about 55 days. Fewer than a third finished by the time they’d predicted, and most didn’t make even their worst case.

Kahneman tells a story against himself. He once led a team of experts writing a school curriculum and a textbook.

Why plans run late

Part of the reason is in the shape of the risks, which the toy at the top shows.

• A step can only finish a little early, but it can run very late. If a job is planned at two days, the best case might be a day and a half; the worst case could be a week. Even if each step is as likely to be early as late, the lates are bigger than the earlies.
• Steps add up. Across many steps, the big delays almost always outweigh the small savings, so the whole project usually runs over, even though no single step was badly estimated.
• We plan the best case. When we picture the work, we imagine it going smoothly. We don’t picture the illness, the supplier delay or the part that needs doing twice, because we can’t know which one it will be.
• Something always comes up. Each individual problem is unlikely; that some problem happens is very likely. With twenty things that each have a one-in-ten chance of going wrong, the chance that none of them does is about one in eight.

There are also less innocent reasons. People asking for money or approval have an incentive to make a project look quick and cheap, and the danger of an optimistic estimate falls on someone else later. The planning researcher Bent Flyvbjerg calls this “strategic misrepresentation”, and argues that for large public projects it can matter as much as honest optimism.

Megaprojects

The planning fallacy is most visible on the largest scale. The Sydney Opera House was meant to open in 1963 at a cost of about 7 million Australian dollars. It opened in 1973 and cost about 102 million, around fourteen times the estimate.

It isn’t an exception. Flyvbjerg has built a database of thousands of large projects, from bridges and dams to IT systems and the Olympics. In his book with Dan Gardner, How Big Things Get Done, he reports that fewer than one project in ten came in on budget and on time, and only about one in two hundred delivered on budget, on time and with the benefits promised. The pattern runs across countries, sectors and decades.

The fix: look at what happened before

The most reliable correction is to stop estimating from the inside, step by step, and look at similar projects from the outside. Kahneman called these the inside view and the outside view. The method of using the outside view is called reference class forecasting, and Flyvbjerg developed it for practical use.

Instead of asking “how long will my steps take?”, ask: “how long did projects like this take, for us or for others?” If your last three renovations ran 50% over, plan for this one to do the same.

It works because the track record already contains all the problems you can’t foresee. You don’t need to know that the supplier will be late or that you’ll catch flu in March. You just need to know that, in projects like this, something like that usually happens. The British government now requires a version of this, adding an “optimism bias” uplift to the cost estimates of big public projects.

Hofstadter’s law

The writer Douglas Hofstadter put the frustrating heart of it into a joke: things always take longer than you expect, even when you’ve taken that rule into account. You can know all about the planning fallacy and still fall for it, because knowing doesn’t change how your mind builds a plan. It builds the smooth version.

That’s why the fix has to come from outside your head. Data from past projects, someone else’s estimate, a written record of how wrong you were last time: these work where good intentions don’t.

Other ways to plan better

• Add a buffer you take seriously, sized from past overruns, not a round number. And don’t spend it on day one.
• Break the work down, and estimate the uncertain steps as ranges (“2 to 5 days”) rather than single numbers. Studies find that estimating the parts separately tends to give longer, more realistic totals than estimating the whole.
• Ask someone else. People are often better at estimating other people’s projects than their own, because they don’t picture the smooth version.
• Do a pre-mortem. Imagine the project is late and ask what went wrong. It surfaces risks that a plan built on hope leaves out.
• Track your estimates. Write them down and check them afterwards. It’s the fastest way to learn how much you, personally, underestimate.
• Plan backwards from a deadline with the slack at the end, not scattered through the steps, so it’s visible and protected.

A worked example: the kitchen

Say you’re planning a kitchen renovation and you list the steps: strip out the old units (2 days), electrics and plumbing (3 days), plastering (2 days, then a week to dry), fit the new units (3 days), worktops (a 2-week wait after measuring, then a day), tiling and painting (2 days). Add it up and you get about four weeks.

Now take the outside view. Ask three friends who’ve done kitchens, or a builder, how long theirs really took from first day to finished. Suppose the answers are seven, eight and ten weeks. That’s your reference class. The inside view wasn’t wrong about any single step; it just didn’t include the plumber who couldn’t come until Thursday, the cracked tile that needed reordering, the wall that turned out not to be square.

A sensible plan here is about eight weeks, with the slack kept at the end and protected. If it takes six, you’re delighted. If you plan for four, you’ll be living with a microwave in the hall for a month longer than you told everyone.

What it doesn’t mean

It doesn’t mean planning is pointless. A plan tells you what has to happen and in what order, and that’s valuable even when its dates are wrong. It doesn’t mean optimism is bad either: optimism is part of why anyone starts a renovation or a thesis at all. The point is to keep the optimism for the decision to start, and use the track record for the date you promise.

Test yourself

The free thinking traps test has a planning question alongside other predictable mistakes that catch careful people, and the sunk cost fallacy covers what tends to happen once a project is already late.