Forecasting Blues (and what to do about it)

---

The Boeing Dreamliner has been delayed 5 times and is more than 2 years behind schedule, severely denting corporate credibility and profits. Standish reports that 68% of IT projects deliver over budget, are late or missing key features, with 24% of projects being abandoned or canceled. Flyvbjerg et al. (2002) report on how public works projects are consistently over-budget by an average of 28%.

This inability to produce reliable forecasts has costs to the credibility of the organization, leads to disappointed investors and customers and makes efficient resource management next to impossible.

Why are corporate forecasts often so wrong and what can be done?

I’m writing a series of blog posts to answer this question. Each post will deal with a particular reason for bad forecasting including:

1. Over optimism
2. Strategic misrepresentation
3. Anchoring and adjusting
4. Organizational hierarchies
5. Fallacy of data

1) Optimism Bias

Optimism bias is the demonstrated systematic tendency for people to be over-optimistic about the outcome of planned actions. This includes over-estimating the likelihood of positive events and under-estimating the likelihood of negative events.

At the outset of a forecast, for example when a project plan is being put together, optimism bias can lead to unachievable timelines, underbudgeted costs and sky high sales forecasts.

This optimism bias transcends gender, age, education, and nationality—although it seems to be correlated with the absence of depression in planners.

Documented examples include:

* Second-year MBA students overestimated the number of job offers they would receive and their starting salary.
* Students overestimated the scores they would achieve on exams.
* Almost all newlyweds in a US study expected their marriage to last a lifetime, even while aware of the divorce statistics.
* Professional financial analysts consistently overestimated corporate earnings.

Optimism has the advantage of encouraging risk taking. How many start-ups would be formed if the entrepreneurs were true to them selves about their chance of success for example?

However, optimism bias can also have significant costs. For example, a project which is expected to take 6 months and $1MM to complete, will run out of money in month 7. In addition, if sales are below forecast, investors may be disappointed and the credibility of the entrepreneur is reduced. Managing resources across a portfolio of optimistic projects is a planners nightmare. When is project A really going to complete…?

How can managers and investors gain better forecasts by avoiding optimism bias?

1. Reference Class Forecasting (RCF) – takes into account the “inside view” and also the “outside view”. It considers past experiences and typically requires the planner to make their estimates more reasonable by comparison with past forecasting errors. For example, the UK Government requires project planners to apply an uplift to forecasted costs for major infrastructure projects (bridges, roads etc). This increases the projected costs by up to 60% (depending on what type of project). However, if the planner anticipates this uplift, they will likely sandbag the original forecast, and defeat the object.
2. Prediction Markets (duh!). By aggregating information from many sources, the forecast will likely contain more knowledge of the likely outcome. Most projects have been done before, and by including experience and also views from those less attached to the outcome, a more accurate and reliable forecast can be obtained. Unlike RCF, prediction markets are much harder to be gamed by the planner. A well-crafted prediction market draws from a diverse group of people, ensuring a balanced point-of-view.