The Curious Case of the CHAOS Report 2009

IT Project Management | By Jorge Dominguez | Read time minutes

Black and white outline of a human head with a brain of curly lines

The Standish Group collects information on project failures in the IT industry and environments with the objective of making the industry more successful and to show ways to improve its success rates and increase the value of the IT investments.

Problem: it measures success by only looking at whether the projects were completed on time, on budget and with required features and functions (met user requirements). What happened to the rest of the "six triple constraints!"

The organisation leaves out of its measures the quality, the risk, and customer satisfaction. Not that we are complaining. They have the right to measure whatever they want, and we have stated before that we have to consider the CHAOS Report results in an article on my theory on why IT projects fail. But we, PMs, already know that all these measurements work in tandem and need to keep this in mind.

The report shows that software projects now have a 32% success rate compared to 35% from the previous study in 2006 and 16% in 1994. On the other hand, 44% of projects were challenged (late, over budget and/or with less than the required features and functions), while 24% failed (cancelled prior to completion or delivered and never used).

Measure19941996199820002002200420062009
Successful16%27%26%28%34%29%35%32%
Challenged53%33%46%49%51%53%46%44%
Failed31%40%28%23%15%18%19%24%

So, must we conclude that project success is a little worse than in 2006 (32% vs 35%) but definitely better than in 1994 (16%)? For sure, there is better project management expertise (more certified project managers), better training, and better tools and techniques. On the other hand, project complexity and environments have increased while the time to deliver has been reduced. Look at the table above and make your own conclusions.

In our opinion, project success in IT has improved when looking at all the many angles that the CHAOS Report is not considering. Nevertheless, the figures are still low and need to improve much more.

Still, the CHAOS Report continues to be an important measure for the IT industry despite leaving a lot of curious minds wandering about the methods used. Don't you think so? Well, I do.


Jorge Dominguez wrote the article "The Curious Case of the CHAOS Report 2009" and recommends you visit www.Expiriance.com for more information about project management.


[Update] Since Jorge wrote his article, we can see from the later results that work is still needed to achieve successful outcomes from software development projects. This table summarises the outcomes of projects between 2011 and 2015 using the definition of success factors on time, on budget with a satisfactory result.

Measure20112012201320142015
Successful29%27%31%28%29%
Challenged49%56%50%55%52%
Failed22%17%19%17%19%

A bright note from the research is that with the take up of agile development methods in recent years, agile approaches resulted in more successful projects and fewer outright failures across all project sizes.

The Chaos Report remains an essential and critical indicator of the state of the software development industry. The latest report from the Standish Group is CHAOS 2020: Beyond Infinity, written by Jim Johnson.


Recommended read: Project Success Quick Reference Card by Henny Portman. Based on CHAOS 2020: Beyond Infinity Overview. January 2021

What's Next?

You may also be interested in