Install Toolbar

Site Map

See Article Categories

The Curious Case of the CHAOS Report 2009

By Jorge Dominguez
Man with head in hands

The Standish Group collects information on project failures in the IT industry and environments with the objective of making the industry more successful and to show ways to improve its success rates and increase the value of the IT investments. The latest results have been compiled into the CHAOS Report 2009 published by the organisation in April.

Problem: it measures success by only looking at whether the projects were completed on time, on budget and with required features and functions (met user requirements). What happened to the rest of the "six triple constraints!"

The organisation leaves out of its measures the quality, the risk, and customer satisfaction. Not that we are complaining. They have the right to measure whatever they want and we have stated before that we have to consider the CHAOS Report results in a recent article on my theory on why IT projects fail. But we, PMs, already know that all these measurements work in tandem and need to keep this in mind.

The report shows that software projects now have a 32% success rate compared to 35% from the previous study in 2006 and 16% in 1994. On the other hand, 44% of projects were challenged (late, over budget and/or with less than the required features and functions) while 24% failed (cancelled prior to completion or delivered and never used).

Measure 1994 1996 1998 2000 2002 2004 2006 2009 2011
Successful 16% 27% 26% 28% 34% 29% 35% 32% 37%
Challenged 53% 33% 46% 49% 51% 53% 46% 44% 42%
Failed 31% 40% 28% 23% 15% 18% 19% 24% 21%

So, must we conclude that project success is a little worse than in 2006 (32% vs. 35%) but definitely better than in 1994 (16%)? For sure, there is better project management expertise (more certified project managers), better training, and better tools and techniques. On the other hand, project complexity and environments have increased while the time to deliver has been reduced. Look at the table above and make your own conclusions.

In our opinion, project success in IT has improved when looking at all the many angles that are not being considered by the CHAOS Report. Nevertheless, the figures are still low and need to improve much more.

Still, the CHAOS Report continues to be an important measure for the IT industry in spite leaving a lot of curious minds wandering about the methods used. Don't you think so...? Well, I do.

Jorge Dominguez wrote the article "The Curious Case of the CHAOS Report 2009" and recommends you visit www.Expiriance.com External Link for more information about project management.

Speech bubble Don't forget to leave your comments below.

Comments (3)

I recently expressed similar concerns with the way the Standish Group went about gathering and publishing data for the Chaos report. For me, it generates more questions than it answers. And the figures certainly do not reflect my experience of the world over the last 30 years I have been in IT.

http://aterny.wordpress.com/2013/07/16/can-we-believe-the-chaos-report/
#1 - Chris Davies - Monday 5th August 2013 - 14:22
Consultant, PMP
1. Why does the table in this article have figures for 2011, when the article itself is dated 7/1/2009?

2. Where does the information in this table come from? I'm wondering not only where it comes from, but the methods used to get these figures?

3. I would observe that anyone setting out to evaluate overall project success or failure over time should be clear about the criteria used, and further, that anyone who quotes such findings should also provide the criteria used.
#2 - Charles Stewart - Friday 14th February 2014 - 18:45
Chaos Manifesto 2013
I think you will find this interesting, it will answer some of your questions:

http://versionone.com/assets/img/files/ChaosManifesto2013.pdf
#3 - Duncan - Friday 14th February 2014 - 19:04
Name
Email (will not appear online)
Title
Comment

Article Categories

Most IT Projects Fail. Will Yours?
Studies on project failure make depressing reading. They suggest that 75% of all US IT projects are considered to be failures by those responsible for initiating them.

Why Software Projects Fail and How to Make Them Succeed
Let's start with a worrying statistic. According to the Standish Group only 16 percent of software projects are successful, 53 percent challenged and 31 percent cancelled.

Ten Tips for Running Successful Projects
Successful projects don't just happen. They require structured planning, insightful management and good interpersonal skills. Use these ten tips to help make your next project a winner.

10 Rules of Highly Successful Project Management
A successful project manager is one who can envision the entire project from start to finish, and have the prowess to realise this vision. These ten rules will help improve your projects.

Information Icon Meeting Challenges

The POST method is a way to give clarity at the beginning of a meeting.

  • Purpose: What is the purpose of the meeting?
  • Objective: What are you trying to achieve in the meeting, what does success look like?
  • Structure: What is the structure of the meeting we are having?
  • Timing: How much time is allocated to the meeting?

Speech bubble Discover our online forum where you can ask questions, find out tips from other people and share your experience.