~ By Rodney J Smith
High profile IT project failures are nothing new; in fact few projects are 100 percent successful in delivering everything the customer wants on time and within budget. So it is small wonder that businesses put loads of effort into trying to minimise the risk of their projects becoming one of these statistics.
The traditional way of doing this is to spend considerable time and effort gathering together all the requirements for a new system, then designing a complete solution up front, from which detailed programming estimates can be produced. The theory behind this so-called Waterfall methodology is that the estimates (and thus budget) will be accurate, and there will be no major surprises once development gets underway. Unfortunately this approach is based on a number of faulty assumptions, which means it rarely works in practice!
In fact, building a computer system is akin to driving down a country lane at night; the driver can only ever see as far as her headlights allow, but each bit of progress that is made illuminates areas that could not be seen at the start. This is true even in domains that are relatively well understood.
There is an inevitable lag between initiation of a software project and its eventual delivery, and even if it were possible to identify a complete and accurate set of requirements up front (it's not - see above), the constantly changing business landscape dictates that requirements will change along the way. Waterfall projects typically resist absorbing such changes due to the knock-on effect on the design and schedule, with the result that the eventual delivered system does not match the (current) requirements.
Surprisingly, a number of studies indicate that on average, fixed scope projects cost almost twice as much as estimated. Apart from the problems of changing requirements, this may be in part due to initial estimates being artificially low in order to secure contracts in the first place.
This is where software development suffers because of attempts to liken it to the (building) construction industry. In fact, there is a lot of art in the craft of programming, which makes it a lot less predictable than managers would like.
As much as corporate managers would dearly love the Waterfall approach to work, in practice it is extremely unreliable, inefficient and inflexible. Yet in spite of credible alternatives (the Agile family of methods) which appear to deliver far better results in practice, managers frequently persist with this discredited methodology because of the illusion it projects of being less risky and allowing them to remain in control.