Estimating: Part 3
By William R. Duncan | minute read
This is the final article of three on estimating. In the first, I focused on definitions since many people use terms such as estimate and budget as synonyms when, in fact, they are very different. Here is a brief recap of some key definitions:
Estimate: An informed assessment of an uncertain event. Informed means that you have an identified basis for the estimate. Uncertain recognises that multiple outcomes are possible.
Effort: Effort is an expenditure of physical or mental effort on the part of a project team member. Effort is normally measured in terms of staff hours.
Budget: A management metric that is derived from the estimate of the relevant work.
Baseline: A time-phased budget that has received all necessary approvals.
In the second article, I introduced the idea of three-point range estimates and covered the mechanics of how prepare them with your team. I ended that article by asserting that three-point estimates usually take less time to prepare than traditional single-point estimates.
In this last article of the series, I'll cover a potpourri of other estimating topics including the key outcome of estimating, converting effort estimates into budgets, dealing with poorly defined work, and what to do when management thinks it should cost less or take less time.
Key Outcome of Estimating
When I teach people how to estimate, I emphasise that the most important outcome of the estimating process is a project budget that is accurate, i.e., one where the actual results fall within an acceptable range of the budget baseline. Acceptable budget accuracy will vary based upon the project (and based upon the organisation's attitude toward contingencies and reserves, but that's a subject for another day!):
- In New Product Development (NPD), it is often acceptable for the final cost of the project to be 3-4 times the initial estimates.
- In Information Technology (IT), ±10% is a reasonable target if the requirements are agreed and stable, while the NPD range of +300-400% may also be perfectly acceptable if the project is mission-critical.
- For a fixed price consulting engagement, and again ignoring contingencies and reserves, management may be reluctant to accept any overrun. I know, I worked in this environment for many years!
I argue that an estimate is accurate if (a) the actual results for that work item fall within the estimate's range most of the time, and (b) the sum of the actual results is close to the sum of the expected values of the range estimate. In brief, we care about the total. We care about the individual results only as a means to an end.
Let me illustrate. Let's say that we have a project with 100 activities. To keep it simple, let's ignore scheduling concerns for the time being, and let's further assume that each work item has a three-point range estimate of 20, 25, and 30 hours of effort. The budget for each individual activity will be 25 hours (more on that later), and thus the budget for the project will be 2,500 hours.
If this project goes according to plan, most of the activities, probably around 70% of them, will be completed for an actual cost within an hour or two of their budgeted effort. One or two may take as many as 30 hours; one or two may take only 20. But the project total is likely to be very close to 2,500. Statistically, we have about a 95% probability that the project will finish within a range of ±50 hours. Not bad!
For comparison, let's take a similar project with 100 activities, only for this project, each activity has a range estimate of 10, 25, and 40 hours of effort. We will still budget this project for 2,500 hours, but ±50 hours has only about a 60% chance. If we want 95% accuracy, we need to go to ±120 hours.
Three-point range estimates help to support our goal of accuracy by encouraging everyone on the team to spend as much time and effort as they need to finish the work item correctly, and only the time and effort that they need. We avoid a lot of dysfunctional behaviour:
- Team members may use the full budget even if they don't need it.
- Team members may cut corners to avoid exceeding the budget.
- Managers at all levels of the project may waste time explaining minor variances.
- Managers may be tempted to manipulate the reported actuals in order to make the estimates appear "better."
And one final point: there are some circumstances when a single-point estimate is okay. For example, we can safely predict that a one hour team meeting that includes seven people will require seven hours of effort. Is there some chance that only six people will show up? That it will take an hour-and-a-half? Sure, but this is likely to be such a small source of error that we can safely ignore it.
Converting Estimates into Budgets
While an estimate is a range, a budget is a single number. We can use some simple statistical techniques to convert our ranges into budget numbers. For this article, I'm going to ignore other budget preparation issues like unstable requirements and pre-defined budget maximums since these are more properly addressed through project risk management than through estimating.
The formulas for adding up ranges can be found in any basic statistics text. If you are willing to accept my suggestion that a triangular distribution is a good assumption (see the second article in this series), the basic process is straightforward:
- Develop range estimates (low, likely, high) for each detail item in your project plan.
- Calculate the expected value for each item by summing the three numbers and dividing by three.
- Total the expected values. This is your base budget (i.e., it does not include any reserves or contingencies).
And yes, it is just that simple. Here's a worked example for a small project with two deliverables and seven activities that produces a base budget of 290 hours:
| Work Item | Low | Likely | High | Exp. Val. |
|---|---|---|---|---|
| Training Materials | ||||
| Outline | 10 | 20 | 30 | 20 |
| Review draft | 30 | 40 | 80 | 50 |
| Approved doct | 15 | 20 | 30 | 22 |
| Software | ||||
| Module 1 | 25 | 30 | 55 | 37 |
| Module 2 | 25 | 30 | 45 | 33 |
| Module 3 | 20 | 40 | 80 | 47 |
| Module 4 | 65 | 70 | 110 | 82 |
| 290 |
Many of you are probably also familiar with the formula for the expected value that is used in the Program Evaluation and Review Technique (PERT). The PERT formula is different because PERT assumes a beta distribution. I prefer to assume a triangular distribution because it produces more conservative results (a higher budget base) than a beta. Either assumption will work, using range estimates will significantly improve the accuracy of your estimates.
As an added bonus, most of today's popular project management software packages have the capacity to deal with ranges in some way. If this feature is not built in, it is usually available via an add-on.
Dealing with Poorly Defined Work
In order to introduce the basic concepts of effort estimating, we limited our discussion to developing an estimate for a single, reasonably well-defined activity. Although this type of work may prevail in some project environments, most projects will have a significant amount of work to estimate that is not well-defined. In this section, I will cover some approaches to those situations.
No one on the team has experience with this kind of work. This is a two-edged sword. Even if you can find an experienced resource to develop the estimate (e.g., a consultant or an article on the web), that individual may still have trouble estimating how long it will take an inexperienced person to do the work. Here are some other ideas that should help:
- Use a wider range estimate to reflect the greater uncertainty. Unless the unknown work represents a significant percentage of the total work, your overall project budget will not be greatly affected. Remember that we are willing to tolerate budget variances in individual work items as long as the project total is acceptable.
- Verify your completion criteria. If the completion criteria are not clear, try redefining them until they are.
- Break the work down into smaller units. Often, you will find that you do have experience with much of the work. The aspects that you don't have experience with can be handled with a wider range estimate.
- Develop some experience. It may be possible to have one or two team members do a couple of "practice" activities as part of the project planning process. I used this approach quite successfully as part of a database conversion project several years ago.
This is truly state-of-the-art stuff. See previous paragraph, especially the last bullet.
We really have no idea of what's required or we don't know enough at this point to define the work. Neither of these two complaints is an estimating issue; they are both either a scoping issue or a risk management issue. They might also be a defensive response to a skeptical manager. If so, see below.
Conflicting Estimates
Did you ever have two team members who differed widely in their opinions of the likely amount of effort required to complete a work item? Range estimates can help, particularly if the individuals involved are inherently optimistic or pessimistic - if their ranges overlap, it is usually much easier to reach agreement.
But the root cause of such difference is most likely to be the presence of conflicting assumptions. For example, a team member who assumes that previous design work can be reused will produce a lower estimate than one who does not. By surfacing the conflicting assumptions, the team can discuss the alternatives and decide what to do.
One particularly pervasive assumption is the skill level of the person doing the work. Estimating is often done by more senior staff, and they tend to think in terms of "how much effort would I personally have to expend to complete this work." This is likely to produce an estimate that it too low. At the same time, they may become aware of their tendency to overestimate and overcompensate. I recommend that you remind your estimators to constantly assume an "average" resource, and to recognise that it will take 2-3 projects of comparing actuals to estimates for them to get good at knowing what "average" really is.
Converting Effort into Duration
I know that I am going to get a lot of flak from some of my colleagues about this next point. The technically correct answer about how to estimate duration is that you should do just that - look at the effort estimates and develop a three-point range estimate of the duration for each work item. For example, with an effort estimate of 30, 40, and 80 hours to be performed by a single individual working full-time on the project, I think it is reasonable to predict that this person may need more than two full weeks to get this item done if it ends up requiring 80 hours of effort.
In addition, since virtually all network analysis techniques (Critical Path, Critical Chain, PERT, etc.) underestimate the most likely duration of the project, many would argue that a Monte Carlo simulation is also a necessity. And I would agree…if you've got the budget.
If not, take the easy way out - take the effort budget (the expected value of the effort estimate) and convert that into a single-point duration budget based on the availability of the individual assigned.
Maintaining Your Estimates
If the work of the project changes via an approved change order, you will need to estimate the new work and update your budget as well.
If any of the assumptions that went into your estimates are changed, you should also update your estimates. In this case, you may not be able to modify the budget, but you should still re-estimate the work. For example, if one of your most skilled staffers was supposed to work on Activity D, and due to circumstance beyond your control D is being done by an intern from the local community college, you better reset everyone's expectations.
Management Cuts It in Half
The first thing you have to do is find out why. Most such management actions are created by one of the following misunderstandings:
- Your managers may not understand the difference between an estimate and a budget. Educate them. But in the nicest way possible if you want to keep your job.
- Your manager may view your estimate as a negotiating position. Some managers assume that you have "padded" your estimate to provide negotiating room. Again, educate them, explain how the suggested cuts will either reduce quality or increase the risk of an overrun.
- Your manager's cuts may be a negotiating position. Most good managers understand that a more difficult task (where the target is aggressive) can serve to motivate the project team. But even good managers often fail to appreciate that an impossible task (where the target is so aggressive that it is clearly impossible) is a powerful demotivator.
Summary
Underestimating will drive you crazy. You will spend endless hours explaining why you are over budget and behind schedule. You will spend endless hours dealing with unhappy customers and stressed out team members. Overestimating may not be much better. Your projects will not get approved because they will be seen as too expensive. You may even lose your job if your padding is viewed as unethical.
Three-point estimates will not get you into heaven. But they will make this life a lot more rewarding for everyone affected by your projects.
Effort Estimating: A Primer | Estimating: Part Two
William R. Duncan is a principal of Project Management Partners, a project management consulting and training firm headquartered in Lexington, MA USA. He is the former Director of Standards for the Project Management Institute, Inc. (USA) and is currently Director of Certification for the American Society for Advancement of Project Management (ASAPM).
© Copyright William R. Duncan.