~ By Gina Abudi
Return on Investment (ROI) is a monetary measurement that is used to evaluate the efficiency and effectiveness of an investment made by an organisation. Investments take many forms, financial, human capital, equipment, and training programmes, to name just a few. This article will focus on the use of ROI and the Phillips ROI Methodology™ to measure the effectiveness of a project management training programme completed within XYZ Law Firm. The 5 levels of evaluation which will be reviewed within the case study are:
The challenges surrounding training and its effectiveness within the organisation have become more complex over the years. Today, the challenge is even more significant for learning and development professionals. Return on Investment (ROI) as a tool for evaluating project management training is becoming an expectation of senior executives within organisations. In today's tight economy with reduced resources and tighter budgets, learning and development professionals are finding it increasingly necessary to show the monetary value of the project management programmes they are bringing to the organisation.
Today, the success of project management training programmes is measured by the financial contribution of the programme to the organisation. It is not surprising then that ROI measurements for project management training programmes are often requested. With such a large focus on project management in all kinds of organisations, there is an increasing desire to show the monetary benefit of investing in project management training programmes which often reach a wide variety of employees and represent a significant investment for organisations.
The case study example below will follow the Phillips ROI Methodology™ and encompass the four major phases of the methodology:
XYZ Law Firm was interested in developing a project management mindset within the firm. There was support from the managing partners and funding was expected to be approved. The goal of the training was to ensure that completing projects at the law firm was not a severe drain on resources. There was much anxiety behind any project which needed to be completed and the champions of the training programme, which included a few of the partners, attorneys and the paralegals, were confident that with some basic project management training, this anxiety could be controlled and projects done within the firm would be more easily managed. The whole concept of "project management" scared individuals.
The managing partners of the law firm specifically wanted to be updated on project status and they often felt they were unsure what projects were being worked on and how they were progressing. A significant goal was to ensure implementation of best practices and standard processes and increase knowledge sharing.
Overall the goals of the training programme included:
Some of the projects that the law firm worked on included:
Many such projects took longer than originally planned for, went out of scope and went over budget and resource commitment. This was certainly a problem for the client projects the law firm worked on as such projects were under fixed price agreements with the client. If the project went off track, it would impact the profitability for the law firm for that particular engagement.
Based on the goals of the law firm, a three day customised basics of project management training programme was developed. Prior to the start of the programme, data was collected on past project initiatives, specifically around:
Much of this data was tracked, although it was not in one central location and required conversations with many members of the law firm, from managing partners through to paralegals and administrative assistants.
In the first phase of the project, we developed a plan around the data we would be collecting. The plan included the following components for Level 1-4:
We also developed an ROI Analysis Plan to include the items we intended to measure against to show improvement based on the 3-day training programme. The data items to be collected at level 4, which are the impact measures targeted for improvement included:
We set the ROI (hurdle) rate at 20%.
A pilot group of 20 junior attorneys, paralegals and administrative assistants were selected to go through the programme first and measure the benefits of the training. In this way, an isolation factor, an important component of doing an ROI study, was available to compare against a similar group of attorneys, paralegals and administrative assistants who would work on projects also, but would not initially attend the 3 day programme.
The following data was collected from the programme:
In converting the data collected to a monetary value, we found that cost savings (programme benefits) were as follows:
Programme benefits are measured in level 4: Business Impact. To convert to a monetary value, a dollar amount was put to every resource based on salary/time/fringe, etc. If projects did not go out of scope, no additional resources were needed, which was a cost savings. More efficient use of resources also meant that projects had fewer people assigned to them. The $450,000 in increase from better tracking of client projects was the amount that would have been lost from the bottom line had those projects that were measured not been completed successfully.
The total cost of the programme was $320,000. This included planning for the ROI study, food and beverage for participants in the programme, facilitator costs, development of the customised programme, time out of the office for training, salary/fringe of participants, etc. Any cost associated with the programme or planning for the programme was included in the total costs. The biggest component of the cost was salary/fringe of participants, especially since the attorneys and paralegals are billable resources.
Intangible benefits, those benefits that did not have a dollar value associated with them, were also considered. They included:
These intangible benefits were able to be tied back to the programme and certainly contributed to the success of the projects.
During phase 3, we calculate the ROI of the project management training programme, using the ROI formula:
ROI = Net Programme Benefits / Programme Costs x 100
For every $1.00 invested, $2.08 is returned after the costs are recovered. This was significantly over the 20% ROI set at the beginning of the project during the planning of the study.
A report of the results of the study was presented to the managing partners of the law firm. Components of the study were presented to the participants and others in the law firm. The project was considered a success and it was decided that the 3 day customised project management training programme would be rolled out firm-wide. Additionally, a half day programme was created for the managing partners of the firm to provide them training around how to support project initiatives. A future focus was to develop a portal to house all project information. That was a "to do" for the IT/Application Development department and would be a key strategic project for them.
For training and learning departments, the PMO, or lines of business; showing the ROI of the project management training programmes conducted within the organisation, whether in-house developed programmes or vendor-provided programmes, enables you to show the value the programmes have on the business in monetary terms that executives can understand, respect and champion.
The six types of data collected during the Phillips ROI Methodology™ process (see list below) allow consistency in the measurement of project management training programmes, thereby bringing credibility to the process:
To begin measuring your project management training programmes, you might start with a brand new programme or review the current programmes that are in place, following the criteria for measuring project management training programmes discussed earlier in this article. If you review a current programme, gather data on the initial purpose of the programme when it was first developed and implemented. Does it still meet that purpose? What value do you perceive the individuals taking the project management training programme are achieving? How much has been invested in the programme since the beginning? Should this programme remain in place "as is," or should it be updated, or possibly be eliminated from your offerings in its entirety or in part to accommodate new needs among your project managers? An ROI study will enable you to look at the value of your current programmes and enable you to better plan new programmes.
Gina Abudi has over 15 years consulting experience in a variety of areas, including project management, leadership, and development of strategic learning and training programmes. She is Partner/VP Strategic Solutions at Peak Performance Group, Inc. in Gloucester, Massachusetts. Gina received her MBA from Simmons Graduate School of Management. She blogs at: http://www.ginaabudi.com/