Four Steps to Computing Training ROI
SHRM White Paper2/1/2001 / By Frances Lilly, SPHR, CEBS, Reviewed August 2002
In the agricultural and industrial economies, companies needed "hard" workers, but the New Economy is putting a premium on "smart" workers. The explosion of knowledge and technology and the shortage of skilled workers have spawned an abundance of books and articles on the topics of lifelong learning, knowledge capital and intellectual capital. Accountability is a key issue for Human Resources as well as other business units. Consequently, the idea of being able to calculate the return on investment (ROI) of training is enticing. An absolute number in a neat package, a trainer's dream! In some cases, it can be obtained. In other cases, while seductive, it may not be worth the effort.
The more money a company spends on employee training, the greater the concern that these highly skilled people will leave and take their knowledge somewhere else. This results in a loss of knowledge and a poor return on the organization's investment in training. However, research has shown that training actually reduces turnover and absenteeism. Employees will stay where they can grow and develop.
Measuring the ROI
Assessment of training may be conducted at five different levels, as illustrated in the matrix in Figure 1. These include (1) evaluating the reaction of participants, (2) measuring the learning that occurred, (3) assessing the on the job behavior, (4) identifying business results of training and (5) calculating the ROI.
A comprehensive evaluation of ROI only occurs at Level 5. Most organizations compute the value of training using data collected at the other four levels. These computations are based on a number of assumptions and estimates; the lower the level of evaluation, the greater the use of estimates and assumptions. For instance, at Level 1, a trainee's action plan might indicate that he/she estimates a 10% increase in productivity as a result of the training. This productivity gain could be factored for with a confidence rating and then translated into increased revenue or profit.
A thorough ROI analysis is typically conducted for only 10-20% of all training programs. The collection and analysis of data can be time consuming and expensive. Computing the ROI of training at Level 5 involves these four steps: (1) isolating the effects of training, (2) converting these effects (benefits) into monetary values, (3) calculating the costs of the training and (4) comparing the value of the effects to the incurred costs.
(1) Isolating the Effects of Training
To determine the ROI of training you must be able to measure the changes that occur as a result of training. Consequently, you must know what the performance or level of knowledge was before you began the training initiative. Pre-training data measurement might include frequency of errors, labor hours per unit of production or service, dollars of scrap materials, number of returned or defective products, volume of lost sales, absentee rates, turnover rates or survey ratings indicating customer dissatisfaction. Unfortunately, many companies do not maintain detailed records and, therefore, the pre-training data is not available. Delaying training to accumulate pre-training data may not be a wise decision.
An option to pre-training and post-training data comparisons is to use two sample groups of employees. One test group receives the training while the control group does not. Productivity or performance of the two groups is measured and compared. The challenge for HR managers is to make sure that the two groups are similar except for the training variable. This requires assessment and equal distribution of the skills in each group, as well as a careful matching for age, race, gender, educational level, years of experience, previous training and other variables.
Isolating the effects of training requires identification of all key factors that impact employee performance and business outcomes. Focus groups, questionnaires, surveys and observations will facilitate collection of this data. Possible contributors include employees, senior managers, supervisors, customers, vendors and training and human resource development specialists.
(2) Converting the Effects of Training into Monetary Values
The effects, or benefits, of a training program should always be identified, qualified and converted to dollars with input from management. Trainees' supervisors, department or division heads, senior level executives or even the board of directors are in excellent positions to observe changes in performance or impact on the bottom line. Their decisions and data may be far more objective and credible than if the HR manager makes all decisions regarding the scope, impact and duration of training benefits. Costs are known up front. Benefits may accrue slowly over time. Accurately estimating the number of times a course will be used, the number of employees that will be affected, the dollar impact of changes in quantity or quality and the extent to which training will affect results requires skill, insight and clearly defined objectives.
Effects can be tangible or intangible and are frequently referred to as "hard data" and "soft data." Hard data is quantitative, statistical, number oriented and easily translated into monetary benefits. Soft data is qualitative and refers to intangible benefits that are subjective and thus are more difficult to measure and translate into monetary benefits. Samples of both kinds of data are listed below.
Sample "Hard" Data for Determining the Effects of Training Data
Productivity measures (quantity or market value).
Quality measures (number of rejects or cost of rejects).
Materials costs (amount per unit of production or amount of waste or scrap).
Labor hours per unit of production.
Labor costs per unit of production.
Hours of "down time" due to equipment failure, etc.
Absenteeism and tardiness rates.
Turnover rate.
Workers compensation claims - nature and number of injuries or illnesses, days of lost work or "light duty" work.
Number of grievances/legal claims/lawsuits.
Time required to fill vacant positions.
Time required to fill an order; respond to a telephone call, resolve a complaint, etc.
Number of sales or dollar value of sales per customer.
Percent of market share.
Customer satisfaction rating or index.
Number of repeat customers.
Number of accounts or dollar value of accounts more than 30, 60, 90 days past due.
Sample "Soft Data" Effects or Benefits of Training
Improved job satisfaction.
Improved teamwork.
Increased organizational commitment.
Improved succession planning.
Increased communication regarding career paths.
More clearly defined promotion opportunities.
(3) Calculating the Cost of Training
HR managers generally are able to establish the costs of a training program. When calculating costs, remember to carefully consider all indirect costs, such as staff time, use of existing materials, equipment, classrooms, etc. The term "fully loaded costs" is sometimes used to designate that the costs of a program include both direct and indirect costs.
Sample Training Costs:
Course Development - needs analysis, design, writing, illustrating, validating tests and evaluation instruments.
Wages and salaries of HR staff, managers and employees involved in course design and development.
Wages and salaries of instructor and any onsite support staff.
Wages and salaries of trainees (sometimes referred to as "seat time").
Wages for temporary or contract workers hired to maintain productivity/service during regular employees' training time.
Loss of revenue while trainees are involved in training activities.
Payments to outside trainers, outside training companies, or consultants.
Instructional materials - printing, copying, or reproducing manuals, videos, computer disks, and flip charts; pens, pencils, paper, etc. and/or purchase of materials from outside vendor.
Equipment and facilities - including all types of AV equipment, computer hardware and software, classroom overhead or rental of training facilities or equipment.
Administration - marketing, scheduling, registration, testing, documentation, copying, collating, long distance phone calls, postage, (administrative staff time and costs associated with administrative duties).
Logistics - lodging, meals, tips, refreshment breaks and shipping costs.
Travel time - to and from training.
Tuition reimbursement
(4) Comparing the Value of Effects to the Incurred Costs
The ROI formula or equation uses the effects (benefits) data and the incurred costs as follows:
ROI % = Net Program Benefits x 100
Total Incurred Costs
An example would be:
If Net Program Benefits are $120,000 and Total Incurred Costs are $100,000, then:
ROI = $120,000 x 100 or ROI = 120%
$100,000
It may not always be feasible to measure ROI. It is doubtful that companies are setting up web sites and developing e-commerce initiatives after a careful and exhaustive analysis of their ROI. These organizations realize that rapid changes are occurring and they will lose future business if they fail to adapt business methods to new technologies.
Similarly, if HR becomes focused on getting every cost down to a line item and then comparing the costs with the benefits, every line item could be debated. Are you able to clearly show that it was the training that made the difference in a customer's satisfaction? Are you sure you will provide training for (x) number of employees before this training program becomes obsolete? Are you sure that the training is responsible for the improvement in employee performance rather than a new manager or a change in the compensation plan? When did you decide to measure training results, in 3 months or 6 months? What time frame is logical or reasonable for your industry? What would happen to your ROI calculations if turnover increased or decreased? Sometimes you may need to focus on the big picture, the changes that are occurring, rather than a line-by-line item of cost benefit analysis or ROI.
New employee orientation and diversity training are examples of types of training where it may be difficult to measure ROI. New employees have varied backgrounds. For some employees, this is their first job and they might find it difficult to absorb all of the information given to them. Other employees have years of work experience and attended several new employee orientation programs. They already have a base of knowledge and are looking for the kind of information that is specific to your company. Diversity training may also be an initial experience or a review. It may be very difficult to hold all variables constant when you are trying to look at the effect of training. However, few would argue that new employee orientation or diversity training should be eliminated simply because you cannot easily demonstrate an ROI. Orientation programs set the tone for the entire employer/employee relationship. Turnover rates or productivity measurements of new employees using a test group and a control group could provide data for estimating the value of orientation programs if implemented throughout the entire organization.
Difficulties in Calculating ROI
One of the common difficulties encountered in calculating ROI is determining the "shelf-life" of the training program. How many times will a course be used or "cycled" before business strategies or budgets dictate its demise, technology renders it obsolete or all eligible employees have received the training? Changes fueled by technology, global competition, legal and regulatory agencies, employee turnover and other factors make it increasingly difficult to accurately predict a training program's shelf-life. Use a conservative approach because of the rapid changes occurring in the workplace.
It is difficult enough to get managers to send employees to training without imposing additional requirements to collect pre- or post-training data to document the effects of training. HR managers should make data collection as simple and effortless as possible and provide forms and administrative assistance whenever possible. Collaboratively work with IT, HRIS, Production, Engineering, Sales, Payroll and Accounting personnel to "mine" existing data or develop ways to collect, compile and extract the necessary data for a cost/benefit analysis of training.
The presence of unions could also create data collection problems. Frequently, unions are concerned that the employees or union members are being evaluated. It is a good idea to get union leaders to "buy in" to the idea of an ROI assessment. At the very least, a careful review of the union contract is recommended along with clear communication regarding the goals and scope of the training assessment.
Some behaviors or skills may be useful for only short periods of time while others, such as time management, may continue for a lifetime and affect numerous workplace and personal activities. Frequency of use of the newly acquired skills is also a factor in determining how long the skills are retained. Therefore, determining or estimating the scope and duration of the benefits of training can be a daunting task.
Measuring the impact of training for new products, technology and services is difficult for two reasons. First, if no prior training existed or no prior performance measures are available, it is very difficult to ascertain the impact of training. Second, if prior measures of performance are available, how does one determine the amount of improvement that is attributable to training as opposed to how much is attributable to the new innovation? Frequently, this determination is a judgement call or an intuitive decision rather than one based on factual data.
Suggestions for Improving the ROI of Training
HR managers can improve the actual ROI of training by utilizing the following tips:
Understand that business needs will change over the duration of the project. The longer it takes to do a project, the more likely that change will occur. Be prepared to switch gears or re-tool. Stay alert for indications of learning gaps or obsolescence of course materials created by changes in technology, economic factors or employee demographics.
Use the experience of senior managers and employees in estimating the results of training. Estimating requires a little math and lots of experience.
Anticipate and plan for glitches. They will happen. Do not get discouraged. Deal realistically with unexpected events and surprises. Some of them will be problems that may reduce expected outcomes. Others may stimulate creativity and produce very positive results.
Keep an eye on costs and monitor any deviations from estimated or budgeted expenditures. Quick action or adjustments may enable the HR manager to curtail or reduce costs and keep the project within or near budget.
Balance short-term and long-term goals. The pressure to deliver consistent short-term financial performance may limit the resources available for investment in growth opportunities, such as the "development" of employees for future roles.
Secure managers' and supervisors' support for training. Effective training depends on three persons: the trainer, the trainee and the supervisor or manager. All three must agree on expected outcomes and how and when they will be measured.
Align training initiatives with strategic business plans. Training should improve your organization's competitive edge, which may be efficiency, first class service or creativity and innovation.
Track training costs by employee for career pathing and succession planning
HR must shift its emphasis from being the "keeper and distributor of people information" to the more critical role of "developer of people and productivity." Training benefits must be greater than the cost of the training. Training must ultimately improve the profitability of an organization. The four-step process for calculating ROI enables HR mangers to determine the productivity affect of training or its impact on the bottom line. Reducing training to save money may be like stopping a clock to save time.
For further reading:
Cascio, Wayne F. Costing Human Resources: The Financial Impact of Behavior in Organizations. Cincinnati, OH: South-Western Publishing, 1999.
Fitz-enz, Jac. ROI of Human Capital. New York: AMACOM, American Management Association, 2000.
Kirkpatrick, Donald. Evaluating Programs: The Four Levels. San Francisco: Berrett-Koehler, 1998.
Phillips, Jack J. Return on Investment in Training and Performance Improvement, Houston: Gulf Publishing, 1997.
Figure 1
Levels of Evaluation for Determining ROI
Level / Objective of Measurement / Tool or Technique / Comments
1. Reaction (and planned action) / Participant's reaction to and satisfaction with the content and delivery of training. / Participant's complete evaluation forms and/or develop action plans for implementing new knowledge. / Subjective but has some usefulness.
If follow-up is scheduled, participant's action plans will be more realistic.
2. Learning / Skills, knowledge or attitude changes as a result of training program. / Tests via paper and pencil or computerized format. / Tests must be assessed for validity and reliability.
3. Behavior / Changes in behavior on the job as a result of training. / Performance reviews and observations. / Assumption is that if the skills are applied, results will follow.
4. Results / Impact of training on business activities and processes. / Cost reduction, productivity increases, improved quality, reduced labor hours, decreased production/processing time, etc. / Critical tasks are isolating the effects of training and capturing appropriate data.
5. Return on Investment (ROI) / Compares the costs of the training program with monetary results and is usually expressed as a percentage. / Detailed, comprehensive data collection and analysis of costs & benefits. Accounting expertise helpful. Time value of money is a factor. / The most comprehensive and objective evaluation technique, but the process can be very costly and time consuming.
Thanks to Frances Lilly, SPHR, CEBS of the Human Resource Development Committee for contributing this article. It is intended as information only and is not a substitute for legal or professional advice.
For more information on this subject, contact the SHRM Knowledge Center.