This is a complete guide on how to evaluate your training sessions.
Chapter 1: Introduction To Evaluating Training
Chapter 2: Getting To Know People
Chapter 3: Your Evaluation Strategy
Chapter 4: Sample Training Evaluation Form
How To Evaluate Training
Introduction
Every year, organizations and individuals spend literally billions of dollars on training and education programs. However, experience has consistently shown that little time or expense is spent evaluating the effectiveness of the training that has taken place.
- Did the training program work?
- Was it worthwhile?
- Did the results justify the investment?
This guide will help you, as a trainer, evaluate the effectiveness of your training programs and support you in your discussions with your contracting clients, whether they are within or outside your organization.
Feedback
When we look at the training evaluation, we must first understand the role of feedback.
Feedback is the process of giving information to learners to assist them in learning more about themselves, the subject matter, and the effect their behaviour has on others.
Constructive feedback will help learners to progress toward their learning goals.
Feedback can be positive or negative; if introduced correctly, it can be extremely constructive.
- Positive feedback is the feedback you provide learners to recognize a well-done task.
- Negative feedback, on the other hand, is provided to highlight some behavior that requires action or improvement.
Guidelines for constructive feedback include:
- Ensure that the feedback is clear and focuses on the behavior, not the individual’s personality.
- Don’t give too many messages, as it can confuse the learner.
- Ensure that you create enough space for feedback or evaluation.
- Don’t be over-negative.
- Start and end the feedback on a positive note – room for improvement.
Discussion Questions
- Why is it important not to be over-negative?
- Why is it essential to create enough space for feedback?
- What other factors do you think contribute to effective constructive feedback?
Icebreakers are an important part of a trainer’s toolkit, allowing the trainer to foster communication, enhance interaction and creativity, and energize group formation.
Getting To Know People
Most evaluations of training within organizations fall within three broad categories. These are:
- Context – Evaluations within or just after the learning event.
- Input – Evaluations focused on the resources used to meet the learning event (e.g. content, methods, etc).
- Output – Evaluations of the immediate, intermediate, and ultimate outcomes of the learning event
Training professionals usually distinguish among four training evaluation levels, first published by Donald L. Kirkpatrick (1959).
Level 1 – Reaction
This level measures participants’ perspectives about the training course and is the most common way to evaluate a course, providing a measure of customer satisfaction. Advantages
- Low cost
- Easy to employ
- Provides insights into participants’ opinions
- Provides quick feedback regarding successes and failures
Disadvantages
- A quick measure
- Results may not be a reliable basis for changing a training strategy
Level 2 – Learning
This level measures how participants have understood and taken on board the course’s learning objectives. It can include performance tests before and after the course. Advantages
- This provides more detailed evidence of whether a training program is effective.
Disadvantages
- Requires more time and resources
Level 3 – Transfer To The Job
This level measures how the knowledge, skills, and values from a course are used on the job. In most organizations, this is measured 3 to 6 months after training. Advantages
- Provides more substantial evidence that a training investment has the desired impact on job behavior
Disadvantages
- Requires significant time, expertise, and resources
Level 4 Organizational Impact
This level measures all types within an organization (performance improvement, quality improvements, and cost savings)
Advantages
- Provides the most robust possible evidence that a training program has the desired impact on an organization
Disadvantages
- Requires substantial time, expertise, and resources
The trainer evaluates training by employing a wide range of information. As a trainer, you should collect this information from every aspect of the session. Tools that will help you collect this information include:
- Questionnaires
- Interviews and Group Discussion
- Learning Diaries
- Attendance and Punctuality Data
- Assessment Results
- Why Individuals Are Attending the Course
- Supervisor Reports
- Productivity
- Benchmarks
- Reactions
The trainees’ reactions to training are very important to the trainees and the trainer.
These reactions tend to vary during the training program, and the trainer needs to recognize them. The most popular approach to collecting these reactions is questionnaires with rating scales.
Interviews and informal discussions during the training program and through follow-up contact with the participants can also supplement this. Participant reaction criteria include:
- Content – quality, quantity, relevance, level
- Style – pace, format, accessibility of language, interactivity, flexibility
- Media – variety, clarity, readability, user-friendliness, stimulating
- Personnel – presentation, motivation, fit for purpose, commitment, expertise, professionalism, commitment
- Location – fit for purpose, comfort, access, refreshments, facilities
- Timing – about external and internal events
At the end of this document, we have supplied you with several reaction questionnaires you can employ and modify within your training sessions.
Learning
As the primary goal of training is to transfer knowledge, skills, and abilities, it is no wonder that this is one of the most important parts that needs to be evaluated.
The first place to start is with the learning objectives associated with the course. These also represent the first step in the design and content of a training course. Learning objectives should have three components:
- They must be terminal. To illustrate, they must specify what kind of behavior can be used as evidence that the participant has achieved the objective.
- They must define the behavior in terms of the conditions the behavior will be expected to occur.
- The accepted performance criteria that the participant must perform.
In this sense, the evaluation of learning objectives becomes a matter of condition. To illustrate, if the objective was to understand conditions necessary for change, then the evidence would test whether this understanding existed. It is important to remember that if the participant does not reach this learning, it is the training system that has failed, not the trainee.
As a result, the learning objective must be modified to enable the participant to achieve it.
Transfer To The Job
Within work environments, it is always essential that the skills and knowledge acquired are transferable to the job behavior.
In this situation, the objective of the training is to transfer or result in a change of job behavior.
Measuring this change can be difficult, especially when the training involves developing several skills for application within several settings.
This is not an impossible task. In such situations, working closely with the contracting individual (e.g., the manager) to ensure practical measurements/markers are in place is essential. Supervisor reports are used predominately to assess this level of learning.
It is essential to highlight that participants must be allowed to employ the skills and behaviors developed during the course. Without this opportunity, evaluation results become redundant.
Organizational Impact
By far the most challenging aspect of evaluating training is within the area of organizational impact. However, this is also the most important to the organization. To illustrate, most training initiatives are designed to deliver an increase in production/profitability or a reduction in some cost area. Organizational impact areas can include:
- Quality or Quantity of Production
- Absenteeism
- Turnover
- Operation Costs
- Health & Safety
- Work Attitudes
- Work Methods
It is a challenge to identify with certainty that the impact on the organization was a direct result of the training intervention. However, by working closely with the organization it is possible to attribute areas of impact and assign the ultimate value to the training.
The measure of ultimate value to the organization refers to how the organization has benefited from the training in terms of increased profitability, survival, or growth.
Return on investment measures can also greatly support any findings through the collection methods highlighted earlier in this training evaluation guide.
What Issues Should Be Evaluated?
The following table will assist you in determining what issues should be identified and assessed during any training evaluation. These items will also help you design and implement your training programs.
Purpose of Training
Description: The primary reason for developing the training. What performance deficit is to be targeted by the training? What strategic objectives are going to be supported by the training?
Linkage
The link between training and the needs met by them
- Are the training objectives linked to identified needs?
- Are the training objectives linked to identified strategic goals?
Marketing
Description: How the training is promoted.
- Are the participants identified?
- Is the training promoted in terms of their interests?
Goals/Objectives
Description: The goals or outcomes to be met by the training.
- Are the goals of the training program clearly stated?
Outcomes
Description: The desired impact of the training (Individual, On-The-Job, Organizational)
- Are the individual, on-the-job and organizational outcomes identified?
- Who is accountable for the transfer of learning?
Assessment
Description: How the training will be measured.
- How will participant reactions to training be measured?
- How will behavior be measured?
- How will the on-the-job benefits be measured?
- How will the organizational impact be measured?
Setting
Description: Where the training will take place. Where the evaluation will take place.
- What is unique about the training location?
- What is unique about the evaluation location?
Participants
Description: The individuals to whom the training is directed.
- What assumptions are made about the participants?
- What prerequisite skills are needed?
Training
Description: The subject matter.
- Is the subject matter appropriate to the learning objectives?
- Has it been prepared appropriately?
Provision For Differences
Description: Preparing for differences between participants.
- How will differences between participants be recognized?
- How will these differences be managed within the training location?
Logistics
Description: Planning for the delivery of the training.
- Are all resources required available?
Testing The Training
Description: How training will be measured after the course.
- Are any tests linked to learning objectives?
Implementation Of Training
Description: How the training is delivered.
- Are the delivery methods appropriate to learning objectives?
Application
Description: How will the training be applied to the job?
- What on-the-job application is desired?
- How will this be measured?
Effects
Description: The results of training.
- What effects are possible?
- What effects are desired?
- What effects are achieved?
Use Of Information
Description: How information on the effects of the training will be used.
- Who will need to be informed about the effects of the training?
- What decisions will this information inform?
Timelines For Evaluation
As you design your evaluation strategy, you must recognize the timelines for your evaluation. The following will help you deploy your evaluation and training program effectively.
Pre-Training
The reason for the training should be established and communicated to all the participants. This communication should include information on how the training will benefit them personally and how it will benefit the organization.
During Training
It is always important for the trainer to be aware of the participants’ reactions during the training session. This may also require the trainer to be flexible in evaluating training courses and accommodating in the style they adopt during the training session. Reaction sheets should be used, and unanswered questions should be followed up.
Post-Training
The organization’s management should be supported in providing feedback following the training session. This will ensure that the organization is not only supporting the training but also conscious of measuring its effectiveness. Individual follow-ups should also be performed to support the participants in the course.
Who Should Be Consulted During A Training Evaluation?
In most organizations, evaluations focus on the participants and their reactions to the training event. However, the following need to be consulted to evaluate all assessment levels outlined earlier in this guide.
- Sponsor (Senior Management)
- Providers (Trainers)
- Participants
- Line Managers/Supervisors
- Remember—the more people involved, the greater the accuracy of and buy-in to the evaluation’s results.
Your Evaluation Strategy
The last aspect to consider in evaluating training courses is whether it best suits your organization or the organization where the training evaluation will take place.
Value-For-Money
Top management believes training should only take place when it can prove its financial return.
- Identify direct and indirect costs of training
- Identify financial benefits to the organization
- Compare the cost and benefits to create a cost-benefit ratio
- Identify whether the use of other solutions or media will be more cost-beneficial
Investment-Value Strategy
When the focus is not just on the immediate cost benefit but on the results.
- Focus on cost-benefits
- Identify result value
Objectives-Centred Strategy
The training sponsor and trainer work together to set objectives.
- Development of a learning contract that outlines outcomes, measurement, and final evaluation Auditing Strategy Requirement for a comprehensive audit of the training program
- Link to overall training and development plan
- Systematic evaluation of all training events
Business-Led Strategy
Where the organization is going regarding workforce development and the role of training.
- Identify the direction and organizational goals of the business
- Focus on the role of training and development in the achievement of organizational goals
We have included a short sample evaluation for your consideration and use within your training settings.
Sample Evaluation Form
Name:
Job Title:
Department:
Name of Course:
Date of Course:
Instructors Name:
Did the course meet your objectives? Yes No (please circle your answer)
Please explain your answer:
Coverage of the Topic (please circle your answer)
Excellent 5 4 3 2 1 Poor
Comment
Organization of the Course Material (please circle your answer)
Excellent 5 4 3 2 1 Poor
Comment
Relevance of the Course Material Used (please circle your answer)
Excellent 5 4 3 2 1 Poor
Comment Use of Practical Examples (please circle your answer)
Excellent 5 4 3 2 1 Poor
Comment Level of Difficulty (please circle your answer)
Too Difficult 5 4 3 2 1 Too Easy
Comment
Length of Course (please circle your answer)
Too Long 5 4 3 2 1 Too Short
Comment
Facilities (please circle your answer)
Excellent 5 4 3 2 1 Poor
Comment
Suitability of Trainer (please circle your answer)
Excellent 5 4 3 2 1 Poor
Comment Quality of Handouts (please circle your answer)
Excellent 5 4 3 2 1 Poor
Comment
Quality of Overheads (please circle your answer)
Excellent 5 4 3 2 1 Poor
Comment
Quality of Exercises (please circle your answer)
Excellent 5 4 3 2 1 Poor
Comment
Will You Be Recommending This Course To Your Colleagues?
Yes/No (please circle your answer)
What Areas Of The Course Could Be Improved?
Conclusion
That’s it for our guide on evaluating training courses. Contact us to let us know if you like this guide. And please don’t forget to share this guide.
With 30+ years of training experience, Catherine Fitzgerald, B.A., M.A., PGDip, founded Oak Innovation (oakinnovation.com) in 1995. Catherine received her Bachelor’s and Master’s degrees from University College Cork. She holds qualifications in Professional Development And Training from University College Galway. Since 1995, clients include Apple, Time Warner, and Harvard University.
