15 Questions to Help You Improve Your Existing Training Programs

By Seth N. Leibler, President & CEO, and Ann W. Parkman, Executive Vice President

Companies often continue training programs well beyond the point of usefulness or out of habit. In many cases, the original business need has changed or has been met, but the training goes on as originally designed. This is not only a very expensive practice, it tends to erode the training department's reputation for providing good return on investment.

There are many questions you can ask to evaluate existing training courses to determine if they are still needed and to ensure that they are accomplishing their intended business objectives.Below are 15 of the most important ones that will enable you to not only evaluate current courses quickly but, assuming a training need still exists, to identify ways to improve their effectiveness and efficiency.

Training Needs
Because training is such a costly intervention, you first need to make sure that it is absolutely needed. Training can only solve (or prevent) performance problems that are due to a lack of skill. It will have no impact on problems caused by such factors as motivational or environmental obstacles. Consequently, you need to be certain that there is some evidence that a job-relevant skill deficiency exists for each training course you are evaluating. Ask yourself the following:

1. Is there a discrepancy between what people are doing and what they should be doing?
2. Is the cause due to a lack of motivation or other obstacle (such as a lack of time, equipment, supplies, etc.) in their environment? If not, then ...
3. .Is the cause due to a lack of skill?

Objectives
Research shows that learning improves when there are objectives that specifically state what the learners will be able to do as a result of training. After all, it's only fair to let learners know exactly what is expected of them, how their performance will be evaluated, and what constitutes successful learning.But effective and efficient training requires more than broad-based objectives like "Learn how to fix a malfunctioning centrifugal pump." To be useful, objectives must be based on how well employees must be able to perform their jobs, and the performance specified in the objectives must match the performance expected on the job. Using the above example, a performance-based objective would read more like "Given tools, references, and a malfunctioning centrifugal pump, be able to clear the malfunction within fifteen minutes."

4. Does the course have objectives? If so, then ...
5. Are the objectives stated in performance terms?
6. Were the objectives derived from the job you need someone to be able to do?
7. Is each learner given the objectives at the beginning of the course?

Practice
We all know that people "learn by doing." But in many courses, the focus is on what the instructor is doing rather than on what the learner is doing. If practice is included, it's often placed at the end of a lesson to "use up" any extra time that may be available. However, to ensure that learners master the skills they are being taught, training should devote at least 50% of the time to practice and feedback, even if this means sacrificing some content to do it. If you need to trim down your content, target any instruction that is not going to be used in a practice session -- chances are you can reserve this content for a handout that can be used for later reference.

8. Does each learner have an opportunity to practice each skill being taught?
9. Is immediate, constructive feedback available after practice?
10. Can each learner practice until he or she has accomplished each performance objective?
11. Is at least half of the course time devoted to practice and feedback?

Assessing Performance
The thought of tests is usually punishing to most learners. But in a training environment, tests -- or assessments of performance -- ought to be regarded by learners as a positive experience, a chance to show competence and be recognized for it. This "seal of approval" can go a long way toward making learners feel confident in their new skills. That confidence, in turn, will increase the likelihood that they will put these new skills to work back on the job.

To be effective, the performance to be assessed must be the same as that described in the performance objectives -- there should be no "surprises" for the learner. Ideally, learners should receive immediate feedback on their test performance. If performance doesn't meet the criteria in the objectives, then the instructor should diagnose the weakness and provide appropriate and constructive assistance. If performance does meet the criteria, then learners should receive positive recognition or reward.

12. Is each learner required to demonstrate competence in each skill?
13. When test performance is judged to be less than competent, is assistance provided without embarrassing the learner?

Instructor Behavior
No matter how well designed a course is, inappropriate instructor behavior can diminish the learning that occurs. Good instructors will encourage learners to ask questions or comment about the subject matter being learned. They make learners feel that their questions are valued, even if the questions need to be responded to more completely at another time in order to avoid disrupting the class. And they refrain from disparaging comments made at a learner's expense (such as "Congratulations, you finally got it!") that could cause the class to want to avoid further contact with the instructor. Good instructors also show their enthusiasm for the training they are conducting. Their enthusiasm for the subject becomes infectious, leading learners to feel more positive about the training and to expend the effort required to master the skills being taught.

14. Does the instructor behave positively toward learners rather than belittle or insult them?
15. Does the instructor show enthusiasm toward the subject?

The greatest single investment in training is the cost of lost productivity while employees attend training. Therefore, instruction that provides job-critical skills in the shortest period of time will help you ensure that training returns full value to your organization.

Is Your Organization a Candidate for Training ROI?
By Patricia Pulliam Phillips and Jack J. Phillips

Rarely does a topic stir up emotions to the degree that training return on investment (ROI) does. Some individuals characterize ROI as inappropriate for training and performance improvement. Others passionately characterize ROI as the answer to their accountability concerns.

It is our contention that ROI can serve as an opportunity, a challenge, and a tool to improve training and performance improvement programs and solutions. However, ROI is not for everyone and every organization. Some organizations lack the trust and supportive climate that ROI requires. The successful champion of ROI must be willing to learn, change, and try new things; without this attitude and approach, it may be best not to try.

There are a number of revealing symptoms that will indicate if your organization is ready to implement ROI for training and performance improvement programs.

1. Pressure from senior management to measure results. This pressure can be a direct requirement to measure program effectiveness or a subtle expression of concern about the accountability of training and performance improvement programs.
   
   
2. Extremely low current investments in measurement and evaluation. Most organizations spend about 1 percent of their direct training and performance improvement budget on measurement and evaluation processes. Investments significantly lower than this amount may indicate there is little, if any, measurement and evaluation taking place, thereby signaling the need for greater accountability. Expenditures in the 4 to 5 percent range indicate that training and performance improvement are already undergoing serious evaluation.
   
   
3. Recent disasters with training and performance improvement programs. Every organization has had one or more situations where a major program was implemented with no success. When there are multiple program failures, the training and performance improvement function often bears direct responsibility - or at least blame. These failures may prompt the implementation of measurement and evaluation processes to determine the impact of training and performance improvement programs, or more appropriately, to forecast ROI prior to implementation.
   
   
4. A new director or leader in the training and performance improvement function. A new leader often serves as a catalyst to change and may initiate a review process of the success rates of previous programs. These individuals do not have the stigma of ownership or attachment to old programs and are willing to take an objective view. However, the desire to gain an immediate gauge of program effectiveness may lead to impatience if an evaluation process is not already in place.
   
   
5. Managers' desire to build cutting-edge training and performance improvement functions. Some managers strive to build cutting-edge training and performance improvement functions. In doing so, they may automatically build comprehensive measurement and evaluation processes into the overall strategy. These managers often set the pace for measurement and evaluation by highlighting the fact that they are serious about bringing accountability to their function. These functions have formal guidelines around their measurement process and build evaluation into the program development. They often begin with a thorough needs assessment to determine the best solution, then monitor the progress of the program and determine the business impact.
   
   
6. Lack of management support for the training and performance improvement effort. In some cases, the image of the training and performance improvement function suffers to the point that management no longer supports its efforts. While the unsatisfactory image may be caused by a number of factors, increased accountability often focuses on improving systems and processes, thereby shoring up the department's image.

The Typical Organization
While training ROI is suitable for any organization, the organizations currently implementing ROI as part of their training and performance improvement evaluation process share some similar characteristics. These include:

• Size of the organization: Typical organizations currently implementing ROI are large. Yet small organizations, which have an even greater reason to conserve resources and ensure they're getting the most out of their dollar, can implement ROI with credible results.
  
  
• Size and visibility of the training and performance improvement budget: The budget is usually large and has the attention of the senior management team. Regardless of how it is measured - whether as total budget, expenditure per employee, percentage of payroll, or percentage of revenue - a large budget demands appropriate measurement and evaluation.
  
  
• Focus on measurement: Typically, organizations implementing ROI are focused on establishing a variety of measures throughout the organization. Organizations already using well-known processes such as the Balanced Scorecard, Economic Value Added (EVA), Six Sigma, and others are ideal candidates for ROI because they already have a measurement-focused environment.
  
  
• Level of change taking place: Organizations using ROI are usually undergoing significant change, which in turn often increases interest in bottomline issues, thereby resulting in a need for greater accountability.

To see if your organization is a candidate for training ROI, take the following quiz.

IS YOUR ORGANIZATION A CANDIDATE FOR ROI IMPLEMENTATION?
Read each question and check off the most appropriate level of agreement on a scale of 1 to 5
(1 = Total Disagreement; 5 = Total Agreement).

SCORING
If you scored:
15-30 You are not yet a candidate for ROI.
31-45 You are not a strong candidate for ROI. However, it is time to start pursuing some type of measurement process.
46-60 You are a candidate for building skills to implement the ROI methodology. At this point there is no real pressure to show the ROI, which is the best time to perfect the process within the organization.
61-75 You should already be implementing a comprehensive measurement and evaluation process including ROI.

How to Reduce the Risk of Hiring
By Dan Raymond, Performance Consultant, &
Karen VanKampen, Director of Performance Consulting

WANTED: The best qualified candidate for the job. All others need not apply.

Wouldn't it be great if hiring were this easy? Unfortunately, it's anything but. Companies lose billions of dollars each year by hiring the wrong people. Why is hiring so difficult? Because the odds are stacked against recruiting professionals. According to Fortune Magazine, 66% of all job applicants stretch the truth on their resumes. Compounding the problem, statistics show that a typical interview increases the likelihood of choosing the best candidate by only 2%!

As a performance improvement professional, you are in a unique position to help your organization minimize the risk of mis-hires. How? By applying the principles of performance technology to design and develop a Criterion-Referenced Instruction (CRI)-based Candidate Selection Tool that can help recruiting personnel pinpoint the best-qualified candidate(s) for any particular job. Here are some tips to help you develop an effective candidate selection tool:

Describe the Job in Detail
The first, most important step in developing a candidate selection tool is to come up with a well-written job description that captures the most critical components of a particular job. This job description should include:

1. Specific, observable, and measurable job tasks
2. Specific, observable, and measurable human characteristics

A Case in Point: CEP recently helped Hilton Hotels Corporation develop a CRI-based candidate selection tool for the Director of Revenue Management (DRM) position. Using job and goal analysis, CEP transformed non-specific job task descriptions and human characteristics into specific items that recruiters could easily measure. For example:

Assess Whether Candidates Meet the Requirements of the Detailed Job Description
With a detailed job description in hand, the job of the recruiter focuses on determining whether a candidate can or cannot perform according to the job description requirements. Every candidate selection procedure, including resume and/or work sample reviews, interviews, tests, social interactions, and reference checks, should integrate the critical job task details and human characteristics required for the position.

With respect to Hilton, CEP provided the following tools to help recruiters find the best qualified DRM candidates:

• Resume Evaluation Checklist: CEP developed a checklist to help recruiters determine which candidates to schedule for interviews. The form listed specific information for recruiters to look for, including evidence of hotel industry experience, management experience, educational background, familiarity with specific software programs, overall professionalism, etc.
  
  
• Interview Tool: CEP developed a list of behavior-based questions to help recruiters assess each candidate's prior work experience and capabilities. For example, to determine a candidate's ability to develop hotel pricing, a behavior-based question would read as follows: "Tell me about a specific instance when you were required to develop hotel pricing. How did you go about doing that? What information sources did you use?" CEP also developed an interview evaluation form to help recruiters assess candidate responses.
  
  
• Review Work Samples Tool: CEP also developed a tool to help recruiters request and review work samples. For the DRM position, the job aid specified that the work samples should be analytical or technical documents produced in a current or recent job assignment. The job aid also included questions to help recruiters evaluate the quality of the work samples.
  
  
• Work Simulation Exercises: For "finalist" candidates, CEP came up with exercises that would give recruiters an opportunity to observe candidates completing a specified task in a simulated work environment. The exercises were designed to validate the candidates' skills and their ability to apply them in realistic, work-related situations. In one exercise, for example, candidates assumed the role of a hotel DRM. Armed with appropriate data, the candidates received a series of "phone calls" for room bookings and were responsible for deciding, while still on the phone with each caller, whether or not to take the reservation at the rate being requested.

The Benefits of CRI-Based Candidate Selection
Hilton has used the DRM candidate selection tool with great success. According to the Corporate Director of Revenue Management and Training Programs, "We used the tool when recruiting at several universities. We hired 18 Revenue Management Analysts (aspiring DRMs) and, for the past six months, all of them have been very successful and are performing well. The tool qualifies candidates so well that we know they will be successful." In fact, since the introduction of the DRM candidate selection tool, there has been zero turnover in employees hired using this tool.

In addition to minimizing the risk of hiring the wrong job candidates, a CRI-based candidate selection tool can also help your organization meet legal requirements* by:

• Focusing only on job-related requirements, and
• Ensuring consistency in the selection system used for all candidates

For more information on how CEP can help you design and develop CRI-based candidate selection tools, contact Paula Alsher at 770-458-4080.

How May I Serve You?
Improving Telephone Skills at the Department of Labor's
Office of Workers' Compensation Programs

A brand-new study by Public Agenda, a US public policy research organization, confirms what most of us already know - customer service in many organizations leaves something to be desired. Telephone customer service, in particular, elicited one of the most negative reactions from survey respondents, with 94% saying the experience was "very frustrating."¹

A recent Department of Labor (DOL) survey of workers' compensation claimants produced similar results - marks for telephone customer service for claims examiners were found to be alarmingly low. In response, the Office of Workers' Compensation Programs (OWCP), which is part of the Labor Department's Employment Standards Administration, conducted analysis to find out why. The analysis showed that, among other reasons, claims examiners lacked needed customer service skills.

New-Hire Training
CEP partnered with the OWCP to design Criterion-Referenced Instruction (CRI)-based training that would ensure that new-hire claims examiners would master the critical telephone skills needed to perform to expectations. CEP began the project by working with exemplary performers to identify the tasks that new claims examiners needed in order to be successful. CEP then determined the standards for acceptable performance, making sure that these standards were written in observable and measurable terms.

The new-hire training that CEP developed consisted of self-paced modules that were designed to give learners the skills and knowledge needed to:

• Be professional, pleasant and empathetic
• Process telephone calls according to established standards
• Handle difficult callers

To ensure mastery of each skill being taught, the training also included realistic call simulations, skill checks, as well as an Expert Telephone Skills Training Scorecard. The training successfully provided new hires with both the skills and the self-confidence needed to perform to expectations. Because of this success, within a year of implementing the new-hire training, the OWCP asked CEP to expand the program to encompass experienced claims examiners.

Training for Experienced Examiners
Designing and developing training to provide experienced personnel with skills that, for all intents and purposes, they should already have, presents unique challenges. It's not uncommon for experienced employees to react negatively to the mere mention of training. "Hey, I've been doing this for ten years! Are you telling me I don't know how to do my job?!"

To overcome this potential challenge, the OWCP made certain that the low marks from the claimant survey were communicated to all claims examiners, along with management's commitment to improve customer service. The OWCP also placed a time limit on the training program in order to minimize the time that employees needed to spend away from their jobs - the program had to be only one day in length.

CEP tackled the project by conducting a focus group comprised of exemplary claims examiners representing the various OWCP district offices. CEP and the focus group:

• Examined the new-hire training to determine what elements made sense for experienced examiners and what areas were too basic
• Identified additional needs that were unique to experienced examiners (for example, workload management, handling complex calls, etc.)
• Revised the call simulations to reflect the increased complexity and difficulty that experienced examiners faced on the job
• Ensured that the training program would be enjoyable and non-threatening to the experienced examiners

After making the necessary modifications to the training program, CEP then conducted a train-the-trainer session to prepare the district offices to roll out the training. The train-the-trainer session focused on giving OWCP "coaches" the skills needed to provide positive, constructive feedback to experienced examiners.

These actions helped CEP and the OWCP ensure the successful rollout of the telephone skills training program to the experienced examiner target population. In fact, the training has proven so successful that CEP is now helping the OWCP to design and develop customer service skills training for the Energy Employees' Occupational Illness Compensation Program.

For advice or support on your next call center project, call Paula Alsher at 770-458-4080.

Periodically, we send out e-mails regarding special discount offers on workshops and books, as well as information about new products and services that will help you achieve positive bottomline results. If you would like to be added to this distribution list, please click here.

If you would like to be removed from our newsletter mailing list, please send an e-mail to info@cepworldwide.com with "Unsubscribe to Newsletter" in the subject field.

The Center for Effective Performance, Inc.
2300 Peachford Road, Suite 2000, Atlanta, GA 30338
770-458-4080, 1-800-558-4CEP
770-458-9109 (Fax)