Mapping Your Way to Effective Learning and Development
By Ann W. Parkman, Executive Vice President, and Dan Raymond, Senior Performance Consultant

Formal training makes economic sense when an obvious need for training to develop skills and knowledge exists. New hire training or training to teach experienced employees a new enterprise-wide software are perfect examples. But there are also countless cases in which formal training -- whether classroom-based instruction or e-learning -- does not make economic sense. Here are just a few examples:

• People who are promoted or are assigned different projects generally have some level of experience in their company or industry. The specific skills and knowledge they need for their new assignments will vary on a case-by-case basis.
• Senior- or executive-level people who are brought on board will almost certainly bring a high level of business experience. This, of course, doesn't mean they know everything; it simply means that what they don't know will again vary markedly by person and position. It is unlikely that a company's existing training curriculum or an external executive development program can target the specific needs of these individuals.
• Some positions may enjoy such low turnover, or be so specialized, that hiring is conducted infrequently at best. In these situations, the cost of developing and maintaining formal training may far outweigh the benefits.

Many organizations make some attempt to provide learning and development (L&D) opportunities that are outside the scope of their corporate training curriculums, such as industry conferences or tuition reimbursement programs. But very few organizations apply any form of upfront analysis to match needed skills and knowledge with available L&D opportunities.

In many cases, this means that people learn their new jobs or assignments through unstructured, on-the-job trial and error (as in "Go sit by Ellen and watch her work."). Even if they are fortunate enough to attend an L&D session, it's generally up to each person to know what skills and knowledge are lacking and to fill the void. This is an expensive and ineffective solution, especially when you consider that "people don't know what they don't know."

Mapping the L&D Process
We have found that one of the most effective ways to overcome this training challenge is to map out all available L&D opportunities by job position. An L&D map is a tool that provides employees and their supervisors with clear direction on:

• What skills and knowledge have already been mastered and what areas need work
• Sources for needed skills and knowledge, including but not limited to:
  • Industry conferences
  • Reading assignments
  • Meetings with and/or observations of subject matter experts
  • Coaching and mentoring
  • Formal training programs
  • Structured on-the-job training
  • Continuing education courses
• A formal structure for monitoring progress in each area on a regular and ongoing basis

L&D maps offer a number of benefits, including flexibility, the ability to take advantage of immediate L&D opportunities, and individualized mapping based on each person's specific needs. Below is an overview of the steps involved in developing an L&D map for a hypothetical Account Manager position at XYZ Co.

1. Identify or validate the job-critical tasks, skills, and knowledge needed.
It's been ten years since XYZ has had to fill the Account Manager position. Because of this, the performance technologist starts by conducting a task and skill analysis to identify all of the job-critical tasks for which this job is accountable. The task and skill analysis* includes a review of the current job position description along with interviews with the outgoing Account Manager and his direct supervisor, Pat.

During the interviews, the performance technologist is able to determine that, in addition to the critical tasks identified during analysis, there are a number of basic skills that any new Account Manager must have before he or she will be hired. There are also a number of preferred skills that XYZ would like a new Account Manager to exhibit. And there are proprietary or company-specific skills/knowledge that any new Account Manager will need to learn. The resulting report highlights some of the critical skills and knowledge identified for the Account Manager position:

Table 1: Critical Skills/Knowledge for XYZ Account Manager

2. Identify available L&D opportunities.
The performance technologist then works with Pat, Human Resources, and any other available resources to identify appropriate L&D opportunities to meet the "preferred" and "proprietary" skill and knowledge needs.

Table 2: L&D Opportunities for XYZ Account Manager

3. Map out the L&D process with the new employee.
After a comprehensive search, Pat proudly announces that Carol has accepted the Account Manager position. Carol has six years of prior experience, much of it in a similar capacity working for one of XYZ's major competitors. During her interviews with XYZ, she was able to demonstrate competence in two of the three preferred skills/knowledge (describing XYZ's industry and competitors and applying a proven sales process). Therefore, all Carol will need to learn is how to:

• Track and document customer leads and transactions using CRM software
• Explain and demonstrate features and benefits of XYZ's products
• Describe XYZ's sales processes and procedures

On her first day on the job, Carol sits down with Pat to plan out a map based on these specific L&D needs. They establish priorities and identify the resources Carol will need to access in the coming weeks. They also assign deadlines for completing each L&D opportunity, starting with more basic activities and progressing towards more complex skill and knowledge needs. Last but not least, they agree to meet on a regular basis to provide both Pat and Carol with a structured means of measuring Carol's developmental progress. Carol's L&D map for the first three months ends up looking like this:

Table 3: Carol's L&D Map

The Benefits of L&D Maps
An L&D map offers a highly flexible yet structured way of dealing with virtually any type of skill or knowledge need. It includes any necessary formal training programs as well as many other viable learning opportunities. Other benefits include:

• Immediacy: Many classroom-based training programs require a certain number of participants before a class can be held. This obstacle is eliminated by allowing people to engage in other L&D opportunities while waiting for a course to become available. Carol, for example, was able to identify some of the key tools available in the CRM software by reviewing the manual and by meeting with experienced users before attending an external training program.
• Tailored: L&D maps are tailored to the specific skill and knowledge needs of each individual while giving credit for what people already know and can do. Because of Carol's knowledge of the industry, prior sales experience, and demonstrated selling skills, she did not need to attend a regional industry meeting or sales development program as part of her initial L&D map. However, to remain abreast of current trends, she and Pat may choose to add some of these activities as future L&D opportunities.
• Learner-centric focus: L&D maps put people in control of meeting their own learning and development needs. It gives them information and access to available resources and lets them progress as quickly as they are able.
• Measurable progress: Setting regular meeting times for supervisors and their employees to agree on the course of the map and to assess progress toward learning and development is an excellent way to begin a working relationship based on open communication.

The task and skill analysis report can also easily form the basis for a Candidate Selection Tool that can weed out unqualified applicants (for more information on this tool, please go to http://www.cepworldwide.com/newsletter/risk_hiring.html).

For advice or support on L&D maps, contact Allison Ehrler-Meyer at 770-458-4080 or aehrlermeyer@cepworldwide.com.

Making Training More Accountable
By Patricia P. Phillips and Jack J. Phillips

Because of heightened skepticism in reporting, it is more important than ever to develop a training return-on-investment methodology that will stand up under intense scrutiny. The ROI methodology must meet certain operating standards to help ensure that there is consistency in the evaluation process and that a conservative approach is taken. Standards and guiding principles keep the evaluation credible and allow for replication of the methodology.

When implementing training ROI, here are some guiding principles to use as operating standards.

Report the complete story: ROI is a critical measure, but it is only one of many levels of evaluation necessary to explain the full impact of a program. Once the program has been implemented, you should evaluate how participants reacted (including their perceived ability to put the training into action), the extent to which participants improved their knowledge and skill levels, how well people are applying the skills on the job, and finally the business impact. If measurements are not taken at each of these stages, it is difficult to conclude that the results achieved are actually a result of the training and performance improvement program.

Enhance credibility: When collecting and analyzing data, use only the most credible source. Credibility is the most important factor in the measurement and evaluation process. Without it, the results are meaningless. Using the most credible source (often the participants) will enhance the perception of the quality and accuracy of the data analysis and results.

Be conservative: When analyzing data, select the most conservative alternative for calculations. This principle is at the heart of the evaluation process. A conservative approach lowers the ROI but helps build the needed credibility with the target audience. It is always better to be conservative than to provide a generous estimate and have results that are not credible.

Account for other factors: At least one method must be used to isolate the effects of the program. This step is imperative. Without some method to isolate the effects of the program, the evaluation results will be considered highly inaccurate and overstated. Some commonly used strategies include:

• A pilot group of participants in a training program is compared with a control group not participating in the program to isolate the impact of the program.
• Forecasts of anticipated results without the training program are compared to actual post-intervention results.
• Participants estimate the influence a training program has on key measures of impact.

Account for missing data: Sometimes training participants leave the organization or change their job function. If training participants cannot or do not provide post-intervention improvement data, assume that little or no improvement has occurred. It damages the credibility of the evaluation to make assumptions about improvements for which no substantiating data exists.

Adjust estimates for error: It's common to use estimates in reporting financial and cost-benefit information. To enhance the credibility of estimated data, weigh the estimates based on the level of confidence you have in the data and adjust accordingly.

Omit the extremes: Extreme data items can skew results. To eliminate the influence of extreme data items, omit them from the analysis. For example, if you have a list of numbers that all range from 30 to 70 except for one instance of the number 100, the number 100 would be considered an "outlier" or extreme data item and should be eliminated.

Capture annual benefits for short-term programs: Only use the first year of benefits in the ROI analysis of short-term programs. If benefits are not quickly realized for most training and performance improvement programs, they are probably not worth the cost. Therefore, for short-term programs, consider only annual benefits. Reserve multiple-year ROI analysis for more extensive programs where implementation spans a year or more.

Tabulate all program costs: The ROI methodology must include all of the costs associated with the training and performance improvement programs. These costs include the initial needs assessment; development; delivery costs including facilitator, facility and participant costs; opportunity costs associated with employees being absent from their jobs during training; and evaluation costs. Although the term ROI has been used loosely to express any of the benefits of a training and performance improvement program, a credible ROI methodology includes monetary costs. Omitting or understating costs will destroy the credibility of the ROI results.

Collectively, these guiding principles will ensure that the ROI methodology is credible and that it produces accurate values and consistent outcomes. It also ensures that the impact study can be replicated--when two or more practitioners evaluate the same program, they should always result in the same measurement.

Patricia Pulliam Phillips is chairman and CEO of The Chelsea Group, a research and consulting company focused on accountability issues in training, human resources, and performance improvement. Phillips is the author of numerous books and articles on training ROI including, most recently, The Bottomline on ROI: Basics, Benefits, & Barriers to Measuring Training & Performance Improvement (List Price $16.95, 117 pages, 1-879618-25-7, CEP Press, 2002). This new release provides a "CliffsNotes^TM" view of training ROI for individuals who require a basic understanding of the issues surrounding this important topic.

Jack J. Phillips, Ph.D., is with the Jack Phillips Center for Research, a division of Franklin Covey. He is an expert on measurement and evaluation and is the author or editor of over 30 books and 100 articles.

This article appeared in Workforce (www.workforce.com), September 2002.

Generating Triple-Digit Sales Increases at Caterpillar Americas (CACo.): A CRI Success Story

The Way It Was
For years, development for CACo. salespeople consisted of product update training coinciding with new product introductions. Over a one- or two-day session, product managers "downloaded" information to salespeople one product after another, sometimes explaining as many as four new products in a single session. Each product manager explained machine features, detailing additional horsepower, improved cab arrangements, increasing hauling capacity, and the like. The next product manager followed with another hour of similar information download. By day's end, salespeople's heads swam with details of machine minutiae. Sometimes, as was the case for Brazilian dealer salespeople, sessions were information only; machines were not even available for viewing or demonstration. This "training" did little to develop salespeople's ability to sell value.

The lack of access to machines back at the dealerships was another handicap for salespeople. Because of conservative dealer inventory practices, many salespeople rarely had exposure and access to machines. In some cases, they had not seen models they were expected to sell! This was at a time when the competition was improving and many long-standing Caterpillar product advantages were fading. Machines no longer sold themselves, as they once had. Salespeople required new skills and knowledge to communicate product value and influence customer buying decisions.

The Experts' Way
To bring about continuous professional development, CACo. sought out best practices in the training and development field. Staff members studied and consulted with nationally known training and performance development leaders and organizations. They learned the latest development practices and methodologies. CACo. staff members became certified in crucial skill sets such as Robert Mager's Criterion-Referenced Instruction, the renowned methodology for causing behavior change.

Three key findings came from the research:

1. All performance improvement activity must be targeted at improving a specific business result. In the real world, this is rarely done. Therefore, most training and other performance initiatives fail to produce any real improved business outcome.
2. Building superior performance takes much more than training. The usual response to performance problems is to provide only training. But most often factors such as ineffective management, weak processes and systems, or incorrectly designed compensation programs are really the cause of poor performance. Training will not fix any of these; it will only improve bona fide skill and knowledge deficiencies.
3. When training takes place, it must be done right. It usually is not because traditional training does little more than pass on information. Conversely, well-designed training concentrates on building specific skills needed to competently perform required tasks.

These insights became the foundational elements for the dealer sales development program known as Pro2000. Its long-term aim was to increase key business results, such as sales, percent of industry sales, revenue, margin, and customer loyalty. The team knew that every day it delayed in launching the program was a day of lost sales opportunities. With the business plan in the balance, time was crucial. Under the pressure of slumping sales, the team designed, developed, and launched Pro2000 in two months, a fraction of the time normally required for a project of this magnitude.

Pro2000 Product Training
One team innovation was the reengineering of product training for the program. The new training design provided classroom overviews of Caterpillar and competitive products followed by extensive "in the iron" opportunities to study and work with the equipment. In a production performance section, for example, they performed job studies to illustrate key machine design and performance concepts. Salespeople then faced written tests and delivered presentations to demonstrate what they learned. Over the next three days, the salespeople learned more about Caterpillar and competitive products than they had in their entire careers.

During development of the training the team set a standard - anything mentioned in the classroom had to be illustrated in the factory tour or demonstrated on the machine in the field. A classroom was set up in the middle of the factory to show product quality and design differentiation of critical components. Component and assembly quality was emphasized again on the assembly line tour. Following the plant work, salespeople went to the field to study and work with equipment. They learned to use the product information to look beyond machine specification sheets to prove value.

Learner feedback indicated the training was very demanding, but it was the right thing to do. Salespeople were surprised by the intense, regimented, goal-oriented execution of the program. The response was overwhelmingly positive. Some salespeople, who did not sign up initially, attempted to register before the last session because of reviews from earlier participants. One dealer salesman commented, "This is what I always dreamed Caterpillar would be like."

Developing Pro2000 Selling Skills
It had been years since CACo. offered selling skills courses to most dealers. In fact, the Pro2000 team believed most dealer salespeople had no formal sales technique training. Although their improved product knowledge was clearly visible in the mock sales presentations made at the product training events, it was often apparent that they lacked selling skills.

Team members revamped the product training process to include selling skills development. Knowing that integrating product knowledge and daily, real-world practice are the keys to bringing about true behavior change, the team included skill-building exercises. In one role-play exercise, for example, one person would adlib the role of a competitive salesperson shooting down a Caterpillar product while another would utilize newly learned selling skills to respond appropriately.

Pro2000 Rewards and Recognition
To stimulate behavior change, the team built a rewards and recognition system that excited dealer salespeople. It awarded program points for meeting or exceeding performance standards in critical behaviors (such as winning back lost customers, finding new customers, keeping loyal customers, product and selling skills development, forecasting and goal setting, etc.). To grab salespeople's attention, the team chose Caterpillar-licensed merchandise as rewards. This was a first in Latin America and something the team knew every dealer salesperson coveted. As soon as participants earned program points, they could order their Caterpillar-licensed merchandise rewards, keeping program enthusiasm high.

Additionally, to recognize effort and accomplishment, participants would receive a plaque for completing the program regardless of point total. The top 15 point earners would be invited on a program recognition trip. Throughout the year, CACo. would distribute a program newsletter to keep participants posted on overall point standings and program developments.

Results Achieved
Every aspect of the program has experienced improvement and delivered results. For example, Lion S.A. in Brazil has seen impressive improvement among its fourteen Pro2000 salespeople. The number of units sold to "win-back" customers jumped from 12 to 41 in the first two years of the program (a 341 percent increase) while new customer units more than doubled from 36 to 81 (a 225 percent).

At Sotreq S.A., Brazil's largest dealership, excavator sales are up 512 percent, backhoe loaders 455 percent, motor graders 254 percent, and wheel loaders 152 percent. Sotreq increased the number of new customers by 60 percent over the previous year and won back 80 percent more competitive machine users in the last two years of the program. "Many of these were loyal competitor users who we considered a waste of time to call on before Pro2000," said Antonio Francisco, program manager at Sotreq S.A. "Pro2000 is the boldest and most spectacular development program that Caterpillar has ever done for salespeople. Caterpillar has managed to put something different into this program that captivates, motivates, and enchants the salemen and keeps them constantly connected to the philosophy of the program."

Reprinted courtesy of Caterpillar Inc. from Caterpillar Americas' Annual Quality Improvement Report on Pro2000.

Deciphering the Hidden Meaning Behind "Fuzzy" Goals
By Robert F. Mager

People are often expected to perform in ways that are not reflected in clear and observable tasks. In addition to performing specific skills, they are expected to "demonstrate responsibility" or "take pride in their work." Since these expectations are vague or fuzzy, how will you proceed? What will you do to help people achieve the desired state?

One way is through goal analysis. Goal analysis is appropriate any time these two conditions exist:

1. Someone describes an intent in abstract or fuzzy terms, and
2. The intent is important to achieve.

The function of goal analysis is to define the indefinable, to help you say what you mean by your important or abstract goals. Using goal analysis, it is possible to describe the essential elements of abstract states - to identify the main performances that constitute the meaning of the goal. Once you know the performances that collectively define the goal, you will be in a better position to decide which of these performances need to be taught and which need to be managed. Then you can select the most appropriate teaching or management procedures and arrange to measure your progress toward success.

Breaking Through the Fuzzies
To turn abstractions into a list of performances, write down everything people would have to say or do for you to agree they are achieving the goal. Without editing or judging, jot down everything that can possibly represent the meaning of the goal.

The reason you must complete this exercise without being judgmental is that it is often very difficult for people to think through the cloud of fuzzies to the specifics you are searching for. Usually, when we ask ourselves the meaning of an abstraction, we answer ourselves in yet another abstraction. It just takes a little time to get used to the process of listing performances.

Here are five strategies for getting things down that may help you describe the meaning of your goal. Use whichever is most productive for you.

1. Answer the question, "What will I take as evidence that my goal has been achieved?" If you want someone to demonstrate responsibility, for example, what would it take to make you agree that he or she is achieving this goal? Some possible responses include:

• Carries out assigned tasks on time
• Carries out tasks regardless of the time required
• Carries out tasks regardless of whether others have completed their own tasks
• Offers solutions to problems outside the immediate job

2. Answer the question, "Given a room full of people, what is the basis on which I would separate them into two piles - those who had achieved my goal and those who had not?" After all, you do make judgments about whether your trainees are acceptable in skill or attitude; you do make statements about their understanding or motivation or feeling. Now is the time to lay on the table the basis for those statements.
   
   
3. Imagine that someone else will be charged with the responsibility for deciding which of your trainees have or have not achieved your goal, and that you are going to tell Person X how to proceed. What will your instructions be? What should he or she look for? How will the person know a goal achiever when he or she sees one? Suppose you want people who are conscientious. Think about how you would tell someone how to recognize this state. Should Person X look for people who:

• Work neatly?
• Finish their work on time?
• Ask for extra assignments?
• Stay until their work is completed?

4. Think of people who have already achieved your goal, people who represent your goal, and write down the things they say and do that make them goal achievers. If you can't think of anyone who has achieved your goal, you have a problem. Perhaps your expectations are unreasonable. Perhaps the goal (as you perceive it) is unattainable. If so, then a change in expectation is in order.
   
   
5. If all else fails, here's a sure-fire way to get started. Just write down all the reasons you would never point to someone and say, "This person represents the goal." What behaviors, or absence of behaviors, would cause you to say, "This is not someone who has achieved this goal, and this is why." Once you've listed the negatives, it's easy to turn them around into positive statements.

Once you have jotted down the things you think might cause you to agree your goal has been achieved, you will need to go back over your list and do some tidying up and sorting out. Why? Because you are almost certain to find items that are at least as broad and abstract as the one you started with. You may also find redundancies and duplications, things you have said in more than one way. You may occasionally find items that describe procedure rather than outcomes, or means rather than ends. These are to be deleted, for the object of the analysis is to figure out how to know an outcome when you see one, not how to make one happen.

If a goal is important to achieve, then it is important to do more about that achievement than to simply talk about it in abstract terms. To achieve it, you need purposeful activity, activity that will get you where you want to go.

Excerpted from Robert F. Mager's Goal Analysis: How to clarify your goals so you can actually achieve them, 3rd Edition (List Price $22.95, 159 pages, 1-879618-04-4, CEP Press, 1997). For more information on this book, click here. For information on Robert F. Mager, click here.

Periodically, we send out e-mails regarding special discount offers on workshops and books, as well as information about new products and services that will help you achieve positive bottomline results. If you would like to be added to this distribution list, please click here.

If you would like to be removed from our newsletter mailing list, please send an e-mail to info@cepworldwide.com with "Unsubscribe to Newsletter" in the subject field.

The Center for Effective Performance, Inc.
2300 Peachford Road, Suite 2000, Atlanta, GA 30338
770-458-4080, 1-800-558-4CEP
770-458-9109 (Fax)