SAP INSIGHT

END-USER PERFORMANCE:

BUILDING AND MAINTAINING ROI

An Outlook on Best Practices

Table of Contents
Executive Agenda The Performance Cycle: Five Phases
Change Management: Achieving User Buy-In Assessment, Strategy, and Development: Devising Content and Format Knowledge Transfer: Delivering the Information Operations: Managing the Process Performance Management: Analyzing the Results

1 3

Best Practices in Action Conclusion

8 10

END-USER PERFORMANCE:
BUILDING AND MAINTAINING ROI
by Stephen Hirsch

EXECUTIVE AGENDA
Software alone is not enough to make your business run more smoothly and cost-effectively. For software to deliver real value to your organization, end users must have the skills and understanding to use it accurately and intelligently. In the earlier years of enterprise resource planning (ERP) solutions, the end user rarely received much more than a passing thought. While most companies delivered some degree of training, that often consisted of a short how-to class delivered well before the system went live. By the time users first stared into the new system, they had forgotten what little they had learned. Worse, they had in many cases developed an all-too-human resentment towards the new solution. Such attitudes not only slowed their time to competence, but also contributed significantly to some of the notorious ERP disasters of the 1990s. If the people couldnt or wouldnt use the software, the software simply wouldnt deliver on its potential. Indeed, studies of system implementations consistently show that human performance, not technology, is the leading cause of failure to achieve expected results. In the years that have followed, companies have become more aware of end-user needs and the critical relationship between user preparation and implementation success. However, more than a vestige of the old ways still remains: the perception that once training has been delivered, the organization reaches a level of competence that somehow remains constant. For example, at the Americas SAP Users Group Annual Conference in 2004, a large national media company revealed that neither its documentation nor user competence had been addressed since the initial implementation of SAP software in 1998 (despite regular upgrades). As a result, the companys return on investment had suffered. The companys ROI began to improve only with a concentrated effort to ensure the ongoing competence and confidence of its users. Indeed, as a 2006 report from IDC indicates, Training on complex systems is a never-ending cycle. There are always new processes, new employees, and new locations that must be brought up to speed or brought online, and no group can be left out.1 The phrase never-ending cycle can be a bit intimidating. But companies need not think that building and maintaining user competence is an insurmountable task. With careful planning and the right tools, businesses can create a performance cycle that establishes user acceptance and understanding up front and maintains and develops user skills in the long term (see Figure 1). As a crossindustry best-practice approach and framework for continuous learning throughout your organization, the performance cycle maximizes your end-user performance and ensures the overall success and ROI of your software implementation.

1. Process Training, Enabled by Tools and Process from SAP Education, Is Critical for Technology Adoption at Kimberly-Clark, IDC Corporation, August 2006.

This article will present the performance cycle as a high-level best practices approach to user performance. Well discuss each phase and suggest steps you can take to help prepare, instruct, and support your workforce as it uses its IT solution. Among other things, well examine the following: Why its important to prepare your users for your new software solution even before you implement it Why you need a user-training strategy, not just a user-training delivery system Why traditional training isnt enough to keep people trained The value of context-sensitive user help How monitoring user performance can help build your companys ROI Tools and technologies for creating contextsensitive user help, diagnosing end-user errors, and managing your corporate learning

As the importance of user performance becomes well known, fewer companies are viewing training as simply a costly expense. Companies adopting or adapting the performance cycle are realizing that creating and maintaining user competence is a key business process and a critical contributor to a rapid, sustainable ROI.

The Performance Cycle

T, EN & M Y, T SS G EN SE ATE PM S A TR LO S VE E D

KN TR OW A LE N SF DG ER E

PERFORMANCE MANAGEMENT

Figure 1
2

OP

ER

AT IO NS

PERFORMANCE CYCLE

GE T N AN CH EME G NA MA

THE PERFORMANCE CYCLE: FIVE PHASES

Essentially, the performance cycle consists of the following five phases: Change management Assessment, strategy, and development Knowledge transfer Operations Performance management Please dont think of the phases as an orderly sequence of events. Once your organization completes its software implementation or upgrade and your IT solution goes live, the various phases will occur simultaneously. For the purposes of this article, however, well assume your company is just getting started and examine each phase as a discrete step. CHANGE MANAGEMENT: ACHIEVING USER BUY-IN The performance cycle begins with what may well be its most critical aspect: organizational change management. In a recent worldwide survey of 186 implementations, SAP found that one of the leading barriers to successful implementation was organizational resistance. End users dont resist new software implementations simply to be ornery or because theyre lazy. According to Benjamin Hoetzel of the University of the Applied Sciences in Germany, Employees fear that they cannot keep themselves up-to-date with technology and therefore fear for their jobs technology evolves at such a pace that it generates so called techno-stress among employees at all levels of an organization.2 A successful change management program, therefore, is far more than simply sending out an e-mail and announcing the new business application. A successful change management project does the following: Ensures that end users not only know what they need to do, but also why they need to do it Involves precise identification of the changes the new solution will bring and careful structuring of the content, timing, and delivery of your messaging Requires identification and enlistment of reliable change leaders (both formal and informal) who can influence their subordinates and peers and a host of other activities directed at both end users and all levels of management The larger the implementation project, the more comprehensive the accompanying change endeavor should be. But the goal will always be the same: to build the buy-in that leads employees to eagerly accept, understand, and learn your solution. ASSESSMENT, STRATEGY, AND DEVELOPMENT: DEVISING CONTENT AND FORMAT As noted above, the various phases of the performance cycle arent necessarily an ordered progression of steps. Your assessment, strategy, and development phase may very well be going on in tandem with or even before your change management process begins. Its during this phase that your corporate training team will determine what your end users need to learn and how they are going to learn it. Youll analyze your users current skills and required skills, their learning styles, your system and network infrastructure, and your overall corporate culture, all with the goal of helping you decide what content to develop and in what format. Youll also need a strategy for deploying that content for best effect, and you should consider the merits and limitations of all different delivery methods (for example, classroom training, e-learning, and online help).

2. Hoetzel, Benjamin R., Change Management Within ERP Projects, University of Applied Sciences, Heilbronn, Germany, 2005.

If you dont have a dedicated staff of training professionals in your company, outside consultants can help plan your user preparation initiative, as both the approaches available for end-user education and the tools to deliver it have evolved dramatically over the last decade. Indeed, the huge training projects that herded entire departments into a formal classroom are steadily becoming a thing of the past. Do your employees actually need classroom training to learn to carry out transactions? Or, in our world of constant process change, would it make more sense to provide them the means to find the information they need quickly to complete a transaction rather than have them memorize how to do it? Is classroom training a cost-efficient delivery system for solution concepts or repetitive processes such as system navigation? Probably not. For such topics youll likely get better results (and save considerable time and money) by employing e-learning. Classroom training still has its benefits and is best suited for topics that require extensive discussion, monitored lab work, or employee networking, for example. But given the costs in time and money associated with classroom training, its best to reserve it for the most appropriate situations.

Theres one important thing to keep in mind during your planning. Deciding to deliver information using e-learning, classroom learning, or some combination of both does not mean you have a learning strategy. A delivery vehicle is only a vehicle the truck that transports the package. One truck may be more efficient than another, but what really matters is the quality of the instructional package itself. A large part of developing a learning strategy consists of honing learning objectives to meet actual business needs (as opposed to nice-to-know information), structuring content to meet them, and planning how to determine the achievement of set goals. Evaluation deserves particular attention: in an age when training budgets are the first to be slashed when times get tough, its important to be able to say, with proof and confidence, this worked. KNOWLEDGE TRANSFER: DELIVERING THE INFORMATION Next is the knowledge transfer section and please note that we did not call it training. Training in formal classes, whether online or in the classroom, is only a part of the overall task. Knowledge transfer will also encompass informal learning and the kind of information that corporate learning evangelist Elliot Masie refers to as fingertip knowledge.

Do your employees actually need classroom training to learn to carry out transactions? Or, in our world of constant process change, would it make more sense to provide them the means to find the information they need quickly to complete a transaction rather than have them memorize how to do it?

Were all familiar with classroom instruction of one kind or another, and by now many of us have experience with e-learning or similar technologybased training. The idea of fingertip knowledge, however, may be quite new for many people. Nonetheless, its based on sound reasoning.3 For years, people have learned how to use software applications by attending a formal class or working with a computer-based training application, or CBT. The lucky ones have been able to retain some of the information long enough to apply it. Too often, however, theres too much of a gap between training and system go-live, and what little was learned in the first place gets lost in the black hole of forgotten skills. If your end users need to learn a series of steps for carrying out a transaction, it makes sense to provide them the information they need at the precise moment they need it, either as a supplement to or replacement for formal classroom training. If your company supports hundreds of end-user business transactions, creating context-sensitive help for each of them may sound like an overwhelming task. However, current technologies make creating and implementing custom help a much simpler process than it might seem. Lets assume your end users need help with the process of creating a purchase order transaction. Assuming your content developers have an automated content-creation tool, they will have created a job aid for that specific transaction during the assessment, strategy, and design phase. A job aid is a step-by-step series of directions for the process, complete with screen shots of your actual environment. Creating it can actually be quite simple: while the content creator runs through the transaction on the companys own system, the content creation tool records the steps, automatically creating a detailed job aid reflecting

the companys actual screens and data. Depending upon the product chosen, it may also create an exact simulation of the process. After editing the job aid with any additional text required, the content creator (or whomever is designated to do so) publishes the result to the companys help database. When the end users are about to execute the transaction, they can simply invoke the help menu to see complete documentation or to run the simulation. As a result, the users no longer need to go running for offline documentation, there are fewer level-one help-desk calls, user errors decrease substantially, and the company has saved a fortune by employing fingertip learning rather than classroom training. Of course, not all learning takes place in a controlled environment. Some experts estimate that informal learning can comprise 75% or more of the organizational learning an employee experiences.4 As Dr. Marc Rosenberg points out in Beyond E-Learning, When this happens, the effect is to democratize learning in the organization, creating a culture that creates more collaborative work and collaborative learning.5 Your end users will naturally seek the help and advice of those coworkers they deem to be the local gurus, or super users, whether or not youve anointed them as such. Companies neednt worry about organically created communities of practice such as these; it will quickly become evident who is able and eager to assume a guru role and which lines of communication provide reliable information. Providing tools that can encourage development of such naturally occurring support organizations will be to your advantage and help foster an environment conducive to learning. For example, some of the more recent tools for building and deploying end-user help also provide functionality for user-to-user and user-to-content developer communication. And if such a concept as a com-

3. Elliot Masie, Presentation at ASUG Annual Conference, May 2006, sponsored by the independent group Americas SAP Users Group (ASUG). 4. For example, see Marcia Conner, Informal Learning, www.agelesslearner.com, 2005, or Sam Adkins, ROI from Workflow-Based E-Learning, www.learningcircuits.org, 2003. 5. Marc Rosenberg, Beyond E-Learning, 2006, p. 77.

munity of practice sounds more like something todays college sophomore would be familiar with, remember that in a few years that sophomore is going to be in the workforce and expecting the kind of learning environment he or she is now experiencing. OPERATIONS: MANAGING THE PROCESS From the SAP point of view, the knowledge transfer and operations phases of the performance cycle almost become one. As you carry out your daily operations, youll also be continuing the previous phase (knowledge transfer) and gathering and using information for the next one (performance monitoring). For example, during the operations phase your help desk will be using data collected by your performance monitoring application to help assess and correct user errors and issues. However, youll also be managing the learning environment youve created on both the individual and corporate level which can actually be a rather complex process. Its in this phase that youll be posting course information; creating, modifying, and monitoring individual learning plans; managing attendance and assessment; distributing

training-related announcements (part of ongoing change management); hosting e-learning; and carrying out any number of other management activities. To keep both corporate and personal sanity, we strongly recommend use of a comprehensive learning management system (LMS). A powerful LMS will allow you to both track, manage, and deliver corporate learning on both a departmental and individual level and it will be particularly helpful during the operations phase. If you can integrate your LMS with your human capital management system, so much the better. Corporate learning isnt something separate from your key business processes; it exists to grow your business and should be managed as part of your overall business context. If you can integrate learning and performance records (for example, test results, competencies, or classes completed) with your other HR information, that will simplify your companys overall personnel management and give you a much more accurate picture of the whole.

Corporate learning isnt something separate from your key business processes; it exists to grow your business and should be managed as part of your overall business context.

PERFORMANCE MANAGEMENT: ANALYZING THE RESULTS The final phase of this continuous performance cycle asks some blunt questions. The first is Is any of this working? The second, even tougher, is something usually asked after the company has analyzed all of its investments in preparation, training, and user support: So what? Unfortunately, until very recently, its been hard to get valid answers to these questions. For the most part, companies depended on performance surveys or post-class assessments to determine whether anyone had learned anything from the classes they attended. The problem with that approach, however, is that it provided a snapshot of an unrealistic landscape in a fixed moment in time. The way a user performs in a formal test environment really has little or nothing to do with the way that same user performs in real life. In addition, user performance isnt a fixed thing. The same end user who performs extremely well in initial post-class testing may become flustered or develop bad habits on the job. As time goes on, factors such as normal employee attrition, the introduction of new or changed business processes, or pass the baton training (especially where there is no performance support) can erode the skills of

entire departments. The worst part of this scenario is that it can go on for months without anyone realizing it. Another approach to assessment involves help-desk reports, which provide a slightly more realistic picture. But these reports reflect only the calls people actually make. The user errors that dont get reported go undetected until they create a problem and usually not even then. The goal for the performance management phase is, quite simply, to find out what your users are really doing. With the proper tools, you can readily examine how your users interact with the software on the job. New software technologies for userperformance management allow you to see where they are doing things properly, where they are employing work-arounds, and where they are making mistakes. Only from actual performance data of this nature can you actually diagnose any problems and work on fixing them. When you know exactly which transactions result in errors or create bottlenecks, you can take action to correct them. This knowledge becomes the critical input to the next round of the assessment, strategy, and development phase of your ongoing performance cycle.

The way a user performs in a formal test environment really has little or nothing to do with the way that same user performs in real life.

BEST PRACTICES IN ACTION

The experience of the global health and hygiene company Kimberly-Clark exemplifies the performance cycle in action. As Kimberly-Clarks personal care products touch nearly a quarter of the worlds population with such brands as Kleenex, Scotts, and Huggies, the companys training requirements are substantial. In fact, its recent North American user-preparation initiative involved more than 25,000 learners. Kimberly-Clark began its project with a welldesigned change management effort focused on building buy-in on both the organizational and individual levels. To ensure that all its North American employees understood the purpose of a major software upgrade and how it would affect them, the company developed key messages and a comprehensive enterprise communication plan. It identified stakeholder groups and affected sites, then tailored communications to meet their concerns. Once the project began, they initiated regular employee communications via an intranet site, a project-specific newsletter, and both formal and informal presentations. Kimberly-Clark also embarked on a thorough planning effort, identifying the elements necessary to achieve successful user preparation. The companys approach was to identify how work gets done before the implementation and how that differed from the vision of how work should get done at the projects end. This gap analysis process served to identify training needs and the delivery methods by which to achieve them. Kimberly-Clark also wisely determined to include business process education along with transaction training to help end users understand how their own roles affected the overall business and the jobs of others. Also as part of the planning effort, Kimberly-Clark standardized templates, tools, and processes to create the reference materials for supporting end users. Kimberly-Clark began its knowledge transfer phase with formal classroom training. But as part of its strategy, those responsible made sure to teach end users where to go for help after the formal classes were over. To make sure that fingertip learning was available, they also implemented contextsensitive user support. According to a study by IDC, Kimberly-Clark estimates more than 4,500 procedural queries were made to their online reference system that might otherwise have been made to the help desk. If these queries had been routed to a help desk, there would have been an estimated 600 hours of productivity

The IDC report also estimated that using an automated online help development system allowed Kimberly-Clark to cut content development time by 55%, and that the overall savings in user and support staff productivity for self-service support could be worth more than [US]$120K to K-C per year.

loss in 2 months or more than [US]$60K in productivity savings annually.6 That number doesnt take into account the cost of the help desk calls themselves, which can be quite costly. The IDC report also estimated that using an automated online help development system allowed Kimberly-Clark to cut content development time by 55%, and that the overall savings in user and support staff productivity for self-service support could be worth more than [US]$120K to K-C per year. Kimberly-Clark also wanted to determine if its training efforts really did result in more efficient system utilization. Given the noise regarding its rollout, the company wanted to establish solid metrics regarding the actual return on the training investment. Company management also wanted ongoing information about user performance to help identify error trends and user issues. For something more accurate than surveys and helpdesk reports, Kimberly-Clark became an early adopter of a user performance monitoring system.

At the time of this writing, Kimberly-Clarks performance management processes have only been in operation for a few months, but they have already proven useful. For example, as reported by IDC, K-Cs initial findings showed that 95% to 98% of errors encountered were user errors. Five errors accounted for 90% of user problems; four transactions are involved in the majority of errors. The company plans to use data of this nature to help identify training gaps and fine-tune business processes.

6. Process Training, Enabled by Tools and Process from SAP Education, Is Critical for Technology Adoption at Kimberly-Clark, IDC Corporation, August 2006.

CONCLUSION
The rationale behind the performance cycle is straightforward: when your end users perform better, support costs go down, error rates decline, and user competence and confidence increase. But we should reiterate that user competence is not a static condition. User change is constant in any thriving enterprise, and unless users are trained on their new responsibilities, an organization can lose 30% to 50% of its end-user capability in just a few years through new hires, promotions, or other organizational changes. With this in mind, lets review how the performance cycle can build and maintain the ROI of your software implementation: Change management: As new hires enter the workforce or roles change, make sure they understand how they fit in the value chain. Giving them transactional training is certainly important, but its only part of the preparation process. If you keep their understanding as high as their skills, theyll be more careful workers. Additionally, as you refine business processes or introduce new functionality to your solution, make sure to prepare your workers for the changes; transitions will go more smoothly when you do. Assessment, strategy, and development: Keep analyzing your user environment and all its elements: have informal learning networks developed, and can you use them? Did one delivery method provide better results at lower costs? Do you have the proper materials for people who have switched roles or joined the company after go-live? And most important, if your business processes have changed, do you have the material and delivery capacity to support them? Knowledge transfer: Make sure that as business processes evolve, you modify your online documentation, simulations, and job aids to reflect the changes. Nothing will frustrate your users more than outdated, inaccurate information. The tools discussed in this article make such maintenance far easier than it used to be. Operations: Giving your end users the opportunity to grow post-go-live can keep enthusiasm for your solution high. Use your LMS to create, post, deliver, and track personal or departmental learning plans. Make sure to update personnel records to note classes taken, skills developed, or certifications obtained. Performance management: Always give yourself the power of knowledge. Know when and where your users are performing well and where they are having trouble. Use your performance management data to help determine where to add training or support or where to make modifications to existing content and strategies. And always remember to provide your company stakeholders with an ongoing view of performance so everyone knows of the improvements. That in itself becomes part of the next stop around the cycle change management. Again, while the performance cycle provides an excellent framework for all your user preparation and maintenance activities, it does not have to be a formal process. Whats most important is to make sure its elements exist and that you carefully consider the questions it raises. When youve successfully addressed these questions, your people will have the skills and understanding to deliver the full value your solution provides.

10

ABOUT THE AUTHOR Stephen Hirsch has spent over 20 years in the computer industry, 10 of them with SAP America. He is currently a business development manager for SAP Education. Prior to his years in the computer business, Hirsch was an associate professor of English and chairman of the humanities division at the College of Saint Rose in Albany, NY.

2006 SAP AG. All rights reserved. SAP, R/3, mySAP, mySAP.com, xApps, xApp, SAP NetWeaver and other SAP products and services mentioned herein as well as their respective logos are trademarks or registered trademarks of SAP AG in Germany and in several other countries all over the world. All other product and service names mentioned are the trademarks of their respective companies. Data contained in this document serve informational purposes only. National product specifications may vary. These materials are subject to change without notice. These materials are provided by SAP AG and its affiliated companies (SAP Group) for informational purposes only, without representation or warranty of any kind, and SAP Group shall not be liable for errors or omissions with respect to the materials. The only warranties for SAP Group products and services are those that are set forth in the express warranty statements accompanying such products and services, if any. Nothing herein should be construed as constituting an additional warranty.

www.sap.com

50 081 807 (06/10)