KPI Deployment Guide

EUROPEAN ORGANISATION FOR THE SAFETY OF AIR NAVIGATION
EUROCONTROL
KPI Deployment Guide
AIM/AEP/S-LEV/0007
Edition Edition Date Status Class
: : : :
0.1 12 Nov 2001 Draft General Public
EUROPEAN AIR TRAFFIC MANAGEMENT PROGRAMME
DOCUMENT IDENTIFICATION SHEET

DOCUMENT DESCRIPTION
Document Title KPI Deployment Guide
EWP DELIVERABLE REFERENCE NUMBER PROGRAMME REFERENCE INDEX AIM/AEP/S-LEV/0007 EDITION : EDITION DATE : 0.1
12 Nov 2001 Abstract This guide presents main activities required to deploy a performance measurement system.
KPI Checklist
Indicator Deployment
Keywords Activity Guide
Template
CONTACT PERSON :
Paul Bosman
TEL : 3333
UNIT :
AIM
DOCUMENT STATUS AND TYPE STATUS Working Draft Draft Proposed Issue Released Issue CATEGORY Executive Task Specialist Task Lower Layer Task CLASSIFICATION General Public EATMP Restricted
o o o
o o
o o
INTERNAL REFERENCE NAME :
AHEAD Electronic Filing System
AIM/AEP/S-LEV/0007
DOCUMENT APPROVAL The following table identifies all management authorities that have successively approved the present issue of this document.
AUTHORITY
NAME AND SIGNATURE Ertan Ozkan
DATE 12 Nov 2001
Author/Editor
Conrad Cleasby Programme Manager
12 Nov 2001
Quality Assurance
Page iv
Draft
Edition: 0.1
AIM/AEP/S-LEV/0007
DOCUMENT CHANGE RECORD The following table records the complete history of the successive editions of the present document.
EDITION 0.1
DATE 12 Nov 2001 Creation
REASON FOR CHANGE
SECTIONS PAGES AFFECTED All
Edition: 0.1
Draft
Page v
AIM/AEP/S-LEV/0007
- Page left intentionally blank -
Page vi
Draft
Edition: 0.1
AIM/AEP/S-LEV/0007
TABLE OF CONTENTS
1. 1.1 1.2 2. 3. 3.1
INTRODUCTION ......................................................................................................1 Purpose and scope ................................................................................................1 References ..............................................................................................................1 IMPROVEMENT MODEL .........................................................................................3 ACTIVITIES..............................................................................................................4 KPI Definition ..........................................................................................................4 3.1.1 3.1.2 3.1.3 3.1.4 3.2 3.2.1 3.2.2 3.3 3.4 Activity: Define KPIs....................................................................................4 Activity: Identify Production Process ...........................................................5 Activity: Brainstorming on KPIs ...................................................................6 Activity : Develop Data Collection Plan .......................................................7 Activity: Monitor KPI Compliance ................................................................8 Activity: Gather and Analyse Data...............................................................9
Monitoring and Analysing Data .............................................................................8
Activity: Improve Component ..............................................................................10 Activity: Refine KPIs.............................................................................................12 CONCLUSIONS AND FUTURE WORK .................................................................13 KPI DESIGN FORM..................................................................................15 MAJOR PITFALLS...................................................................................16 KPI CHECKLIST ......................................................................................17 CUSTOMER FORMS................................................................................20
4.
APPENDIX A APPENDIX B APPENDIX C APPENDIX D D.1 D.2 D.3 D.4 E.1 E.2 E.3
Who Are Our Customers?....................................................................................20 Customer Segments.............................................................................................22 Customer Evaluation Form ..................................................................................23 Customer Background Information.....................................................................24 IMPROVEMENT FORM............................................................................25 Improvement Matrix..............................................................................................25 Improvement Plan ................................................................................................26 Improvement Selection Checklist........................................................................27 DATA COLLECTION PLAN .....................................................................28
APPENDIX E
APPENDIX F
Edition: 0.1
Draft
Page vii
AIM/AEP/S-LEV/0007
1. 1.1
INTRODUCTION Purpose and scope

This document presents the general framework for the deployment of Key Performance Indicators (KPIs) within the context of a quality management system. It contains activities derived from a simple 3-tier improvement model. Activities are given in a step-by-step manner in order to give a clear picture what is required. Each activity is given in the following template:
Purpose: Purpose of the activity Steps: Steps needed to carry out the activity Input Artefacts: Required Inputs Frequency: Frequency of the activity Role: Who is responsible for this activity Tips and Tricks: Additional comments, tips and tricks
Resulting Artefacts: Created outputs
Each activity contains references to the applicable guides and required documents either in its input artefacts and/or its steps. If necessary, the activity is supported by a flowchart. Appendices contain some useful information that has not been covered in the guides. The document is organised as follows: Chapter 2 presents 3-tier improvement model and basic loop in a performance measurement system. Chapter 3 gives activities required to deploy a performance measurement system. Chapter 4 presents conclusions and future work Throughout this document the term performance measurement system is used to denote the decision support system that contains a set of Key Performance Indicators (KPIs) with well-defined measurement, monitoring, analysis and improvement procedures.
1.2
References
[1] Introduction to KPI Models, Edition 0.1, AIM/AEP/S-LEV/00004, 29 Oct 2001. [2] KPI Portfolio, Edition 0.4, AIM/AEP/S-LEV/0005, 23 Oct 2001. [3] KPI Top-12, Edition 0.1, AIM/AEP/S-LEV/0006, 13 Nov 2001.
Edition: 0.1
Draft
Page 1
AIM/AEP/S-LEV/0007
[4] KPI Measurement, Monitoring and Analysis Guide, Edition 0.1, AIM/AEP/SLEV/0008, 30 Nov 2001. [5] Change Management the 5-step Action Kit, C. Rye, ISBN 0749433809, Kogan Page Limited, 2001. [6] Major Pitfalls of Performance Measurement Systems - Module V, Randy R. LaBarge, Pacific Northwest National Laboratory, U.S. Department of Energy, PBMSIG Fall 99 Meeting, 17 November 1999, Washington D.C. [7] Handbook for Basic Process Improvement, May 1996, http://www.balancedscorecard.org/files/bpihndbk.pdf. [8] The Process Improvement Notebook, Department of the Navy, Total Quality Leadership Office, Publication No: 97-01, 1997. [9] Operational Performance Measurement Increasing Total Productivity, W. Kaydos, ISBN 1574440993, CRC Press LLC, 1999.
Page 2
Draft
Edition: 0.1
AIM/AEP/S-LEV/0007
2.
IMPROVEMENT MODEL
The main objective of a performance measurement system is to provide a basis (i.e., facts) for continual improvement: Business Improvement (Achieve the Objective) Performance Measurement System (Define KPIs, Collect and Analyse Data) Technological Know-how (Databases, Intranet, O/S, etc.)
Figure 1. 3-Tier Improvement Model
The above improvement model summarises basic skills required to deploy a performance measurement system. At the lower layer technological know-how is required to support data collection and distribute data to the related persons. In the second layer is where KPIs are defined, collected, monitored and analysed. The top layer, simply the objective, is to improve the business (i.e., product, process or system). However this is not a one-shot approach. There is a continuous cyclical process. This cycle focuses on the internal processes (diagnostic measures) and external outcomes (strategic measures). The system's control is based on KPIs that are tracked continuously over time to look for trends, best and worst practices and areas for improvement.
Define KPIs
Refine the KPIS
Monitor KPI Compliance
Improve Product/Service, Process, System
Gather and Analyse Data
Figure 2. Loop in a Performance Measurement System
Edition: 0.1
Draft
Page 3
AIM/AEP/S-LEV/0007
3. 3.1
3.1.1
ACTIVITIES KPI Definition

Activity: Define KPIs
Purpose: To generate a possible set of KPIs covering system, customer satisfaction, human factors, product/services and processes. Steps: 1. [Optional] Create the list of products produced and/or services provided by your organisation 2. [Optional] Identify your major (segments of) customers and their profiles (i.e., services, products, delivery methods and requirements) 3. [Optional] Identify your system (organisational) requirements and goals 4. [Optional] Identify list of principal processes used in production and service provision via the Activity Identify Production Process. 5. Create a possible list of KPIs for the following categories: financial, customer satisfaction, human factors and business processes via the Activity Brainstorming on KPIs. 6. Formulate each KPI by using the KPI Design Form as a result of brainstorming meeting. 7. Apply KPI Checklist on the list of possible KPIs (result of the Activity Brainstorming on KPIs) in order to select your initial set of KPIs. 8. Refine data collection methods for each KPI via the activity Develop Data Collection Plan. 9. [Optional] Set an objective for each selected KPI (if you have historical data) Input Artefacts: Resulting Artefacts: None Frequency: Once initially and after a change in the system, products, processes, etc. Role: Quality manager Tips and Tricks: Steps between 1 and 4 are optional prerequisites of this activity. They are given in order to illustrate the complete picture. Ideally implementation should begin with identifying external customer segments and determining what they want. Then a strategy should be developed which will lead to defining the organisation s key deliverables, KPIs and production processes. KPIs should then be implemented in the principal processes according to where the greatest return or risk exists. Though this sequence is conceptually sound it is not always necessary to have a well-defined strategy before implementing KPIs. Since everybody wants whatever they are buying cheaper, better and faster. There are usually significant areas of opportunity available that can be addresses with KPIs while any strategic questions are being resolved [8]. For Step 2 you may use the following forms: o Who are our customers? o Customer Background Information o Customer Evaluation Form o Customer Segments List of Selected KPIs
Page 4
Draft
Edition: 0.1
AIM/AEP/S-LEV/0007
3.1.2
Activity: Identify Production Process
Purpose: To chart the current activities involved in achieving a particular service or product and their interrelationship and identify all of the component parts of the current process and commit each to a 'Post-it'. Steps: 1. Establish the dependencies of each component to others and move 'Post-its' to define logical sequences of events. 2. Establish component activities or sequences which can parallel other components or sequences and move 'Post-its' to illustrate these parallel relationships. 3. Add lines and arrow connections to connect activities and sequences and illustrate the flow of activity. 4. [Optional] For clarity, realign 'Post-its' to place the 'primary' process (the one to which other activities and parallel processes contribute) in a straight line across or down the overall chart. 5. [Optional] Add to each component or sequence the name of the party responsible for its completion. This may help to identify inefficiencies resulting from 'hand-offs' between parties. Input Artefacts: None Frequency: Before process improvement and selecting KPIs for the process Role: Quality manager and responsible person(s) for this process Tips and Tricks: Make sure that each (human) business actor express a role, not a person! This technique is frequently used during Business Process Re-engineering to establish the current situation. It could also be used to illustrate a proposed new process prior during the planning and implementation stage of a change situation. Process mapping is essentially a form of flowcharting and users may wish to increase clarity by using proper flowchart symbols. Resulting Artefacts: Chart of Production Processes
Edition: 0.1
Draft
Page 5
AIM/AEP/S-LEV/0007
3.1.3
Activity: Brainstorming on KPIs
Purpose: Generating large numbers of ideas around the KPI theme from a group of people in a short period of time. Steps: 1. Gather an appropriate group together. Typically, this will comprise those with a vested interest in the KPIs, that is, those involved with or impacted by it. 2. Advise the group of the problem or issue and check that all participants fully understand the concept of the KPI. 3. If appropriate break the issue down into manageable topics, eg 'the deployment of KPIs ' might split into 'how to generate the right KPI specifications', 'how to measure KPIs , 'how to report KPIs' and 'how to interpret KPIs and use for improvement . State the sub-issue which the group are to tackle first and note it at the top of a flipchart page. 4. Invite the members of the group to call out their suggestions to resolve the problem or issue. Record every idea, as stated, on the flipchart so that the developing list is visible to the group. Encourage succinct statements. 5. Do not allow discussion or criticism and aim for quantity not quality. 6. Permit laughter at wacky ideas they often have tremendous potential but quash ridicule of group members when they present ideas which may seem outrageous. Input Artefacts: Introduction to KPI Models KPI Portfolio KPI Top 12 Frequency: Before designing the KPIs. Role: Brainstorming meeting is organised by the quality manager Tips and Tricks: Brainstorming is a powerful, versatile and simple technique for generating ideas but demands disciplined facilitation by the group leader (who should not participate in the ideas generation process itself). It can be applied to almost any problem. Resulting Artefacts: List of possible KPIs
Page 6
Draft
Edition: 0.1
AIM/AEP/S-LEV/0007
3.1.4
Activity : Develop Data Collection Plan
Purpose: Determination of the tracking of the whereabouts of specific 'events'. Typically these events are product defects or problem areas within a process and/or a system and It enables their location and frequency of occurrence to be established. Steps: 1. Identify the 'event' to be tracked. 2. Determine a means of data collection. Examples might be: A plan of the layout of the area in which the 'event' is occurring; An illustration of the product incurring the 'event' (defect); An example of the product itself; A process-map of the process within which the 'event' is occurring; A checklist listing the zones within the overall organisation in which the 'event' is occurring. 3. Determine who is best placed to collect the 'event' data. Input Artefacts: KPI for which data to be collected Frequency: For each selected KPI Role: Quality manager and/or responsible person(s) where the event is occurring (i.e., a process, system, etc.) Tips and Tricks: The accuracy of data collection is paramount while determining data collection method and care should be exercised when determining how, and by whom, data is to be collected particularly when large geographical areas are under examination. For production processes, charts of production processes created in the activity Identify Production Process helps the team determine who should collect data and where in the process data should be collected. Use the form Data Collection Plan in order to formalise your work Resulting Artefacts: KPI Data Collection Plan
Edition: 0.1
Draft
Page 7
AIM/AEP/S-LEV/0007
3.2
3.2.1
Monitoring and Analysing Data

Activity: Monitor KPI Compliance
Purpose: Collect data for the KPIs as stated in the KPI Data Collection Plans and monitor KPI compliance. This activity is the part of the operational control. Steps: 1. Record the event data as appropriate stated in the KPI Data Collection Plans- until such time as an adequate sample has been obtained. 2. [Optional] If there is no objective set for this KPI and derive the KPI objective. 3. Check whether the KPI objective is attained. 4. If the objective is not attained, evaluate results to determine action to resolve the root cause of the problem. Input Artefacts: KPI Data Collection Plans KPI Measurement, Monitoring and Analysis Guide Frequency: Depending the critically and stability of the operations. Usually daily or weekly. Role: Operational Manager Tips and Tricks: Data sample size is also critical. As a general rule, if in doubt increase the sample size. Resulting Artefacts: Operational Monitoring Reports
Page 8
Draft
Edition: 0.1
AIM/AEP/S-LEV/0007
3.2.2
Activity: Gather and Analyse Data
Purpose: Gather KPI measurements and analyse them. Steps: 1. 2. 3. 4. Gather KPI measurements. Create executive reports containing vital information about the performance Check overall KPI compliance. Analyse each KPI and determine problematic components (a component may be one of the following: product, service, process, system, resource or expansion of market) 5. Check whether an improvement is required and possible for problematic components. 6. Prioritise and select the component(s) to be improved and execute the activity Improve Component for the selected component(s) Input Artefacts: Resulting Artefacts: KPI Measurement, Monitoring and Analysis Guide Frequency: Depending the critically and stability of the operations. Usually monthly, quarterly or annually. Role: Quality Manager Tips and Tricks: For step 4-6 you may use the form Improvement Matrix to analyse improvement alternatives. For step you may use the form Improvement Selection Checklist to verify whether a correct component is selected. Executive Reports Selected Components for Improvement
Edition: 0.1
Draft
Page 9
AIM/AEP/S-LEV/0007
3.3
Purpose:
Activity: Improve Component
Improve a selected component. Component can be one the following: product, service, process, system, resource or expansion of market. Steps: 1. Establish a well-defined improvement objective. 2. Organise a team to improve the component. This involves selecting the right people to serve on the team; identifying the resources available for the improvement effort, such as people, time, money and materials; setting reporting requirements; and determining the team s level of authority. 3. Identify the root causes that prevent the component from meeting the objective. The team begins the Plan-Do-Check-Act Cycle here, using the cause-and-effect diagram or brainstorming tools to generate possible reasons why the component fails to meet the desired objective. 4. Develop an improvement plan for implementing a change based on the possible reasons for the component s inability to meet the objective set for it. These root causes were identified in Step 3. 5. [Optional] Check whether KPIs related to the component do exist and they provide all the required data for the evaluation of the improvement objective. If not, define new KPI(s) for this component and collect baseline data. 6. Test the changed component and collect data. 7. Assess whether the changed component is stable. The team uses a control chart or run chart to determine stability. If the component is stable, the team can move on to Step 8; if not, the team must return the component to its former state and plan another change. 8. Assess whether the change improved the component. Using the data collected in Step 6 and a histogram, the team determines whether the component is closer to meeting the process improvement objective established in Step 1. If the objective is met, the team can progress to Step 9; if not, the team must decide whether to keep or discard the change. 9. Determine whether additional improvements are feasible. If yes, re-start the activity from Step 3. 10. Standardise the change and reduce the frequency of data collection Input Artifacts: Resulting Artifacts: Selected Component for Improvement List of KPIs related to the component Frequency: For each improvement Role: Quality Manager Tips and Tricks: You may use the form Improvement Plan in order to document your improvement effort. Improved component
Page 10
Draft
Edition: 0.1
AIM/AEP/S-LEV/0007
Step 1 Establish the improvement objective
Step 2 Organise the right team
Step 3 Identify root causes for lack of capability
Step 4 Plan to implement the change
Step 5 Modify the KPIs (if necessary)
Step 6 Test the change and collect data
No
Step 7 Stable?
Yes
No
Step 8a Yes Improved?
No
Step 8b Keep?
Yes
Yes
Yes
Step 9 More Improvement?
No
Step 10 Standardise the change and reduce the frequency of data collection
Figure 3. Flowchart of Activity Improve
Edition: 0.1
Draft
Page 11
AIM/AEP/S-LEV/0007
3.4
Activity: Refine KPIs
Purpose: Handles changes in KPIs (i.e., addition, modification, deletion). Sets or modifies KPI objectives. Steps: 1. [Optional] Add missing KPIs for critical components. Apply KPI Checklist for the new KPIs. Determine data collection methods for each KPI via the activity Develop Data Collection Plan 2. [Optional] If necessary, refine data collection methods for each KPI via the activity Develop Data Collection Plan after a change in the associated component. 3. [Optional] Delete KPIs that are not providing required feedback. 4. [Optional] Set/Modify KPI objectives. Input Artefacts: List of current KPIs KPI Data Collection Plans Frequency: Not applicable Resulting Artefacts: Modified KPI list Modified KPI Data Collection Plans
Role: Quality Manager Tips and Tricks:
Page 12
Draft
Edition: 0.1
AIM/AEP/S-LEV/0007
4.
CONCLUSIONS AND FUTURE WORK

This document presents general framework for a performance measurement system. The ultimate goal of the performance measurement is to ensure customer satisfaction by providing high quality services/products in a cost effective manner. The deployment of a performance system can be quite costly depending on the number of Key Performance Indicators (KPIs) and the effort required to measure them. Therefore selecting the appropriate KPIs is the key for successful application of this framework and common sense must prevail when selecting KPIs. The framework and forms associated with it are proposals. Therefore it is likely that this document will evolve with your feedback.
Edition: 0.1
Draft
Page 13
AIM/AEP/S-LEV/0007
- Page left intentionally blank -
Page 14
Draft
Edition: 0.1
AIM/AEP/S-LEV/0007
APPENDIX A
KPI DESIGN FORM
You may use the following form to design your KPIs:
KPI DESIGN FORM KPI Name Description <Name of the KPI> <Description of the KPI>
Objective
<Objective of the KPI>
Type Effort Unit Assessment Method Possible Tools Analysis Frequency Comments
o Quantitative o Qualitative o Low o Medium <Unit of the KPI> <How to measure the KPI>
o High
<Name of possible tools which can be used in the KPI measurement and analysis> o Day o Week o Quarter o Year <Additional Comments> o Month o Year+
Edition: 0.1
Draft
Page 15
AIM/AEP/S-LEV/0007
APPENDIX B MAJOR PITFALLS Major Pitfalls of Performance Measurement Systems are given in the [6]: Measures not linked to strategy - Critical to do initially, but also revisit when either the organisational strategy or structure changes Measures not driven into organisation(s) below: Breaks the linkage with overall strategy. Should be driven into staff performance agreements at all appropriate levels. Too many measures: Creates lack of focus on what is really critical to managing your business. Not enough critical measures: You could be missing information vital to operations. Focusing only on the short-term: A cross-section of past (lagging), present and future measures is critical. Conflicting measures: Sub-optimises staff or organisational performance. Example: Measuring reduction of office space per staff member while also measuring staff satisfaction with facilities. Measuring progress too often: Could result in unnecessary effort and excessive costs, resulting in little or no added value Not measuring progress often enough: May not know about potential problems until it is too late to resolve easily. Collecting too much data: Could result in a mountain of data that really doesn t tell us anything more than a lesser amount of the same data. Collecting inconsistent, unrepresentative or unnecessary data: Critical to understand what the data will look like, when it will be collected, at what frequency, by whom and what it means, up front. Reducing the value of data: Too much data roll-up (summary) can mask the impact of potentially significant events or trends Driving the wrong performance: Be careful that the measure(s) you select will result in the desired result. Encouraging competition and discouraging teamwork: Measuring vertically frequently pits one internal organisation against the others. Try to measure horizontally. Failure to base business decisions on data: Developing performance measures or collecting data only to comply with a requirement does nothing to improve the position of the company.
Page 16
Draft
Edition: 0.1
AIM/AEP/S-LEV/0007
APPENDIX C
KPI CHECKLIST
Here is the KPI checklist you may use to select each KPI:
KPI CHECKLIST KPI : <KPI Name> Place a checkmark next to all of the items that apply to your KPI. o o o o o o Does the KPI motivate the right behaviour? Is the KPI measurable? Is the measurement of this KPI affordable (cost-effective)? Is the target value attainable? Are the factors affecting this KPI controllable by you? Is the KPI meaningful? Date: <dd/mm/yyyy>
Note: If you have selected an appropriate KPI, you should be able to check all of the items above.
The selection process is complicated by the number of potential KPIs and must be tempered by considerations such as: Organisational experience with KPIs Type of behaviours to be motivated Cost and effort of collection. Common sense must prevail when selecting KPIs. It should be remembered that the goal is to ensure customer satisfaction by providing high quality services/products in a cost effective manner. To meet above goals, AIS organisations should consider the following principles while selecting or defining new indicators: An indicator must motivate the right behaviour An KPI must be measurable An KPI must be affordable The objective set for an KPI must be attainable Factors effecting the indicator must be controllable by the service provider An KPIs must be meaningful to all the parties. Indicators that motivate the right behaviour: When choosing KPIs, it must be first focussed on the behaviour that is wanted to be motivated. There are no hard and fast rules for this; first a KPI must reflect the performance at the right areas:
Edition: 0.1
Draft
Page 17
AIM/AEP/S-LEV/0007
Processes and factors most critical to business Processes and factors where most improvement needed The most dissatisfied client group The most politically powerful client group Then it must motivate a pre-defined behaviour in these areas: To reducing costs and/or defects To increase production To speed time-to-market Etc. An initial set of KPIs must be picked up in order to measure performance related to these behaviours. Measurable: If it is not affordable to measure something to represent a service quality, that KPI is worthless and should not be considered. As an example user (i.e., end-to-end) response time is a very popular KPI among users. Certainly, this is one of the key factors shaping the usersopinions about the quality of service they are receiving. Unfortunately, measuring user response time on an end-to-end basis is still technical challenge today. At the other end of the feasibility spectrum is the availability of a service. This is relatively straightforward and can be measured with a minimum effort and difficulty. Affordable: No organisation has unlimited resources. The amount that can be spent on delivering any service is limited. Therefore, in setting objectives for a KPI, it is also necessary to consider whether the desired objective (i.e., service level) is affordable. This might also be thought of as being cost effective. An KPI should not require a heavy investment of time and money. In some cases, it will be necessary to devise alternative KPIs if the required data is not easily obtainable. Another important issue in this topic is the easiness of data collection. If KPIs cannot be easily gathered, then they will quickly lose favour, and eventually be ignored completely. No one is going to spend an excessive amount of time to collect KPIs manually. Ideally, all KPIs will be captured automatically, in the background, with minimal overhead. Attainable: To be useful, the KPIs must be set to reasonable, attainable objectives. It may be difficult to select an initial, appropriate setting for a KPI, especially when there is no available historical record of those KPIs. Organisations with active KPI programs will have the data needed to set a proper baseline. Others will have to perform an initial assessment to establish that baseline. Unless strong historical measurement data is available, it should be expected to re-visit and re-adjust the setting at a future date. In addition a realistic tolerance level must be included while setting the objectives in the absence of historical measurement data. Controllable: A KPI must represent something that is controllable. That is, the service provider must have the ability to exercise control over factors that determine
Page 18
Draft
Edition: 0.1
AIM/AEP/S-LEV/0007
the level of service delivered. The service provider has control over bringing an AIS publication to the post office, but has no control over the speed by which the post office delivers the mail. Thus, an objective that "all AIS publications will be delivered to our clients within 48 hours after production completion" is unfair and likely to be de-motivating to the service provider. Examples for this issues can be extended very easily: In a country where there are frequent power failures and the budget of AIS organisation does not permit the purchase of a standby generator to prevent service interruptions during these failures, strikes by union workers and poor service by a telecommunication company without a reasonable alternative. These are just a few of the factors that can place certain objectives behind the control of the service provider. Meaningful: In the context of the service level agreement a KPI must be meaningful and understandable to all the parties (i.e., client and service provider). Another way of stating this is to say that it must be relevant. Lets take an AIS organisation, which provides some of its services/products via Internet, as an example. Service provider decides to uses packet collisions and dropped packets in order to illustrate availability of its LAN component. They can be easily captured however they mean little to anyone other than a network engineer or system administrator. Although these KPIs might impact on the availability from the client s perspective, the relevance of such KPIs to the service they receive is quite difficult to grasp.
Edition: 0.1
Draft
Page 19
AIM/AEP/S-LEV/0007
APPENDIX D D.1
CUSTOMER FORMS
Who Are Our Customers?

The ultimate goal of any improvement effort is to better satisfy the needs of the customer. Thus, before formulating any improvement goals, the organisation must first assess who are their customers who are they in business to support? Thinking about customers can raise fundamental questions about the purpose and mission of the organisation. The following questions from Deming are useful to stimulate thinking about customers: Who makes the decisions about whether to buy your product or service? How do you distinguish between quality as your customer perceives it and quality as your managers and work force perceive it? How does the quality of your product, as your customer sees it, agree with the quality that you intended to give him/her? Do your customers think that your product lives up to their expectations? What do you know about the problems of your customers in the use of your products? What tests do you make of your products in service? Do you depend on complaints from customers to learn what is wrong with your product or service? Are your customers satisfied with the service that you provide? If yes, what is satisfactory about it? How do you know? Will your customers of today be your customers a year hence? Two years hence?
Page 20
Draft
Edition: 0.1
AIM/AEP/S-LEV/0007
WHO ARE OUR CUSTOMERS? Customer Products/Services Used
Edition: 0.1
Draft
Page 21
AIM/AEP/S-LEV/0007
D.2
Customer Segments
Since most organisations have many customers, it is typical for organisations to separate customers into smaller subgroups, often referred to as segments, using a variety of methods. Most organisations segment customers based on what products and/or services they use. Customer segmentation assists in defining the requirements of different groups. It also provides a basis for understanding similarities among customers. It also clarifies which customers may be impacted by improvement efforts focused on particular products or services.
CUSTOMER SEGMENTS SEGMENT CUSTOMERS
SEGMENT CUSTOMERS
SEGMENT CUSTOMERS
SEGMENT CUSTOMERS
SEGMENT CUSTOMERS
SEGMENT CUSTOMERS
Page 22
Draft
Edition: 0.1
AIM/AEP/S-LEV/0007
D.3
Customer Evaluation Form

This form should ideally be filled after customer interviews. However if interviewing customers is too costly or impossible in your schedule simply fill the form with your perception.
CUSTOMER EVALUATION FORM Date Name of the Customer Contact Person Tel Fax E-mail Postal Address <dd/mm/yyyy>
What products and services are currently provided to this customer?
What are the most important features or characteristics of the products/services provided to this customer?
What are the major delivery format and delivery methods used for this customer?
What aspects of products/services is the customer satisfied with?
What needs improving?
Additional Comments and Observations
Edition: 0.1
Draft
Page 23
AIM/AEP/S-LEV/0007
D.4
Customer Background Information

This form is to be used before customer interviews and/or evaluation.
CUSTOMER BACKGROUND INFORMATION Date Name of the Customer Contact Person <dd/mm/yyyy>
What products and/or services has this customer acquired or used from your organisation in the past?
How often does this customer acquire products/services from us?
How long has this customer been using our products/services?
How much of our budget is related to products/services for this customer?
Does this customer have any pattern in the acquisition or use of our products/services?
Does any complaint data exist to help clarify customer requirements?
Do other customer satisfaction data exist?
Does this customer refer other organizations to you? Who?
Page 24
Draft
Edition: 0.1
AIM/AEP/S-LEV/0007
APPENDIX E E.1
IMPROVEMENT FORM
Improvement Matrix
The improvement matrix is a tool for the initial and continuous analysis of the possibilities a business has when attempting to maximise their competitive position or advantage in the marketplace. Through an integrated brainstorm and analysis of all the possibilities a business has for improvement regarding elements on the "Y" axis and their relationship to the elements on the "X" axis of this matrix, a company may illustrate relationships of possibilities that may enhance their competitive advantage in the market. Possibilities for improvements / enhancements to business become apparent when the range of possibilities is clearly illustrated; how can a consumer or specialty product add value through service attachments or how can a service provide the necessary products to sustain itself or further its image for competitive image in the marketplace... and so on, throughout the possibilities that may emerge through careful analysis of market potentials.
IMPROVEMENT MATRIX Resources Commodity Industrial Product Consumer Product Speciality Product Service/Industry Service/Consumer Service/Speciality Etc. Processes/ Systems Components to be Improved Product Expansion Service of Market Etc.
Edition: 0.1
Draft
Page 25
AIM/AEP/S-LEV/0007
E.2
Improvement Plan
IMPROVEMENT PLAN Selected Component: <Name of Selected Component> Description Type of Component to be Improved Improvement Goal(s) <Describe the process improvement goal or desired change in the products or services associated with the improvement effort> Customer Impact <Identify the desired customer impact of the improvement effort> Products and Services Effected <Describe the products/services that will be affected by improvement activities> Resources <Give required resources in order to achieve the improvement (human, training, financial, etc.)> Reporting Requirements <Clarify the improvement team s reporting requirements: to whom do they report, how frequently and in what format> Suggested Timeline <Briefly describe schedule for the improvement> Existing KPIs Name Date: <dd/mm/yyyy>
<Briefly describe the component chosen for improvement. Use one form for each component. Note the reasons for selecting this component> o System o Process o Product o Service o Resource o Expansion of Market
Description
KPIs to Develop Name
Description
Page 26
Draft
Edition: 0.1
AIM/AEP/S-LEV/0007
E.3
Improvement Selection Checklist

This form may be used for each component that is considered for improvement.
IMPROVEMENT SELECTION CHECKLIST Component : <Component Name> State problems of expectations identified for this component 1 2 3 4 Date: <dd/mm/yyyy>
Place a checkmark next to all of the items that apply to your process. o o o o o o o o o The component and its associated improvement effort can be defined. (Be careful not to pick something too big. It should be possible to complete the improvement effort within 90 days.) A problem in the component occurs frequently. (A Pareto analysis may be helpful.) The problem area is well known and it is visible by the customer (external) or in the office (internal). Improvement of this component is important to the customer or the business. People will appreciate it if the component is improved. There is a good chance of success in improving the component. No one else is currently working on this component. Required changes can be put into effect with little or no outside help This is truly an improvement effort, not just an attempt to impose a solution on a problem
Note: If you have selected an appropriate component, you should be able to check all of the items above.
Edition: 0.1
Draft
Page 27
AIM/AEP/S-LEV/0007
APPENDIX F
DATA COLLECTION PLAN
The Data Collection Plan summarises the important information about this critical task on to one form. Complete the form for each KPI for which data will be collected. It is likely that this form will be used many times.
DATA COLLECTION PLAN KPI Name: <Name of KPI> What kind of data is to be collected? <Describe the type of data that will be collected> How will the data be collected? <Specify how the data will be collected. Systematic procedures concerning the construction of a recording form, consistent use of the recording form, and data sampling must be formalized and described here> When will the data be collected? <Describe the appropriate time interval to be used in data collection in this section. This interval will be determined by the cycle time of the process> Where will the data be collected? <Describe where the data will be collected. All measurements must occur in the same location> Who will collect the data? < Record the names of the individuals who will carry out the specific data collection task> Date: dd/mm/yyyy
Page 28
Draft
Edition: 0.1

KPI Deployment Guide

Uploaded by

Copyright:

Available Formats

KPI Deployment Guide

Uploaded by

Document Information

Copyright

Available Formats

Share this document

Share or Embed Document

Sharing Options

Did you find this document useful?

Is this content inappropriate?

Copyright:

Available Formats

KPI Deployment Guide

Uploaded by

Copyright:

Available Formats

EUROPEAN ORGANISATION FOR THE SAFETY OF AIR NAVIGATION