Needs Assessment Guide

Assessing Vocational-Technical Training Needs in the Eastern Caribbean Region
A Practitioner s Guide
Februar y 2002
Assessing Vocational-Technical Training Needs in the Eastern Caribbean Region A Practitioner s Guide
This Guide was prepared by Dixie Sommers, Director of Labor Market Studies, Center on Education and Training for Employment, College of Education, The Ohio State University, Columbus, Ohio, USA. This project was supported by GTZ.
Page 2
Assessing Vocational-Technical Training Needs
CONTENTS
INTRODUCTION
Why Do Needs Assessment? What is Needs Assessment? Needs Assessment Within a Company
1
1 1 2
OVERVIEW OF THE NEEDS ASSESSMENT PROCESS PHASE 1. LAYING THE FOUNDATION

Laying the Foundation: Step by Step
5 7
7
PHASE 2. ESTABLISHING THE POLICY FRAMEWORK

Establishing the Policy Framework: Step by Step
11
12
PHASE 3. ENGAGING KEY STAKEHOLDERS

Engaging Key Stakeholders: Step by Step
15
15
PHASE 4. REVIEWING THE ECONOMY

Purpose of the Economic Review Content of Economic Review Reviewing the Economy: Step by Step
23
23 23 25
PHASE 5. SELECTING TRAINING NEEDS INDICATORS

A Note about Industries and Occupations Classification Systems Relating Training Programs and Occupations Occupations and Skills Employment Forecasts Types of Training Needs Indicators Different Indicators for Different Decision Drivers Specific Indicators Selecting Training Needs Indicators: Step by Step
27
27 27 28 30 31 31 32 34 41
PHASE 6. GATHERING ADDITIONAL INFORMATION

Administrative Data Surveys Group Processes Gathering Additional Information through Surveys: Step by Step Gathering Additional Information through Group Processes: Step by Step
47
47 48 52 53 58
PHASE 7. ANALYSIS AND PRESENTATION

Analysis and Presentation: Step by Step
61
61
PHASE 8. PUTTING NEEDS ASSESSMENT RESULTS TO WORK 63

Putting Needs Assessment Results to Work: Step by Step 63
TRAINING NEEDS ASSESSMENT WITHIN A COMPANY

Reasons for Providing Company Training Three Levels of Company Needs Assessment Resources for Company Needs Assessment
65
65 66 67
REFERENCES ENDNOTES
69 73
Page 4
INTRODUCTION
This Guide is designed to assist countries in the Eastern Caribbean region in assessing their needs for vocational-technical training. Conducting needs assessment is part of the overall strategy of these countries for investing in vocational-technical training to provide the human resources needed for economic growth and improving the income of their citizens. The Guide is oriented to support decision-making in investments in occupational training. Thus, vocational education and training as discussed here includes education and training programs that provide trainees with skills and knowledge required for employment in specific occupations or a range of related occupations. Vocational education and training is distinguished here from education and training that is focused on imparting basic skills, such as reading comprehension and basic mathematics. While basic skills may be developed and improved during vocational education and training, basic skill development is not the primary purpose.
Why Do Needs Assessment?

Well-designed and competently delivered vocational-technical programs serve important purposes in any economy. They provide individuals with the skills and knowledge needed to succeed in the labor market and help meet employers needs for trained workers essential to business survival and expansion. By increasing the human capital available, appropriate vocational-technical programs also contribute to the overall development of the economy, growth in income, and the standard of living. Vocational-technical training can be expensive. Its development also takes time and leadership effort. Needs assessment is a very practical tool for achieving the purposes of vocational-technical training and using financial and leadership resources efficiently. Needs assessment helps ensure that training is related to the opportunities in the economy and is not wasted on training workers for occupational fields where jobs will not be available or that will not lead to economic self-sufficiency and economic growth. Needs assessment helps ensure that the individuals emerging from vocational-technical programs will have the skills and knowledge employers are seeking in their workers. Needs assessment results are used by those who make decisions about the specific types of training programs that will be offered and how training resources should be invested.
What is Needs Assessment?

Needs assessment is a defined process for identifying the type of training programs that should be offered to fulfill the purpose of vocational-technical training. The needs assessment should be based on a clear understanding of the role vocational-technical training is to play in the nations economy and educational system, and policy decisions on what are the driving forces for making program decisions. Decisions may be based on
Page 1
meeting student demand, meeting short-term employer needs, or meeting longer-term economic goals. Needs assessment may include identifying occupations that are in demand in the labor market, that are appropriate for vocational-technical training, and that meet other criteria. The needs assessment may provide additional information about the characteristics of these occupations that is useful in developing training programs, such as employer hiring practices and entry-level requirements. Using this and other information, decisionmakers can make informed choices about what training programs should be expanded or changed to meet the expected future demand. The results of the needs assessment will be an important part of the information used in decision-making about training, but it will not be the only information used. In market economies, it is also important for training decisions to be driven by student interest, and for student interest to be informed by clear, accurate, and accessible information about career opportunities. Needs assessment will not answer every question, however: ! Generally, needs assessment is not a substitute for a good labor market information system that addresses all segments of the economy. Needs assessments usually are focused on specific industry or occupation areas and are not designed to provide employment statistics on a regular production schedule. Ideally, a good labor market information system makes the needs assessment process easier, faster, cheaper, and more effective. Needs assessment, at least as described in this Guide, will not provide information on the specific skills and knowledges required in the occupation, information that is essential to develop the training curriculum. Additional activities are needed, particularly job analysis for the occupations identified during the needs assessment, using methods such as DACUM. Needs assessment does not provide information about specific job openings for use in job search and job placement activities. Needs assessment is not designed to provide career information for use by young people and their parents, or by adults, in making decisions about education, training, and careers. Needs assessment results can supplement other career information, however.
! !
In addition, other factors beside the results of the needs assessment will probably be taken into consideration in making vocational education and training program decisions. These may include the availability of teachers, facilities and equipment, funding resources, and a variety of other issues.
Needs Assessment Within a Company

This Guide discusses needs assessment from an economy-wide perspective. Training needs assessment may also take place within a company or organization. These assessments are conducted to help the company focus its training efforts and may address training needs to solve performance problems, training for newly hired employees, or
Page 2
training to develop employees for advancement. Within-company needs assessments identify the required level and type of skills or knowledges, assess employee actual or expected (in the case of newly hired workers) skills and knowledges, and analyze the gap between the two sets of information. This gap is the training need. Needs assessment is followed by development, delivery, and evaluation of the training. While this Guide is not designed for primarily for assessments within companies or organizations, some of the same principles apply. Needs assessments within a company are discussed briefly in the final section of the Guide, and information on additional resources is provided.
Page 3
Page 4
OVERVIEW OF THE NEEDS ASSESSMENT PROCESS

Conducting a successful needs assessment involves eight broad phases: 1. 2. Laying the foundation by gaining authority and support for the project, including funding and staff resources. Establishing the policy framework for vocational-technical training by defining the goals and scope of the vocational-technical training system and its role in the nations economy and educational system. This includes how vocationaltechnical training relates to the countrys economic development strategy. Engaging key stakeholders in the needs assessment process. Reviewing the economy by using existing economic and labor market data to identify what is already known about industry and occupational employment trends and other factors. Selecting training needs indicators for use in decision-making about vocationaltechnical programs, and determining whether the information is currently available. Gathering additional information, including determining the method for obtaining the information and actually collecting the information collection. This phase includes discussion with employers as well as gathering data. Analysis and presentation of the needs assessment results and recommendations to stakeholders and decision-makers for their use in deciding what types of training should be offered. Once these decisions are made, other activities (not part of the needs assessment) may take place, such as job analysis to identify the skills required and curriculum development. Putting the needs assessment results to work by developing and updating program curriculum, monitoring the labor market, providing career information, and collecting and analyzing program information.
3. 4.
5.
6.
7.
8.
While these phases are generally sequential, some of them can be conducted at the same time. For example, Phase 4, Reviewing the Economy, can be done at the same time as other early phases in the process. Each of these phases is described in detail in the following sections of the Guide, including step-by-step procedures that can be adapted to local conditions.
Page 5
Page 6
PHASE 1. LAYING THE FOUNDATION

Before any needs assessment activity begins, it is important to lay the foundation for the overall effort. These initial efforts will ensure that the needs assessment has the appropriate support from its sponsoring organization, that expectations are clear, and that resources are available. At this stage, a communication strategy should also be considered.

Step 1. Review this Guide.
To attain an overall understanding of what the needs assessment requires, you should read this entire Guide at the outset of the project. Step 2. Solidify the Support of the Sponsor.
1. 2. 3. 4. 5. 6.
Be sure the organization that is sponsoring the needs assessment referred to throughout the Guide as the Sponsor is firmly behind the project. The Sponsor may be a government ministry, a business organization, an educational institution, or some other organization. Regardless of the type of organization, the Sponsor must be committed to supporting the project and should have an interest in the use of the needs assessment results in making decisions about vocational-technical training. Part of solidifying Sponsor support is to clarify expectations. The Sponsor should have a clear and realistic understanding of: ! ! ! ! ! What the needs assessment will provide, What the needs assessment will not provide, How long it is likely to take, Approximate cost of the project, and How funding will be obtained.
Review the Guide. Solidify Sponsor Support. Develop the Project Plan. Prepare a Project Budget. Identify Project Staff. Develop a Communication Strategy.
Step 3. Develop the Project Plan. Prepare a written, broad plan for the overall project, including a general description of the needs assessment process to be used, who will be involved, and an approximate schedule. This plan should be updated at each phase of the needs assessment with additional detail as decisions are made.
Page 7
Step 4.
Prepare a Project Budget.
Although exact costs cannot be identified at the outset, prepare an estimate of the cost of the needs assessment. The sources or potential sources of funding should be identified. The cost estimate should include the following types of costs: ! ! ! ! ! ! ! Staff, Stakeholder Group support, Conducting the review of the economy, Information collection activities, Analysis activities, Communications activities, and Dissemination of the needs assessment results, including printing or web dissemination
The budget should identify actual or potential sources of funding and other resources. These may include the Sponsor, and private as well as public sources. TIPS: The resources provided to the project do not always have to be financial. For example, organizations such as universities, statistical agencies, or key businesses such as banks, may have the expertise to perform the review of the economy. If they understand the purpose and benefits of the needs assessment, they may be willing to perform the analysis as a nonfinancial contribution to the project. Be sure the Sponsor agrees to the general range of costs, as well as the strategy for obtaining funding and other nonfinancial support. Step 5. Identify Project Staff.
The needs assessment project will require staff support, including the following types of individuals: ! Project Leader. A single individual should be identified as the Project Leader or Project Director. This person will have overall responsibility for the planning and implementation of the project, maintaining communication with the Sponsor, organizing and supporting the Stakeholder Group, and supervising or working with staff and contractors. This individual should have the support of the Sponsor, as well as the following skills and knowledge: o Good communication skills,
Page 8
o Understanding of the countrys vocational-technical education system, o Understanding of the roles of key stakeholders (ideally, the individual will be known by leaders in some of these groups), and o Project planning and implementation experience. ! Analyst. One or more individuals will be needed to gather and analyze information, and prepare the project report. In small-scale projects, the Project Leader may do some or all of these analysis tasks. The Analyst should have the following skills and knowledge: o Ability to identify and evaluate information sources, including communication with information providers, o Ability to organize and assemble information, o Ability to use basic information analysis techniques and, if appropriate, analytical software, o Knowledge of basic statistics and survey principles, o Knowledge of occupational classification principles, and o Good communication skills, especially written communication. ! Information collection staff. These individuals will be responsible for designing and conducting information collection activities, which may include employer surveys or qualitative methods. The specific skills and knowledge required will depend on the information collection method used. Administrative support staff. This person will provide clerical and other support to the project, as directed by the Project Leader.
Step 6. Develop a Communication Strategy. The communication strategy should identify who needs to know about the project, and the purpose of communicating with them. For example, is the communication to make them aware of the project, or ask for their involvement in some way? The plan should also identify specific communication activities that will be conducted. Audiences for the communication activities might include: ! Key employer groups. Employers will likely be called upon to respond to information collection requests and are a key customer of vocational-technical training. The understanding and support of the project by key employer groups can help gain the cooperation of all employers in the information collection activities and acceptance of decisions based on the needs assessment results. Key education groups and leaders. Education leaders will be impacted by decisions based on the results of the needs assessment and therefore should be made aware of the project. Political leaders. These leaders will be called upon to act on funding and other aspects of the vocational-training system based on the results of the needs Page 9
assessment. They should be aware that a needs assessment is being conducted that will help guide these decisions. ! ! ! The economics statistics agency. This agency will be called upon for information and perhaps to provide the review of the economy. Other stakeholders. Other groups and individuals may be identified who need to know about the project. General public. Information about the project, especially about its results, should be made available to the public.
Examples of communication activities include: ! ! Meetings with and/or letters to the most important stakeholders. Developing a brochure explaining the purpose of the needs assessment project and who is involved. The brochure could be targeted to a specific audience, for example, employers. Distribution of the project report or results to targeted audiences. Press stories about the project and the results. Providing information on the web, such as a project description, questions and answers, and a contact for more information.
! ! !
TIP: It is important to have the Sponsors support for the communication strategy and approval of any communication materials that include the Sponsors name. The Stakeholder Group could be involved in developing the communication strategy.
Page 10
PHASE 2. ESTABLISHING THE POLICY FRAMEWORK

In this phase, the policy context for the needs assessment is established. The broad goals and scope of vocational education and training are reviewed, and the role of vocational education and training in the educational system and its relationship to other parts of the system are considered. The relationship between vocational education and training and the countrys economic development strategy is also addressed. Vocational education and training can be designed to serve a number of different broad goals, usually set by national policy and often defined according to who has been identified as the primary client or driver of the system. Vocational education and training may be designed to respond to several types of drivers1: ! Meeting Student Demand for Training. The vocational education and training system may prioritize responding to the choices of individual students. National policies may promote individual choice through enrollment and funding arrangements. Needs assessment in a student-choice driven system will focus on anticipating student demand and providing capacity to respond as demand changes over time, including adding or dropping courses and programs. Student choice should also be supported through career information and counseling services. Gasskov notes that in wealthier countries with student-demand driven systems, pre-employment training is often seen as one of the avenues for the broad socialization of young people rather than a way of catering to labour markets.2 Meeting Short-term Labor Market Demand for Trained Workers. In this approach, training is geared to respond to current signals of demand for particular skills and requests from employers for particular types of training. This demand-driven system will be responsive to immediate and particular needs and may have immediate payoff in terms of employment of trainees and productivity of enterprises. Care must be taken in interpreting labor market signals, however, so that demand for skills and skill shortages can be distinguished from unmet demands resulting from low wages or poor working conditions. Also, unless short-term and long-term needs are similar, short-term demand driven training may not be responsive to national strategic human resource development and economic development needs. Meeting Long-term Strategic Goals for Human Resource Development and/or Economic Development. In this approach, long-term trends in demand for human resources are examined and priorities for programs are established to meet these long-term needs. Long-term strategic planning is often aimed at supporting national economic development strategies and human resource development goals and developing vocational education and training capacity and responsiveness. Long-term planning is based on economic and demographic forecasts, as well as trends in student demand and the labor
Page 11
market. It should be noted that forecasting the future may be inaccurate, and decisions need to be reviewed regularly to take into account any fundamental changes in trends. The needs assessment process described in this Guide addresses these three goals. If other goals have been adopted, decisions will be needed about how the process should be changed as well.
Establishing the Policy Framework: Step by Step

Step 1. Review Vocational Education and Training Goals. It is critical that the goals be clearly established at the outset of the needs assessment process, as different goals require use of different assessment information and methods.
Establishing the Policy Framework: Step by Step 1. Review vocational-technical training goals. 2. Determine scope. 3. Determine target geographic area. 4. Review economic development strategy. 5. Review education and training capacity.
Which of the three drivers of vocationaltechnical training are important for making program decisions: meeting student demand for training, meeting short-term labor market demand for trained workers, or meeting long-term strategic goals for human resource and/or economic development? Are there other policy goals that drive program choices? Step 2. Determine the Scope of Vocational Education and Training.
For purposes of the needs assessment, the scope of vocational education and training needs to be defined. The needs assessment should focus on the level and type of training for which investment decisions will be made. Generally the scope should exclude shortterm training, which usually is inexpensive and does not impart extensive skills, and thus is not a significant investment target for publicly supported training. The scope also will likely exclude education leading to bachelors level or higher degrees. The planning needs for higher education are probably being addressed elsewhere in the education system. Beyond these general guidelines, the decision on scope must be made based on policy considerations and the particular structure of education in the country. Step 3. Determine the Target Geographic Area.
Determine the geographic area for the needs assessment. The area may be the country as a whole, some region within the country, or perhaps a local educational district. In making this decision, keep in mind that the labor market served by the educational system probably does not conform to political boundaries. Individuals cross boundaries
Page 12
to seek work and often live in jurisdictions different from where they work. A useful concept to keep in mind is that a labor market area is a geographical area within which individuals seek work and employers find workers. In a practical sense, it is an area within which most workers live within commuting distance to their jobs. Step 4. Review the Countrys Economic Development Strategy. Decision-making about vocational-technical training will take place within the context of the countrys overall economic development strategy. The economic development strategy will likely identify labor market issues and include priorities or initiatives about the labor market or education. The strategy also may target certain sectors or industries for development efforts. The economic development strategy should be reviewed to identify issues and strategies that are relevant for vocational training needs. These may include strategies related to increasing income and strategies related to developing the education system. Step 5. Review Vocational Education and Training Capacity. The policy framework should include a review of the countrys education system and the role of vocational-technical education within the system. The structure and capacity of vocational education and training should be reviewed. How is the system organized? How many students can it accommodate with its current facilities and teaching staff? What programs does it currently offer and how many students are served by each program? What are the characteristics of the students, such as their age and gender? In what geographic areas are the programs offered? A written description of the systems structure and capacity should be prepared for use by the Stakeholder Group (See Phase 3).
Page 13
Page 14
PHASE 3. ENGAGING KEY STAKEHOLDERS

Probably the single most important step in conducting a successful needs assessment is to obtain the support and involvement of key stakeholders. These stakeholders are individuals and organizations who have a direct interest in successful vocationaltechnical training. They also include those who will use the needs assessment results in making decisions about what training will be offered, the level of resources to be provided, and how the resources will be allocated. Businesses, as employers of vocational-technical program students, are major stakeholders. Their involvement and cooperation are critical to the success of the needs assessment process. Businesses and officials responsible for economic development also have an interest in vocational-technical training because of its contribution toward human capital and increasing income levels. Those who finance and operate the training system are also important stakeholders, and their understanding and acceptance of the needs assessment results are critical. Organizations that represent the individuals who receive training also have a stake. These may be labor organizations (unions), community organizations, and representatives of students (for adult programs) and parents (for secondary programs). These stakeholders must be brought into the needs assessment process at the beginning through the creation of a Stakeholder Group. The role of this group must be carefully and explicitly defined at the outset of the needs assessment project. The Stakeholder Group may serve as advisors, providing suggestions and recommendations on the needs assessment process, or may have a more directive role in planning and managing the needs assessment. The appropriate role may depend on the amount of time and commitment stakeholders are willing to provide and the level of involvement needed to gain their acceptance of the needs assessment results. Stakeholders should have a continuing role after the initial needs assessment is completed, although the formal Stakeholder Group may be dissolved. Employers cooperation is necessary for the next step of identifying the skill and knowledge that are required in the demand occupations identified by the needs assessment (see Phase 8). The continuing support of all stakeholders will help generate the sustained effort needed to implement change.

Step 1. Define the Role of the Stakeholder Group.
Determine the answers to the following questions: ! What role should the Stakeholder Group play in the needs assessment project? Will they provide advice, or will they direct and oversee the project?
Page 15
What activities will the Stakeholder Group perform? Potential activities include: o Conducting outreach to gain employer participation in data collection, o Making recommendations based on needs assessment results, and o Communication with the public and others about the project.
1. Define Stakeholder Group role. 2. Identify Stakeholders. 3. Solicit members. 4. Arrange for care and feeding of the Group. 5. Convene Stakeholder Group. 6. Say Thank You.
What role, if any, will the Stakeholder Group have in making decisions about vocational-technical program offerings?
Write a brief statement of the expected role of the Stakeholder Group in the needs assessment process. This statement should be used during the creation of the group, so potential members are aware of what they are being asked to do. The statement also should be reviewed by the Stakeholder Group at the beginning of its work. If the group itself is to help decide its role, its initial activity should be to develop a revised or new statement of their role. TIPS: Make sure the role of the Stakeholder Group is meaningful! You will be asking Stakeholder Group members for commitment of their time and energy, and they will want their effort to make a difference. If Stakeholder Group members do not feel they have a meaningful role, they may drift away from the effort or attack the results. Give the Stakeholder Group a formal name. This will help the group members identify with the project and make the project visible to employers, educators, and the public. Visibility is part of gaining acceptance of the needs assessment results. Do not expect the Stakeholder Group members to do the work of performing the needs assessment. This is the job of the staff. Step 2. A. ! ! ! ! ! Page 16 Identify the Stakeholders. Identify key organizations that should be represented, such as the following: Businesses: Chambers of Commerce, business associations, key employers in the economy, Education leaders: national ministry level, institution level, Labor organizations: unions, Economic development: those responsible for implementing national strategy, Community organizations, and Assessing Vocational-Technical Training Needs
Parent or student organizations.
B. Determine the desired level of the individual representatives on the Stakeholder Group, such as: ! ! The level of business representatives, for example, chief executives, human resource directors, plant managers, supervisors, and The level of education leaders, for example, rank within national ministry, school superintendent, or teacher.
TIP: All Stakeholder Group members do not have to be at the same level. A mix of levels is appropriate, such as including chief business executives and teachers on the same group, as long as custom and the way the group conducts its business encourage participation by all members. It is best not include superiors and subordinates from the same organization, however, as this may inhibit discussion. C. Determine if any existing organizations can serve as the Stakeholder Group, for example, a national training board that includes all of the identified stakeholders. ! ! If existing groups are identified, determine whether and how to approach them about assuming the role of the Stakeholder Group. If the decision is not to approach existing groups, determine what communication with the existing groups about the creation of the Stakeholder Group is appropriate. The support of these existing groups may be important when decisions are made based on the needs assessment results. Solicit Individuals to Join the Stakeholder Group.
Step 3.
A. Be sure the group represents all important segments of the economy/population, such as: ! ! ! ! Different geographic areas, Different ethnic or race groups, Female and male members, and Major industries in the countrys economy.
B. Decide on the number of members desired. Keep the group to a workable size, with at least 10 members but ideally no more than 15 to 20.
Page 17
C. ! ! !
Issue invitations to prospective members. Choose the way of issuing the invitation that best fits your culture and political environment, for example, formal invitation or an informal invitation. Decide on who should issue the invitation. Should it be from a high-level official? From a business leader? Be sure the invitation is clear on who the Sponsor is, the expected role of the Stakeholder Group, the commitment of time that is expected, and how long the group is expected to be active. Indicate whether expenses for participation will be paid for.
! TIP:
If you are seeking the participation of business chief executives and other highlevel individuals, the invitation should come from a high-ranking source. Step 4. Arrange for the care and feeding of the Stakeholder Group. Effective Stakeholder Groups need support, leadership, and attention to meeting logistics. ! ! Staffing. Be sure the group members know who will provide staff support and what the staff will do for the group. Group leadership. Determine how the group will be led. Will there be a chairperson? If so, will the chairperson be selected by the group or by the Sponsor? Group leadership and decision-making processes may depend on the custom in your country. Meeting planning. Staff and the group leadership should work together on setting schedules for the groups meetings and other activities, establishing meeting agendas, identifying materials needed either prior to or during meetings, and deciding how meeting activities will be recorded (for example, meeting notes or minutes). Meeting logistics. Staff should ensure that appropriate meeting space is provided, along with any equipment such as audio/visual equipment, and that any refreshments or meals are provided as needed. Staff should also arrange for any other meeting resources, for example, outside presenters or facility tours. Public recognition. The Sponsor, staff, and group leadership should decide whether public announcements of the creation of the Stakeholder Group are appropriate and how to communicate with political leadership, the public, the business and education community, and others on the role and activities of the Stakeholder Group. If there is to be a group chairperson, be sure the Stakeholder Group members know how the chairperson is selected and what the chairpersons role will be.
TIP:
Page 18
Step 5. Convene the Stakeholder Group. Once the Stakeholder Group is established, it should begin its work promptly. The Stakeholder Groups work will involve several phases: getting started, beginning the work, conducting the needs assessment, and wrapping up. A. Getting Started.
The following activities will develop common understanding among group members of their role and purpose and provide them with a shared foundation of information important to the needs assessment process. These activities should be carried out at the beginning of the Stakeholder Groups work: ! Introductions. At the first meeting, begin with introductions so all members and the staff are identified. Provide a list of group members and their organizational affiliation and names and contact information of staff. Name tags or name tents on the meeting table can also help members get acquainted. Role clarification. At the first meeting, review and discuss the written statement of the Stakeholder Groups role and responsibilities. If the group itself is to help decide its role, its initial activity should be to develop a revised or new statement of their role. Revisit the role statement as necessary throughout the life of the Stakeholder Group. Leadership. If the group is to select its own chairperson, selection should be done at the first meeting. Ground rules. Depending on the custom in your country, ground rules for how the Stakeholder Group will conduct its business may be discussed and put in writing. This will help avoid conflict later on. Discuss what needs assessment is, its purpose, and how the results will be used in making decisions about vocational-technical training programs. Be sure these points are clear and that all group members understand them. Introduce the staff who will provide Stakeholder Group support. Provide information Stakeholder Group members need about logistics, such as plans for meeting locations and how to get paid for their expenses (if payment is being provided). Review the needs assessment process. Clarify any questions about the purpose, scope, and timing of the needs assessment project. Identify the points on which Stakeholder Group input, advice, recommendations, and possibly decisions (depending on the role of the group) are to be offered.
! !
! ! !
Page 19
Review the definition of vocational education and training and provide information on the current organization and scope of vocational-technical training.
TIP: Ask a representative of the Sponsor to attend the opening of the first meeting. If the Sponsor has selected the chairperson or other group leadership, the Sponsor should introduce the chairperson/leader(s). The Sponsor can also express appreciation for Stakeholder Group members participation and discuss Sponsor expectations of the Group. B. Beginning the Work.
The following activities start the real work of the Stakeholder Group, and should begin quickly after the start-up activities are completed. This stage of the work should result in decisions about how the needs assessment project will be carried out: ! ! ! ! ! Present the Economic Review and discuss its implications for vocationaltechnical training. (See Phase 4.) Present economic development strategy and discuss its implications for vocational-technical training. Provide advice or decisions on the scope of the needs assessment, including discussion on how the information should be used in decision-making. Provide advice or decisions on how the needs assessment will be conducted. Determine how to encourage employer participation in information collection and discussion to assure representative results. Conducting the needs assessment.
C.
Once decisions have been made on how the needs assessment will be conducted, the assessment should begin. As the assessment is being carried out, the burden of the work is primarily on the staff. The Stakeholder Group may have the following types of activities: ! ! ! Participate in publicity or communications activities, as identified in the project plan, especially employer outreach. Receive progress reports from the staff on implementation of the needs assessment procedures. Provide guidance or assistance as the needs assessment is conducted, including how to deal with issues such as low employer participation.
Page 20
D.
Wrapping up.
The following activities should be carried out as the end of the Stakeholder Groups work nears: ! ! ! ! Receive and discuss the needs assessment results and the analysis of the results prepared by the staff. Decide on any recommendations that stem from the needs assessment results. Review and approve the final needs assessment report. Transmit the final needs assessment report to the Sponsor.
Step 6. Say Thank You. At the conclusion of the Stakeholder Groups work, it is appropriate and important to provide the members with recognition and appreciation for their effort and contributions. The form of recognition will vary depending on customs in your country but could include: ! ! ! ! Letter from the Sponsor expressing appreciation. Presentation of a certificate or other token of recognition, perhaps at a final meeting or at a public event. Including Stakeholder Group member names in the final needs assessment report. Including Stakeholder Group member names in any publicity about the results of the needs assessment.
TIP: Do not neglect this step! You want the Stakeholder Group members to end their work feeling valued. Remember, you may be asking them to participate in future projects! This step also clearly marks, in a graceful way, the end of the Stakeholder Groups existence.
Page 21
Page 22
PHASE 4. REVIEWING THE ECONOMY Purpose of the Economic Review

The Economic Review provides an understanding of what factors and trends are driving the countrys labor market, which is the essential context for developing a demand-driven vocational-technical training system. An understanding of the structure and trends of the countrys population, output, income, labor force, industries, and occupations is fundamental to identifying needs for skills. This information will also guide the decisions about industries or occupations that might be targeted during the needs assessment. To achieve this purpose, the Economic Review should be presented with the needs assessment audience in mind: the audience members are likely not economists and may not be familiar with all of the relevant economic concepts and data. The Economic Review report and the presentation of the Economic Review results should be brief and clear and should avoid technical jargon. Ideally, the Economic Review should be conducted by an individual who has a background in economics and knowledge of the available information sources and economic analysis methods, as well as familiarity with the countrys economy. This person should bring objectivity as well as knowledge to the task. Objectivity means that the individual will describe what is known about the various topics and will not perform the review with the intent of advocating a particular policy decision or conveying a biased view of the economic situation. Phase 4 can be conducted at the same time Phases 2 and 3 of the needs assessment are underway. Scheduling the work on the Economic Review in this way should ensure that the Economic Review is completed for presentation to the Stakeholder Group at the beginning of their work, as discussed in Phase 3, Step 5.
Content of Economic Review

A comprehensive Economic Review should cover the following topics. In many cases, information about all of these topics is not available or may not be of sufficient quality or detail to fully depict the economic conditions. In such instances, the review should cover as much as possible without distortion. A. ! Population and Labor Force Population. The Economic Review should begin with an examination of population size and trends, to provide an understanding of the general supply of workers expected in the future. If population forecasts are available, they should be examined to identify the expected size and rate of growth for the number of working age individuals, who will comprise the future labor force of the country. Within the working age population, it is important to examine the age
Page 23
structure: what portion of the population is expected to reach retirement age? What portion will be in the prime working years? What portion will be younger inexperienced workers? How many young people will be entering the educational system? ! Labor Force Size and Trends. Similar to the examination of population, the size and growth of the labor force and labor force participation rates should be described. Factors that are impacting labor force growth should be identified. Labor Force Composition. The composition of the labor force by age, gender, and other factors important in the country (race or ethnicity, for example) should be described, including any recent changes. If population forecasts are available, it may be possible to estimate the size and of the future labor force and its composition by age and gender.3 The labor force comprises the expected future labor supply. Broad Economic Conditions and Trends ! Gross Domestic Product (GDP) Size and Trends. GDP information indicates the overall level of production and growth in the countrys economy. Trends in GDP per capita or GDP per worker are particularly useful, as they indicate the returns to human capital and progress in improving the standard of living. Economic development strategies, including strategies to increase human capital through education, may be aimed at increasing GDP per capita. Income. Trends in income of individuals or households are also useful indicators of progress in improving the standard of living. GDP by Sector. Understanding the demand side of the labor market begins with understanding the industries present in the economy and trends in output of products and services. Output by major industrial sector (agriculture, manufacturing, etc.), as well as GDP per worker, indicates the contribution of various sectors to economic growth. This information will be useful in the needs assessment if decisions are needed to target training toward high growth or high output sectors. Import/Export Trends. Information on imports and exports, especially by type of product, indicates the countrys links with the external markets and may indicate the direction of growth. Employment Conditions and Trends ! Total Employment Level and Trends. Growth or decline in GDP will result in changes in the demand for workers, measured by the level and trends in total employment. The examination of employment trends should capture trends by gender and other key demographic groups. Employment and Trends by Industry Sector. Review the employment level and trends in the countrys important industry sectors. Ideally, the industry analysis
B.
! !
C.
Page 24
will include development of employment forecasts, which requires historical employment data as well as expertise in economic forecast methods. ! Employment and Trends by Occupation Group. Employment by occupation indicates the overall demand for workers with different types of skills. If data are available, the Economic Review should describe the level of and trends in employment by occupation group, including forecasts of future employment.
D. Education Conditions. Education information indicates the overall level of human capital available in the economy. The overall level of education of the population, and how the education level may be changing, should be described. Education level and trends for groups within the population, such as for different age groups and for men and women, should be shown.
Reviewing the Economy: Step by Step

Step 1. Identify and Obtain the Available Data. For each part of the Economic Reviews content, identify the available data sources, including information about definitions and concepts, and documentation of the data sources. Obtain the actual data, preferably in electronic format. Step 2. Analyze the Data. Analyze the available data to identify significant points regarding the structure and trends relevant to each part of the Economic Review (population, GDP, employment, etc.). Develop a brief descriptive narrative and charts and simple tables that depict the most important information. Step 3. Identify Implications for Vocational-Technical Training. Examples of implications might include answers to the following questions: ! ! How are population trends impacting current and future education enrollments? Does the economic development strategy target particular industries or geographic areas, for which vocational-technical training needs may be targeted during the needs assessment? What groups within the population are not fully participating in the economy because of their lack of education and training or
Reviewing the Economy: Step by Step 1. Identify and obtain available data. 2. Analyze the data. 3. Identify implications for vocational education and training. 4. Prepare Economic Review report. 5. Present the economic review.
Page 25
other reasons (for example, low labor force participation of women)? ! ! What industries are contributing the most to the countrys growth and income potential? How are occupational trends affecting the level of education and skill requirements?
Step 4. Prepare the Economic Review Report. Write a report containing the results of the review. Keep in mind that many readers of the report will not be economists or necessarily be conversant with some of the basic concepts used. Clear presentation, using non-technical language, is essential. The report should also be brief, and not require an extensive time commitment on the part of the reader. TIPS: Use charts and clear, brief tables to illustrate important points. Be sure the text also makes the same points, however, as some readers understand text more easily while others prefer graphics. Provide a brief glossary of essential terms that a general reader may not know. Write background notes about all information sources! Documentation is part of maintaining objectivity, as it lets readers know where they can find the information. Keep in mind that needs assessment is a continuing process. You may need to update the Economic Review in the future, and documenting the information sources will make it easier the next time. Be sure to mention major limitations of the data if they are likely to impact the conclusions drawn from the data about vocational-technical education. Step 5. Present the Economic Review. The Economic Review will be presented to the Stakeholder Group and perhaps to other audiences. As with the written report, the presentation should be made with the nontechnical nature of the audience in mind.
Page 26
PHASE 5. SELECTING TRAINING NEEDS INDICATORS

In this phase of the needs assessment, the specific information requirements are determined and data availability is determined. Decisions are also made about what additional data need to be gathered. Methods for gathering the additional data are discussed in Phase 6.
A Note about Industries and Occupations

The distinction between occupations and industries is fundamental to the needs assessment. Occupation refers to the nature of the work performed by individual workers. Typically, occupations are defined according to the tasks performed by the worker. Training imparts skills to individuals to prepare them to perform work in various occupations. Examples of occupations are carpenter, teacher, and hotel front desk clerk. By contrast, industry refers to the type of product or service produced by the employing organization, or the activity conducted by the organization. Examples of industries include construction, education services, and hotels. Jobs in a specific occupation may be found in many industries, or in only one or a few industries. Carpenters, for example, are employed primarily in the construction industry, but are also found in other industries such as education and hotels, where there is a need to repair and maintain buildings and other wood structures.
Classification Systems
In the needs assessment, information about occupations and industries should be based on an established classification system, preferably a system that is used in the countrys economic and statistics data systems. Classification systems usually consist of titles and definitions of specific industries or occupations, and a hierarchical coding system. For industries, the definitions may describe the types of products or services produced and perhaps the production methods used. For occupations, definitions usually describe the work being performed and may list tasks and related job titles. Use of an established classification system will eliminate the additional effort of developing occupational or industry categories for use in the needs assessment, and will allow users to refer to definitions that provide additional detail about the nature of the industries or occupations. Using recognized classification systems also allows comparison of the needs assessment results with other available data, making analysis and updates easier. If the country has not adopted a specific classification system, other existing classification systems may be used. 4 For occupations, the International Labor Organizations 1988 International Standard Classification of Occupations (ISCO-88) is the most widely adopted system. Although it is rather dated, the ISCO may still be appropriate for developing economies. More recent systems are the U.S. Standard Assessing Vocational-Technical Training Needs Page 27
Occupational Classification, and Canadas National Occupational Classification, both adopted in 2000. For industries, the United Nations Statistics Division maintains the International Standard Industrial Classification (ISIC), last updated in 1989. The United States, Canada, and Mexico have adopted the North American Industrial Classification System (NAICS), which classifies businesses based on the activities in which they are primarily engaged. In the U.S., NAICS is replacing the 1987 Standard Industrial Classification (SIC). To manage information for labor market analysis, it will also be very useful to have a classification structure for education and training programs. The United Nations Statistics Division also provides the International Standard Classification of Education (ISCED) issued in 1997. In the U.S., the 2000 Classification of Instructional Programs serves this purpose. Sources for the classifications mentioned above are listed in the References section of the Guide.
Relating Training Programs and Occupations

Labor market analysis requires that the relationships between education and training programs and occupations be identified. Programs may prepare individuals for a single occupation or, more often, several occupations. Conversely, preparation for some occupations is through a single type of program, while other occupations can be entered through any of several different types of training programs. The relationship between a program classification and an occupation classification is often referred to as a crosswalk.5 A crosswalk is a database that can be used to present the information organized either by program or occupation. For example in Figure 1, crosswalk information for nursing occupations is show, listed in occupation order. Note that for the occupation 32502, Registered Nurses, one program category is shown, 51.1601, Nursing (RN Training). This program is related to the Registered Nurse occupation at the Bachelors through Doctoral levels. For occupation 32505, Licensed Practical Nurses, four programs are listed: 51.1601, Nursing (RN Training) at the Associate levels and below; 51.1613, Practical Nurse (LPN Training); 51.1614, Pre-Nursing; and 51.1699, Other Nursing.
Figure 1. Occupation-Program Crosswalk for Nursing Occupations, listed in Occupation Order Occupation Program Code Title Code Title Level
32502 32502 32502 32502 Registered Nurses Registered Nurses Registered Nurses Registered Nurses 51.1601 Nursing (RN Training) 51.1601 Nursing (RN Training) 51.1601 Nursing (RN Training) 51.1601 Nursing (RN Training) Doctoral Post Master's Certificate Masters Post Baccalaureate Certificate
Page 28
32502 32505 32505 32505 32505 32505 32505 32505 32505 32505
Registered Nurses Licensed Practical Nurses Licensed Practical Nurses Licensed Practical Nurses Licensed Practical Nurses Licensed Practical Nurses Licensed Practical Nurses Licensed Practical Nurses Licensed Practical Nurses Licensed Practical Nurses
51.1601 Nursing (RN Training) 51.1601 Nursing (RN Training) 51.1601 Nursing (RN Training) 51.1601 Nursing (RN Training)
Bachelors Associate At least one- but less than twoyear award Less than one-year award
51.1613 Practical Nurse (LPN Training) Adult Part-Time 51.1613 Practical Nurse (LPN Training) Postsecondary Full-Time 51.1613 Practical Nurse (LPN Training) Secondary 51.1614 Pre-Nursing 51.1699 Other Nursing 51.1699 Other Nursing Secondary Adult Part-Time Postsecondary Full-Time
In Figure 2, crosswalk information is presented for nursing programs. For program 51.1601, Nursing (RN Training) is related to three occupations. At the bachelors and masters level, this program is related to occupation 15008, Medicine and Health Service Managers, which includes people who work as nursing directors. The program is also related to 32502, Registered Nurses, and 32505, Licensed Practical Nurses. Finally, at levels of less that two years of training, the program is related to occupation 66026, Pharmacy Aides.
Figure 2. Occupation-Program Crosswalk for Nursing Programs, listed in Program Order Occupation Program Code Title Code Title Level
15008 15008 32502 32502 32502 32502 32502 32505 32505 32505 66026 66026 32505 32505 32505 32505 32505 32505 Medicine and Health Services Managers Medicine and Health Services Managers Registered Nurses Registered Nurses Registered Nurses Registered Nurses Registered Nurses Licensed Practical Nurses Licensed Practical Nurses Licensed Practical Nurses Pharmacy Aides Pharmacy Aides Licensed Practical Nurses Licensed Practical Nurses Licensed Practical Nurses Licensed Practical Nurses Licensed Practical Nurses Licensed Practical Nurses 51.1601 Nursing (RN Training) 51.1601 Nursing (RN Training) 51.1601 Nursing (RN Training) 51.1601 Nursing (RN Training) 51.1601 Nursing (RN Training) 51.1601 Nursing (RN Training) 51.1601 Nursing (RN Training) 51.1601 Nursing (RN Training) 51.1601 Nursing (RN Training) 51.1601 Nursing (RN Training) 51.1601 Nursing (RN Training) 51.1601 Nursing (RN Training) Bachelors Masters Doctoral Post Master's Certificate Masters Post Baccalaureate Certificate Bachelors Associate At least one- but less than two-year award Less than one-year award At least one- but less than two-year award Less than one-year award
51.1613 Practical Nurse (LPN Training) Adult Part-Time 51.1613 Practical Nurse (LPN Training) Postsecondary Full-Time 51.1613 Practical Nurse (LPN Training) Secondary 51.1614 Pre-Nursing 51.1699 Other Nursing 51.1699 Other Nursing Secondary Adult Part-Time Postsecondary Full-Time
Page 29
Occupations and Skills

More and more attention is being paid in vocational education and training to discussing skill requirements in the labor market rather than occupations. 6 Skills information has always been critical for developing training curricula and usually has been obtained through job analysis and employer input. The new focus on skill requirements stems from a number of considerations, including recognition that occupational classifications are rather rigid and hard to keep up to date, and that even within occupations with stable definitions, the skill requirements of the work may be changing rapidly. Also, labor markets are becoming increasingly fluid and hard to predict. Employers are more often looking for workers with certain skills rather than specific occupational training or qualifications, and employment forecasts based on past trends may not be good indicators of the future. In discussing skills, it is useful to refer to a hierarchy of skills, ranging from basic or fundamental skills to specialized skills. The Occupational Information Network (O*NET) from the United States provides one such hierarchy. O*NET is built around a content model7 that includes three skill levels: ! Basic Skills- skills that facilitate learning and information acquisition. Examples include: o Content Skills - fundamental skills needed to work with or acquire more specific skills, such as reading comprehension and active listening. o Process Skills - procedures that contribute to the more rapid acquisition of knowledge and skill, such as critical thinking and active learning ! Cross-Functional Skills skills that facilitate performance of activities that occur across jobs, such as social skills and complex problem-solving skills. Examples include: o Technical Skills- skills used to design, set up, operate, and correct malfunctions involving application of machines or technological systems, such as equipment selection and testing. o Systems Skills skills used to understand, monitor and improve sociotechnical systems, such as systems perception and judgment and decisionmaking. o Resource Management Skills- skills used to allocate resources efficiently, such as management of time and financial resources. ! Occupation-Specific Skills skills for performance in specific occupations.
In needs assessment, discussions of skill requirements often are concerned with basic and cross-functional skills (or their counterparts in other skills hierarchies). 8 Information on these skill requirements is useful for ensuring that vocational education and training programs and other parts of the educational system are teaching the needed levels of Page 30 Assessing Vocational-Technical Training Needs
basic and cross-functional skills. Information on occupation-specific skills is needed for curriculum development in vocational education and training programs, however. Ways of identifying these skills are discussed in Phase 8.
Employment Forecasts
Forecasts of employment trends for industries and occupations are important training needs indicators, especially when long-term strategic goals are being addressed. Other forecasts, such population, labor force and school enrollment, are also useful. While the development of such forecasts has been underway for several decades in industrial countries, they remain a topic of some controversy. 9 Forecasts are by definition conjectures (sometimes based on very sophisticated analysis) about future trends. Forecasts therefore may be inaccurate. Use of simplistic methods or poor quality data can add to their inaccuracy. Also, forecasts may be quickly outdated by major unexpected events. Consequently, care must be exercised in interpreting forecasts, and they should be used along with other indicators in decision-making. Forecasts generally are prepared using one or more techniques: ! Economic or demographic models provide quantitative estimates of population, labor force, employment and job openings for some future date based on models of population change and economic growth. These models may be quite sophisticated or may involve simple extrapolations, and usually require extensive data resources. Group processes, especially the Delphi technique, are used to forecast future trends and provide qualitative information on the participants expectations of broad trends. See Phase 6 for a description of the Delphi technique. Employer surveys, although largely discredited as a method for producing quantitative forecasts, continue to be used. This approach is based on the seemingly obvious premise that employers know their future needs and are willing to share them. This method, however, lacks any firm theoretical foundation and does not assure that responding employers are using the same overall assumptions about economic conditions. Employers also may be unwilling to provide forecasts that reveal their plans regarding plant relocations, expansions, or contractions. When used with care, employer surveys may provide useful results in the near-term if they focus on identifying factors that are driving change.
Types of Training Needs Indicators

The specific information used in needs assessment may be thought of as a set of indicators, that is, measures that represent some aspect of the need for vocational education and training. We will focus on four types of indicators:
Page 31
Overall Labor Market Indicators: measures of the overall structure and growth of the population, the economy, and the labor force, without regard to specific occupations. Occupational Demand Indicators: measures of the need for new workers with the skills required, including basic and cross-functional skills as well as occupation-specific skills. Student Training Demand Indicators: measures of students demand for training in general and for training in specific programs. Training Supply Indicators: measures of the capacity of the training programs and educational system to meet the demand for training.
! !
Generally, it is important to use several indicators when making decisions. It is not unusual for indicators to point in different directions, so using several indicators may help clarify the situation. Also, understanding of the accuracy and definition of the data is important when interpreting the indicators. For high-stakes decisions, as many indicators for which sound information can be obtained should be used. Developing the indicators involves obtaining and analyzing various demographic, labor market, and education information. The analysis is most successful when a comprehensive and objective employment statistics system is available that gathers and disseminates data on a regular basis using sound statistical and data collection methods. While a comprehensive labor market and education statistics systems may not be available, all Eastern Caribbean countries have some basic information. Additional information will likely need to be collected during the needs assessment, as described in Phase 6.
Different Indicators for Different Decision Drivers

In Phase 2, we discussed the broad goals of vocational education and training and the three alternative drivers of decision-making: ! ! ! Meeting Student Demand for Training, Meeting Short-term Labor Market Demand, and Meeting Long-term Strategic Goals.
Because these drivers determine the nature of the planning decisions made, they also impact the type of training demand indicators that should be selected for the needs assessment. For each alternative, different types of information are needed.
Page 32
Table 1 depicts, in general, how the four types of indicators relate to the decision drivers. For example, overall labor market demand indicators are generally not relevant in decision-making related to meeting student demand, but are helpful where meeting shortterm labor market demand is the priority. Overall labor market demand indicators are critical to planning related to long-term strategic goals, however. These general relationships are discussed in more detail below. There are some exceptions to the general relationships shown in Table 1, which will be apparent as specific indicators are examined later in this chapter. Overall Labor Market Indicators. Indicators of overall economic and labor market conditions and trends are helpful when decisions are driven by short-term labor market demand for training, and critical when long-term strategic needs are the driver. In a student-demand driven environment, this information is generally not relevant, although it should underlie any career information provided to students. Student Training Demand Indicators. In a student demand-driven environment, needs assessment is a process of identifying current and anticipated student demand for training, especially demand for specific types of occupational training. The available training capacity and the extent to which existing training programs are meeting student demand are also evaluated. Ideally, the needs assessment should compare student demand with labor market demand for occupational skills provided by the training. This comparison suggests how well students training choices reflect the labor market. Significant mismatches between student choices and labor market demand can be addressed by improving the information available to students in making their decisions, such as career information, career awareness programs, and job search information. Student recruitment may also be needed for training programs that provide good labor market opportunities. Short-term Labor Market Training Demand Indicators. Short-term labor market demand for training can be assessed primarily by analyzing various occupational demand indicators, such as the number of job openings and employer hiring activity. Care should be taken to evaluate the nature of shortages or hard-to-fill vacancies, Table 1. Relating Types of Indicators to Decision Drivers Decision Drivers Short-Term Labor Type of Indicator Student Demand Market Demand Overall Labor Generally not Helpful Market Indicators relevant Occupational Helpful Critical Demand Indicators Student Demand Generally not Critical Indicators relevant Training Supply Critical Critical Indicators Assessing Vocational-Technical Training Needs
Long-Term Strategic Goals Critical Critical Generally not relevant Critical
Page 33
especially the wages and working conditions being offered. Low wages or poor working conditions, rather than lack of training, may be the cause of short-term shortages of workers. Short-term shortages may also result from the time lag between identifying training needs and the completion of training programs by students. Student training demand indicators are also useful in assessing short-term training needs. Training needs may not be met if student interest in the training programs is insufficient. Training Supply indicators should also be examined to determine whether training capacity is available or needs to be expanded to meet student demand. Long-Term Strategic Training Needs Indicators. In planning to meet long-term strategic goals, such as human resource development or economic development goals, the needs assessment is directed primarily to identifying long-term occupational demand rather than short-term, immediate employer needs. Economic development goals and policies should be explicitly recognized, especially if a policy decision was made in Phase 2 that the training system will be directed toward helping meet these development goals. Occupation demand indicators are among the data needed, especially industry and occupation employment forecasts, and estimates of job openings resulting from employment growth and from the need to replace workers who leave the labor force.
Specific Indicators
Table 2 lists 33 specific indicators, listed according to the type of indicator. The first column of this table provides a sequence number for easy reference to each indicator, and the second column shows the name of the indicator. The next three columns show the relevant decision drivers for the indicator. For example, indicator number 3, labor force trends and forecasts, is relevant to decisions about short-term labor market demand and strategic goals. The remaining columns discuss how each indicator may be used in needs assessment, provide additional comments, and note the general sources of information for the indicator. Note that at the end of Table 2, indicator 29, Number of completions by program, and indicator 31, Number of students enrolled by program, appear twice, once under Student Training Demand and once under Training Supply. These indicators can be used in both categories.
Page 34
Table 2. Specific Needs Assessment Indicators

Indicator
1 Population trends and forecasts by age, gender and race/ethnicity. # # Student Demand Short-term Strategic Demand Goals
Use in Needs Assessment

Indicates overall trends in the number of individuals likely to be seeking training.
Comments
Forecasts are subject to error. Error on population forecasts is generally less than on employment forecasts, except where migration factors are significant, volatile, or not well measured. Gender and race/ethnicity information is important where equity issues are identified for policy response. These factors help drive employment growth by industry.
Information Sources
Census or household surveys, forecasts produced through analysis and demographic modeling.
Overall Labor Market Indicators
Trends in Gross Domestic Product and imports/exports by sector Labor force trends and forecasts #
Indicators of overall economic growth and structure of the economy. Employment forecasting models may require these data. Indicates overall labor supply.
Forecasts are subject to error. Similar to population forecasts, labor force forecasts are generally have less error than employment forecasts. Gender and race/ethnicity information is important where equity issues are identified for policy response.
National income accounts maintained by national statistics agency or bank. Household surveys, forecasts using population forecasts and labor force participation trends by age and gender.
Unemployment rates by age and gender
Unemployment rates by education level Educational attainment of the population by age and gender
School enrollment trends and forecasts by education level and type of school
Indicates current demand for target population groups. Unemployment rates for young persons indicate overall demand for new workers. Indicates extent of labor market demand for individuals with various education levels. Indicates patterns of overall demand for various types and levels of education. Useful for anticipating overall student demand for vocational education and training. Indicates patterns of overall demand for various types and levels of education. Useful for anticipating overall student demand for vocational education and training.
Household surveys
Household surveys
Gender and race/ethnicity information is important where equity issues are identified for policy response.
Census or household surveys
Gender and race/ethnicity information is important where equity issues are identified for policy response. Forecasts are subject to error.
Education data systems.
Page 35

Indicator
8 Current employment and employment trends by industry Current wages and wage trends by industry # Wage information must be interpreted in light of whether barriers to entry and lack of wage competition exist. Includes, for example, new plant construction or expansion, major new construction projects, layoffs or plant closings. This type of information may be anecdotal and should be interpreted with care. Forecasts are subject to error. Use of forecasts should be accompanied by monitoring economic trends to identify unanticipated major shifts. Useful in defining labor market areas. Useful in understanding how the labor supply may be affected by transportation networks, growth or decline in neighboring areas, and other factors. Student Demand Short-term Strategic Demand Goals

Indicates overall labor demand trends in the industry, which drive demand for occupations important in the industry. Indicates current demand in the industry, which drives demand for occupations important in the industry.
Comments
Information Sources
Employer surveys or administrative records.
Overall Labor Market Indicators, Continued

#
Wage information is critical to interpreting vacancy and employer-reported skill shortage information.
Employer surveys or administrative records.
10
Information on industry investment, expansion, and contraction decisions or plans. Employment forecasts by industry
Indicates expected short-term change in demand, e.g., increased demand for new workers or changes in skill requirements in current occupations. Indicates overall labor demand trends which drive demand by occupation. Industry employment growth is critical in preparing occupational employment forecasts. Indicates the source of labor supply for a geographic area.
Employers, news media, economic development agencies, employment services. Forecasting using analysis and modeling.
11
12
Commuting patterns # #
Census
Occupational Demand Indicators

13 Current employment by occupation # # Indicates current demand. In Student Choice, useful for comparing student choices with current employer demand. May be used to prioritize programs with similar student demand indicators. Indicates current demand. In Student Choice, useful for comparing student choices with current employer demand. May be used to prioritize programs with similar student demand indicators. Employer or household surveys
14
Current job vacancies by occupation #
Vacancy information must be interpreted in light of wages and other factors. Vacancies may reflect low wages and/or poor working conditions rather than training needs.
Employer surveys, employment services.
Page 36

Indicator
15 Time required to fill job vacancies by occupation New hires by occupation Current wages and wage trends by occupation Student Demand Short-term Strategic Demand Goals

Indicates current demand.
Comments
Vacancy information must be interpreted in light of wages and other factors. Vacancies may reflect low wages and/or poor working conditions rather than training needs.
Information Sources
Employer surveys or discussions, employment services. Employer surveys, administrative records, employment services. Employer or household surveys.
Occupational Demand Indicators, Continued

# Indicates current demand. # Indicates income potential of the occupation. Useful in long-term strategic planning when economic development and income goals are important. # Wage change over time also indicates demand. May be used to prioritize programs with similar student demand indicators. Indicates current demand for general educational qualifications and for occupational skills Indicates expected demand for new workers in the occupation. # Forecasts are subject to error. Use of forecasts should be accompanied by monitoring economic trends to identify unanticipated major shifts. Essential if equity issues are to be addressed in training program policy, such as equity for women in training and employment. Hiring practices information also useful for program development (designing programs to prepare students to meet hiring requirements other than occupation specific skill requirements) and job placement activities. Wage information is critical to interpreting vacancy and employer-reported skill shortage information. Wage information must be interpreted in light of whether barriers to entry and lack of wage competition exist. Income potential of occupations is important where economic development goals are a priority. Household surveys
16
17
18
Unemployment rates by occupation Forecast of job openings by occupation.
19
Includes job openings resulting from employment growth (new jobs) and from replacement of workers who die or leave the labor force (replacement needs).
Prepared from analysis of vital statistics, labor force participation trends, occupational employment by age and gender, and occupational employment forecasts. Census or household surveys; employer surveys. Employer surveys or discussions, employment services.
20
21
Gender and race/ethnicity of current workers in the occupation Employer hiring practices by occupation #
When used with occupational employment and forecasts, indicates trends in training needs by gender and race/ethnicity. Indicates how employers meet their demand for skills by identifying the training possessed by hired workers.
Page 37

Indicator
22 Employer-reported skill shortages and training needs Student Demand Short-term Strategic Demand Goals

Indicates current skill demand.
Comments
As with current vacancy information, skill shortage reports must be interpreted in light of wages and other factors. Also, employers may have difficulty identifying or expressing skill shortages. Forecasts are subject to error. Use of forecasts should be accompanied by monitoring economic trends to identify unanticipated major shifts.
Information Sources
Employer surveys or discussions, employment services.
Occupational Demand Indicators, Continued

#
23
Employment forecasts by occupation #
Indicates future demand in the occupations related to training. Useful for comparing student choices with current employer demand. May be used to prioritize programs with similar student demand indicators. Indicates education requirements in the occupation.
Forecasts produced using analysis and forecasting models.
24
Education level by occupation # #
25
Employer satisfaction with existing programs Tracer studies: followup studies of employment outcomes and wages by occupation of vocational education and training graduates
26
Indicates potential employer interest in hiring future graduates of the program. Indicates labor market demand for recent vocational education and training graduates. Also may be used to develop cost-effectiveness measures for vocational education and training programs. Useful for comparing student choices with current employer demand. May be used to prioritize programs with similar student demand indicators. Indicates employer demand for skills imparted by various types of education and training.
Employer satisfaction information may also be valuable in making adjustments to curriculum and program design. Information on employment and wage outcomes of previous students should be provided to current and prospective students to assist them in making training decisions.
Developed from analysis of demographic data, job analyses, employer surveys, and/or reverse tracer studies (see indicator 26). Employer surveys or discussions. Education data systems Surveys of program completers
27
Reverse Tracer studies: studies of training sources of current workers in specific occupations.
Household surveys, employee surveys
Page 38

Indicator
28 Application-toenrollment ratios by program # # Student Demand Short-term Strategic Demand Goals

Indicates extent to which past student demand has been met.
Comments
Captures unmet student demand information. Data must be interpreted in light of reasons students are not enrolled (e.g., lack of student qualifications, lack of financial resources, student choosing other training options or employment). Demographic characteristics of students (age, gender and race/ethnicity) may be important. Does not reflect unmet student demand. When used as indicator of supply of trainees, completion information should be combined with information from tracer studies. Not all completers will seek employment in related occupations. Also, some completers may already be employed in the occupation. Demographic characteristics of students (age, gender and race/ethnicity) may be important. Data must be interpreted in light of reasons for students not completing the program (e.g., unsatisfactory student performance, student dissatisfaction with training, leaving for related employment). Demographic characteristics of students (age, gender and race/ethnicity) may be important. Does not reflect unmet student demand. When used as indicator of expected new supply of trainees, enrollment information should be combined with enrollment-tocompletion ratios and tracer studies. Not all current enrollees will complete the programs, and not all completers will obtain employment in related occupations. Demographic characteristics of students (age, gender and race/ethnicity) may be important.
Information Sources
Education data systems
Student Training Demand Indicators
29
Number of completions by program (past or expected) # # #
Indicates extent to which past student demand has been met. Indicates supply of trainees currently available to meet current labor market demand. Also a Training Supply Indicator.
30
Enrollment-tocompletion ratios by program #
31
Number of students enrolled by program
Indicates current student training demand. Indicates expected new supply of trainees currently available to meet current labor market demand. Also a Training Supply Indicator.
Page 39

Indicator
32 Student satisfaction with existing training programs Number of completions by program (past or expected) # # # Student Demand Short-term Strategic Demand Goals

Comments
Information Sources
May be part of tracer studies.
Student Training Demand Indicators, Continued

#
Training Supply Indicators

29 Indicates extent to which student demand has been met. Indicates supply of trainees currently available to meet current labor market demand. Also Student Training Demand Indicator. Does not reflect unmet student demand. When used as indicator of supply of trainees, completion information should be combined with information from tracer studies. Not all completers will seek employment in related occupations. Also, some completers may already be employed in the occupation. Demographic characteristics of students (age, gender and race/ethnicity) may be important. Does not reflect unmet student demand. When used as indicator of expected new supply of trainees, enrollment information should be combined with enrollment-tocompletion ratios and tracer studies. Not all current enrollees will complete the programs, and not all completers will obtain employment in related occupations. Demographic characteristics of students (age, gender and race/ethnicity) may be important. Capacity measured by number of training seats available (actual training supply) is preferable to using data on enrollments or completions. Enrollments and completions do not fully represent training capacity if training seats are not filled. Education data systems
31
Number of students enrolled by program
Indicates current student training demand. Indicates expected new supply of trainees currently available to meet current labor market demand. Also a Student Training Demand Indicator.
33
Number, location, and capacity of existing vocational education and training programs
Indicates current supply of training. Useful for comparing anticipated demand with the current supply of vocational education and training. Where enrollment is below capacity, may indicate oversupply of training (i.e., lack of student demand).
Page 40
Selecting Training Needs Indicators: Step by Step

Step 1. Identify and Document Available Data. In this step, you will review the list of indicators and find out what data are available to you. The result will be a list of indicators for which data are currently available. This step will require the assistance of individuals who are expert in the various data sources, particularly individuals from your countrys population and employment statistics agencies, and someone who is familiar with the available data collected on education and training programs.
Determining Information Requirements: Step by Step 1. Identify and document available data. 2. Decide what additional data should be collected.
In this review, you should focus on the list of indicators that are appropriate for the decision drivers that were confirmed in Phase 2, Establishing the Policy Framework, Step 1, Review the Goals of Vocational Education and Training.
To assist you in this step, a worksheet is provided at the end of this section that lists all the indicators from Table 2, along with the relevant decision driver for each indicator. The last three columns of the worksheet provide an area for you to record whether data are available for this indicator, what the data source is, and, if data are not available, whether you recommend collecting it. In the data identification process you should assess the quality of the available information, especially its currency and accuracy. You will need to decide whether the available data are of good enough quality for use in the needs assessment. Remember that making high-stakes decisions may require higher quality information. TIP: In addition to the worksheet, you should keep notes on each of the indicators and data sources, recording the following types of details: ! ! ! ! ! ! Who provided information about the available data sources (name, position, telephone number, etc.), Definitions, concepts, and data collection methods, The media in which the data are available (paper, type of electronic file), When the data were collected and their reference period, The geographic area covered, The classification system used and the level of detail provided in the data, such as occupational or industrial detail, Page 41
Your assessment of data quality (currency, accuracy, whether the definitions and level of detail are appropriate to the decision being made, whether the coverage is appropriate), Other details you think are important.
Step 2. Decide What Additional Information Should Be Collected. Completing Step 1 will tell you what information presently is available. You will need to decide whether the available information is sufficient or whether additional information must be collected. In making this decision, you should consider the nature of the decision to be made. Generally, the more critical the decision, the more indicators you should have and the higher their quality should be. Also, the existing indicators may conflict with each other and suggest different conclusions. In these cases, additional information is probably needed to reach a sound decision. Data collection methods are described in Phase 6. In making decisions about what specific information to collect, you should consider several factors: ! ! ! ! Difficulty and cost of collecting the additional data, Time required to collect the data, Expertise needed to assist in the data collection and whether the expertise is available, and Benefit expected in comparison with the cost and time needed.
TIP: Making the decision about data collection is an excellent opportunity to involve the Stakeholder Group. You can present the results of Step 1 to the Stakeholder Group, informing them of the available data, and discuss recommendations on what should be collected, along with information about difficulty, costs, time, expertise needed, and benefits.
Page 42
Phase 5 Worksheet: Identification of Data Available and Indicators Recommended for Data Collection
Vocational Education and Training Goal Indicator Overall Labor Market Indicators 1 Population trends and forecasts by age and gender 2 Labor force trends and forecasts by age and gender 3 Unemployment rates by education level 4 Unemployment rates by age and gender 5 Educational attainment of the population by age and gender 6 School enrollment by education level and type of school 7 Trends in Gross Domestic Product and imports/exports by sector 8 Current employment by industry 9 Employment forecasts by industry 10 Current wages and wage trends by industry sector. Occupational Demand Indicators 11 Current employment by occupation 12 Current job vacancies by occupation 13 Time required to fill vacancies 14 New hires by occupation 15 Current wages and wage trends by occupation 16 Unemployment rates by occupation 17 Employment forecasts by occupation 18 Forecasts of job openings by occupation 19 Education level by occupation 20 Gender of current workers in the occupation
Student Demand Short-term Demand Strategic Goals
Data Available?
Data Source
Recommend Data Collection?
# # # #
# # # # # # # # # # # # # # #
# # # # # # #
# # # # #
Page 43
Vocational Education and Training Goal Indicator Occupational Demand Indicators, Continued 21 Employer hiring practices by occupation 22 Employer-reported skill shortages 23 Occupational distribution of employment within industries (staffing patterns) 24 Information on industry investment and expansion and contraction actions or plans (e.g., new plant construction or expansion, major new construction projects, layoffs or plant closings) 25 List of individual employers, with industry designation and contact information 26 Employment forecasts by occupation 27 Forecasts of job openings by occupation 28 Employer satisfaction with existing programs 29 Tracer studies: employment outcomes and wages of vocational education and training graduates 30 Reverse Tracer studies: training sources of current workers in specific occupations. Student Training Demand Indicators 31 Application to enrollment ratios by vocational education and training program 32 Number of students enrolled by program (also a training supply indicator) 33 Number of completions by program (past or expected) (also a training supply indicator) 34 Enrollment to completion ratios by vocational education and training program
Data Available?
Data Source
# # # # # # # # # # # # # # # # # # # # # #
Page 44
Vocational Education and Training Goal Indicator Student Training Demand Indicators, Continued 35 Student satisfaction with existing training programs Training Supply Indicators 36 Number of students enrolled by program (also a student training demand indicator) 37 Number of completions by program (past or expected) (also a student training demand indicator) 38 Number, location and size of and enrollment in existing vocational education and training programs 39 Related vocational education and training programs
Data Available?
Data Source
# # # # # # # # # # # # #
Page 45
Page 46
PHASE 6. GATHERING ADDITIONAL INFORMATION

Three general methods are available for gathering additional information: using administrative records, conducting surveys, and carrying out group processes. Data from administrative records have the advantage of using information that already is being collected for another purpose. Problems can arise in getting access to the data and with the timeliness and coverage of the data. Surveys are formal data collection processes and, if well designed and properly conducted, can provide high-quality information for decision-making. Surveys can be costly and take time to develop and conduct. Group processes, including methods such as focus groups and Delphi techniques, are useful in gathering qualitative information such as the expectations, opinions, and perspectives of employers, students, and other target audiences. Information from group processes may be less representative than information from surveys but may provide indepth qualitative information that cannot be obtained from surveys. Group methods may also be used to develop consensus on topics such as goals. Each of the data collection methods is discussed briefly below.
Administrative Data
Administrative data consist of information extracted from existing records such as employer tax reports and school records. Administrative data can be invaluable, as they may provide the universe list of target groups such as employers and provide a less expensive data source than surveys. A number of pitfalls exist when using administrative data, however. ! The cooperation of the owner of the data must be obtained. Creating administrative databases requires access to the administrative records from which the data items are extracted, and these records are held by the organization responsible for the process that generates the records, such as a tax agency or an individual school. Often, these organizations do not see providing data as their mission, or they may have legal restrictions on access to the records. Administrative data may not be timely. Timeliness depends partly on the schedule associated with the administrative process, such as tax filing schedules. Extra time may be required to get and process the administrative records, as compared with operating a survey, especially if the records are not in electronic format. The data may not be in a convenient or even a usable format. For example, it may only exist on paper records that may not be removed from the site. The time, effort, and cost of putting the data into usable form should be evaluated. The data must be coded by, for example, adding industrial classification codes to each employer record or geographic area codes to each address.
Page 47
! !
Administrative data files usually have missing reports or late reports, for example, from employers who fail to file tax reports or file late. The concepts, definitions, and coverage of the administrative data must be examined to get a clear understanding of what the data represent. For example, an employer database derived from tax records will cover only employers who are subject to the relevant tax requirements. This may exclude certain types of employers, such as those in the agricultural sector or the self-employed.
Surveys
Surveys may be used to collect data from any of several different target audiences: employers, individuals surveyed at their residence (household surveys), individuals surveyed at their workplace (employee surveys), individuals surveyed at school (student surveys), and schools surveyed to gather institution and program data (school surveys). Employer Surveys. Employer surveys are a common needs assessment tool, even where comprehensive labor market data are available. Employer surveys are used to provide information not included in the labor market data and sometimes to confirm or update the labor market data. Where labor market data are weak or not available, employer surveys become critical. A side benefit of employer surveys is the message they inherently give to employers: that their input is important and that the vocational-technical training programs are trying to meet their needs. This must be balanced against the effort employers are asked to spend to respond, however. Employer surveys must be carefully designed, maintaining a focus on how the results will be used. Typically response rates in such surveys are relatively low, and it can be tempting to ask too many questions, reducing employer response rates even further. Household Surveys. Household surveys are usually the best method of collecting demographic information and other data about individuals that cannot be collected from employers. Household surveys often include questions on topics that are also covered in employer surveys, such as the industry or occupation of the respondents employment. Household data on these topics permits analysis by demographic characteristics, which usually is usually not possible with employer survey data. An example is the analysis of occupational employment by gender and age of the workers. Age and gender are easily collected in household surveys but may be difficult to collect from employers, who may not have this information easily accessible or may be reluctant to provide information on the personal characteristics of their workforce. Household surveys also capture information about individuals who are not employed, which obviously cannot be done with employer surveys. Employee Surveys. This method collects information from individuals who are contacted at their place of work and is quite useful for gathering in-depth information about specific occupations or industries. These surveys can be quite difficult to conduct, however, as they require the cooperation of both the employer and the worker. Incumbent worker
Page 48
surveys may require methods to protect the privacy of the individual workers responses from access by the employer. As will be noted in the section of this Guide on company needs assessment, incumbent worker surveys are important in assessing training needs within the company. Student Surveys. These surveys collect information from students who are contacted at their school. The cooperation of the school as well as the student is needed. Student surveys, such as tracer studies, are often conducted by schools themselves. School Surveys. School surveys may be used to gather information about the institution and its services, such as enrollment, programs offered, number of students completing the programs and/or receiving diplomas or other credentials, number of teachers, financial information, information on facilities, and other data. School surveys should not duplicate collection of information that is required in administrative reports. Sample Survey or a Census? Surveys may be conducted by contacting a sample of respondents or contacting all potential respondents through a census. The choice between a sample survey and a census is a critical one. Unless there is a specific reason to conduct a census, such as the need for a baseline complete count that is provided by population censuses, the sample survey is the preferred choice. The guiding principle is to use the method that will achieve results that are representative of the overall population being studied. In employer surveys, for example, results are needed that are representative of all employers in the area or all employers in a particular industrial sector. Conducting a census might sound like the best approach to those unfamiliar with basic statistical principles. It may be argued that since a census probably will provide a greater number of responses than a sample survey, census data are just as good as (that is, just as accurate and representative) or better than sample survey results. Unless a very high response rate is achieved, this argument is in error. As any standard statistical reference will discuss, the results from a census represent only the characteristics of the employers or households that actually respond to the survey. The results cannot represent the characteristics of the entire population unless, as the term census implies, nearly all of the population is included in the count. A census also is more expensive to conduct that a sample survey, since all members of the target group (employers, households, etc.) must be contacted. Representative data usually can be more easily and cheaply collected with a welldesigned sample survey. Sample surveys provide data for which measures of accuracy (sampling error) can be calculated. The cost is less than a census, as many fewer contacts are needed. For gathering data on very small populations, sample designs might include virtually all members of the population, especially if detailed presentations of the data are planned. In such cases, conducting a census instead of a sample survey may be appropriate.
Page 49
Response Rates. Response rates are critical to the success or failure of a survey. Unless the response rate is at an acceptable level, the survey results will be useless. For sample surveys, selecting a target response rate is part of the sample design process. The lower the target response rate, the larger the sample must be to achieve results with acceptable accuracy and the more difficult and expensive the survey will be to conduct. For census surveys, very high response rates are required to achieve representative results, as discussed earlier. What are typical response rates? The response rates that can be expected vary tremendously for different types of surveys, and according to how the survey is designed and conducted. Dillman notes that mail surveys he reviewed during preparation of his Total Design Method averaged about 77 percent, and that telephone surveys produced somewhat higher response rates.10 The standard for federal government sponsored surveys in the United States is 80 percent. The response rate for the 2000 Occupational Employment Statistics Survey, an employer survey providing employment and wage data by occupation, was 78 percent. Achieving these high response rates requires skilled and experienced staff and welldesigned and tested survey procedures. Most surveys produce far lower response rates, especially if they are conducted by organizations without experience. Good survey design and implementation can increase response rates. Dillman recommends a variety of ways to improve response rates, based on the theory of surveys as social exchange. He argues that individuals will be motivated to respond by three factors: the reward they expect to gain, the cost to them of responding, and their level of trust that the rewards will outweigh the costs. Rewards in this context are not necessarily economic rewards but may be intangible rewards such as demonstrations of respect, making a contribution to improvement of ones community, or feeling valued because ones advice has been requested. Costs may include the literal cost of the time and effort required to respond, as well as the cost of disclosing personal or business information, and other social costs.11 Based on the theory of social exchange, Dillman has identified and tested a number of basic methods to increasing response rates: ! Increasing rewards by using language in cover letters that shows the value placed on the respondent and their advice, saying thank you in advance and follow-up contacts, indicating how the survey will be used in making decisions that will benefit the community, and making the questionnaire itself interesting to respondents. Reducing the cost to respondents by making the questionnaire short and easy to answer, providing return postage for mail surveys, using wording that asks for the respondents assistance rather than authoritatively demanding a response, minimizing questions about personal or proprietary information, and avoiding embarrassing the respondent by, for example, requiring complex reading or writing that he or she may not be able to perform.
Page 50
Increasing trust by providing tokens of appreciation in advance such as an item or a small amount of money, having the survey sponsored by a legitimate authority, emphasizing the importance of the survey.
Sources of Error in Survey Data. Survey data are subject to various sources of error, which usually can be managed by good survey design and execution. Four types of error are important: sampling error, coverage error, measurement error, and nonresponse error. Sampling error occurs because, in sample surveys, some respondents are selected to represent all respondents. Sampling error can be explicitly measured by calculating the margin of error or standard error of the estimate when the survey results are tabulated.12 Sampling error can be reduced through good sample design and obtaining high response rates. NOTE: Sampling error does not apply to census surveys, and the margin of error for a census cannot be produced. Coverage error occurs because the population being studied is not completely covered by the surveys universe list. Some members of the population therefore do not have the opportunity to be represented. For example, a telephone survey of a population that includes many individuals without a telephone would have high coverage errors. Coverage error can be managed by obtaining a universe list that is as complete as possible, and using appropriate data collection methods. Measurement error occurs when the survey questions are not well designed. Poor question wording may result in respondents skipping questions or providing inaccurate responses to questions they do not understand. Measurement error may also result from questions that, while clear, do not accurately reflect the information the survey is intended to capture. Measurement error can be reduced by good questionnaire design, including the critical step of testing the questionnaire with potential respondents. Nonresponse error occurs because the respondents to the survey are different from those who did not respond in ways that affect how they respond to the questions. For example, if the respondents in an employer survey on wages are disproportionately large businesses, which are more likely to pay higher wages than small businesses, the resulting wage data will have a bias toward higher wages. Nonresponse error can be managed through good sample design, achieving good response rates, and perhaps by statistical adjustments for nonresponse bias. Analysis of the characteristics of nonrespondents in comparison with respondents is very useful in understanding nonresponse error. NOTE: Census surveys that do not achieve high response rates are particularly subject to nonresponse error. Other Resources About Surveys. A wide range of resources is available to provide additional background information about surveys and how to conduct them. Two may be particularly helpful: ! The American Statistical Association publishes several pamphlets on survey topics that provide good background for a non-technical reader. Topics covered include What Is a Survey, How To Plan a Survey, How To Collect Survey Data, Judging The Quality of a Survey, Mail Surveys, Telephone Surveys, Margin of Page 51
Error, and Designing a Questionnaire. These pamphlets are available on-line at http://www.amstat.org/sections/srms/whatsurvey.html. ! Don Dillmans book, Mail and Internet Surveys: the Tailored Design Method (2nd edition, John Wiley and Sons, New York, 2000), is a practical guide to designing and conducting surveys.
Group Processes
Surveys and administrative records are excellent tools for collecting quantitative data. However, qualitative information, such as the opinions and attitudes, may also be valuable in the needs assessment process. 13 Group processes provide the opportunity for face-to-face interactive communication with members of important groups who have knowledge about and a stake in vocational education and training, such as employers, students, parents, and educators. Group processes may also be used with the Stakeholder Group in developing ideas and goals for the needs assessment. Group processes have the advantage of providing a wide range of information quickly and allowing unanticipated topics to be identified and explored. Group processes also have some disadvantages. They do not produce representative information, as they reflect only the views of the participants, which cannot be generalized to the entire population. Also the results are influenced by the skill and motivation of the group facilitator. Because of these limitations, information from group processes should not be the only source of information in the needs assessment. Group process results should be used along with quantitative data from existing data sources, surveys, or administrative data. While a variety of group processes are available, three may be particularly relevant to training needs assessment: the Nominal Group Technique, the modified Delphi technique, and focus groups. Nominal Group Technique. The Nominal Group Technique (NGT) is a structured process to identify and rank problems or issues affecting the group, or managing participation in planning, performance measurement, and other processes. NGT is used to gather generate a large number of individual views and ideas on a particular topic and to achieve consensus on a list of priorities, measures or actions. NGT sessions should include 6 to 10 invited participants and a skilled facilitator. Each session should be limited to discussing one or two questions that have been prepared in advance. If more than one or two questions need to be examined, several NGT sessions, each with different participants, should be held. Modified Delphi Technique. The Delphi technique is characterized by the iterative use of a mail survey over time with the same panel of respondents. Between each iteration, the extent of agreement or disagreement among the panel members is identified, and this information is shared in the next iteration. The process thus helps build consensus. The modified or group Delphi technique adapts the procedure for use in a face-to-face group setting. The Modified Delphi technique is often used to develop forecasts of future trends. Page 52 Assessing Vocational-Technical Training Needs
Focus Groups. Focus Groups are widely used to interview a small group (8 to 12 people) to obtain their in-depth views. A skilled facilitator is required. Focus Groups provide an opportunity for probing and follow-up questioning by the facilitator to explore responses, and the opportunity for interaction among the participants can generate additional insights. Unlike NGT and the Modified Delphi Technique, the focus group process does not attempt to obtain consensus. Resources about Group Processes. A number of resources are available to guide the selection and use of group processes. Especially helpful examples are: ! Planning and Conducting Needs Assessments: A Practical Guide, by Belle Ruth Witkin and James W. Altschuld, Sage Publications, Thousand Oaks, California, 1995. The American Statistical Associations pamphlet What are Focus Groups?, available on-line at http://www.amstat.org/sections/srms/whatsurvey.html. The University of British Columbias on-line list of resources on qualitative group methods, available at http://www.slais.ubc.ca/resources/research_methods/index.htm
! !
Gathering Additional Information through Surveys: Step by Step

Step 1. Get Access to Survey Expertise. Conducting surveys is a specialized field. Unless you have expertise in designing and operating a survey, including expertise in statistics, you should obtain the assistance of individuals or an organization with the needed skill and experience. This type of experience is often available from statistical agencies, universities, private vendors, and international organizations. Capacity for conducting surveys includes: ! Staff with the statistical expertise to design and select a sample, develop and test the survey questionnaire, and calculate final survey results based on the responses received, Staff with experience in managing and conducting the survey operation, including printing, mailing or telephone operations, tracking responses received, and following up on nonrespondents, and
Gathering Additional Information through Surveys: Step by Step 1. Get access to survey expertise. 2. Define the information to be collected. 3. Define and identify the survey universe. 4. Determine the data collection method. 5. Develop the analysis plan. 6. Design the survey sample. 7. Develop and test the survey questionnaire. 8. Gain respondent interest and cooperation. 9. Conduct the survey. 10. Analyze and distribute the results.
Page 53
Computer resources, including equipment and software programs for statistical tasks and survey operations. Define the Information to be Collected.
Step 2.
In Phase 5, you have identified the type of information needed by comparing the list of indicators to available data sources. In data collection, you need to define specifically what you want to collect and from whom. For example, you may have determined that you need an indicator of current job vacancies by occupation (item 12 on the Phase 5 worksheet). To collect this data, you need to decide exactly what you mean by current job vacancies by occupation. ! ! ! ! For what time period? How will job vacancy be defined so survey respondents will provide accurate responses? How will the vacancies be classified by occupation? Will you provide the respondent with a list of occupations, or will you classify job titles provided by the respondents into a standard occupational classification? Do you need to know anything about the job vacancies besides occupation, such as the wage being offered or how long the vacancy has been in existence?
In defining the information to be collected, you must keep at the forefront how the data will be used in the decision-making process. Step 3. Define and Identify the Survey Universe. In this step you will determine from whom you will collect the data. Is it to be collected from employers, households, incumbent workers, or students? Should there be any limits on the universe, for example, only employers in certain industries or only employers with more than a certain number of workers? Once you know the target audience for the survey, you will need to identify a list of specific potential respondents that will constitute your survey universe. For employer surveys, the universe is the complete list of employers, including any desired limits such as industry or number of workers. For household surveys, the universe is a list of households, and so forth. Obtaining a good universe list is key to reducing coverage error in the survey results. TIP: Be sure to examine the universe list to be sure you understand what it includes and how it was derived. Dillman suggests the following questions to evaluate universe lists14: ! Does the list contain everyone in the survey population?
Page 54
! ! ! !
Does the list include names people or businesses that are not in the survey population? How is the list maintained and updated? Are sample units included on the list more than once? Does the list contain other information that can be used to improve the survey?
You should also ask: How recent is the list? Step 4. Determine the Data Collection Method.
You will need to decide whether the survey will be conducted by mail, by personal interview, by telephone, or by Internet. This choice will be driven by the nature of the information to be collected, who the information will be collected from (employers, households, etc.), costs, the availability of information needed to conduct the survey (mailing addresses, telephone numbers), and capacity to implement the alternative collection methods. TIP: The University of Florida Cooperative Extension Service Circular PE-21 provides a useful guide on selecting a data collection technique. The circular is available on-line at http://edis.ifas.ufl.edu/PD016. Step 5. Develop the Analysis Plan. Before developing the actual survey instrument and sample, you should decide how you will analyze and present the resulting data. One good way to do this is to lay out table formats you will use for presenting the results. This exercise is important, because the survey can then be developed to provide the information required in the tables. In an example, you may decide that you will need to analyze job vacancy data from your survey by specific industry and geographic area. This decision will affect the sample design, as a larger sample will be needed to provide accurate data by industry and geographic area that would be needed just to provide an estimate of the total number of job vacancies. Step 6. Design the Survey Sample. In doing this step, you will first need to decide whether your survey will be a census or a sample survey. As discussed earlier, a sample survey is usually the preferred choice. Your statistical expert should develop the sample design, taking into account the analysis plan, the expected survey response rate, and the level of accuracy desired in the resulting data. The design might also include features to adjust for nonresponse biases, that is,
Page 55
biases in the resulting data that are caused by differences in the characteristics of respondents in comparison with nonrespondents. Step 7. Develop the Survey.
In this step you will develop and test the survey questionnaire, including the wording of the questions and the exact sequence and format. You will also need to develop a letter or postcard to be sent out in advance of the survey and, for mail surveys, a cover letter to be mailed along with the questionnaire. Finally, you should develop the procedure, schedule, and materials for follow-up contacts with nonrespondents, such as reminder letters or a script for telephone follow-up. A critical part of this step is to test the questionnaire. You will need to find out how long it will take the respondents to complete the questions, whether they will understand the questions, and whether they will be willing and able to provide an accurate response. TIPS: Keep It Simple! The longer and more difficult the questionnaire, the lower the response rate will be. The questionnaire should cover the necessary topics and use a clear, easy-to-follow format but should not include extraneous questions or long, complicated text. Use American Statistical Associations pamphlet Designing a Questionnaire, which contains a number of simple guidelines to good questionnaire design.15 Test the questionnaire with several actual potential respondents. Check the American Statistical Associations pamphlet How to Conduct Pretests for guidelines. 16 During the test, time how long it takes respondents to complete the survey. Then, in the survey cover letter or instructions, give respondents information on how long it is likely to take them to complete their response. Be sure to plan prompt follow-up with nonrespondents to encourage response. For mail surveys, this should include sending another copy of the questionnaire. Testing has shown that nonresponse follow-up can increase response rates by 20 to 40 percentage points and that several follow-ups may be needed.17 Step 8. Gain Respondent Interest and Cooperation.
You may be able to improve response rates by doing outreach to potential respondents to let them know about the survey and its purpose. If they understand the benefits of the survey, they will be more likely to respond.
Page 56
TIPS: Involve your Stakeholder Group in this outreach to potential respondents! Have the cover letter signed by a known and respected authority. Also include the name and telephone number of an individual who can be contacted for more information about the survey. Step 9. Conduct the Survey. Once the sample has been designed, the survey instrument is developed and tested, procedures and materials are prepared, and outreach has been conducted, your survey operations experts should be ready to conduct the survey. The major steps include: ! ! Drawing the sample, using the universe list and the sample design to produce a list of respondents who will be contacted, Refining the respondent list to be sure complete and accurate information is available for all employers or individuals on the list, such as mailing address (for mail surveys) or telephone numbers (for telephone and personal interviews), Printing the quantity of questionnaires, letters and/or postcards needed, including the quantity needed for follow-up contacts, Sending cover letters or advance post cards to go out with or ahead of the survey, and reminder notices to nonrespondents, Conducting the data collection, including doing the mailing, interviews, or telephone calls, monitoring the collection to identify respondents and nonrespondents, and making the follow-up contacts with nonrespondents, Following up with nonrespondents, such as reminder letters or postcards, contacting nonrespondents by telephone, and doing follow-up mailings of mail surveys, Entering the data from survey respondents, Editing and evaluating the data, including checking for internally consistent responses and deciding whether the response rate is high enough to provide data of acceptable accuracy, and Tabulating the survey results, including calculating the statistical estimates and creating tables using the table formats designed in Step 5.
! ! !
! !
TIP: Be sure that survey staff are trained in how to conduct the survey! Training should cover the purpose of the survey, the use of the information it will provide, the
Page 57
content of the questionnaire, the survey schedule, and the procedures to be used. For telephone surveys, interviewing techniques and practicing the telephone interview should be covered in the training.
Step 10. Analyze and Distribute the Results. In this step, you will use the survey results to develop an understanding of the indicator the survey was intended to provide. For example, in a survey of current job vacancies by occupation, you might prepare a description of the number and type of job vacancies identified in the survey and discuss what this information means for the vocational education and training program decisions that are being made. A written report of the survey results should be prepared that describe the results and your analysis, as well as describing how the survey was designed and conducted, what the response rates were, and the quality of the resulting information. This report should be distributed to the appropriate audiences.
Gathering Additional Information through Group Processes: Step by Step

Regardless of the particular group process method used, some basic planning and execution steps must be carried out. These are listed below, following the structure identified by Witkin and Altschuld18. For more information on specific group process methods, consult the sources listed in the References section of the Guide. Step 1. Establish clear purposes for the group process. Be sure the purpose and outcomes for the group process are clearly defined, so participants will know why their participation is important and what is expected of them and so the facilitator can work effectively. The clear statement of purpose will also be important in selecting the appropriate group process method and developing the procedures. Step 2. Select the group process. Review the alternative group processes Nominal Group Technique, Focus Groups, and Modified Delphi Technique and decide process best fits the purpose identified in Step
Gathering Additional Information through Group Processes: Step by Step 1. Establish clear purpose. 2. Select the group process. 3. Identify a skilled facilitator. 4. Develop procedures and arrangements. 5. Invite the participants. 6. Conduct the process. 7. Analyze the results. 8. Communicate the results.
Page 58
1. Your choice will also be affected by the relative difficulty of the different methods and the availability of facilitators who are skilled in the different methods. Step 3. Identify a skilled facilitator. Get the assistance of an individual who has been trained in the group process method. Note that it is not essential that the facilitator have in-depth knowledge of the subject matter that will be discussed. It is important that the facilitator not try to influence the groups thinking and not interject his or her personal opinions in the discussion. The process is to get the groups thinking, not the facilitators. Step 4. Develop the procedures and arrangements. In this step, you will need to develop the specific materials needed for the process, such as questions for the groups response. You will also need to make arrangements for a place for the group session, materials needed in the room (such as flip charts and markers), and prepare the agenda. Make sure the procedures include a way of recording the groups ideas and consensus. This might include providing someone to serve as recorder or observer, as well as capturing notes and flip charts generated during the session. Recording the results will allow you to refer to them accurately when decisions are being made using the groups input. TIPS: The group may be more effective if the group session is held at a place away from the participants usual place of work. This will help them avoid distractions. Make sure the meeting room is comfortable! And provide breaks and refreshments. Step 5. Invite the participants. The participants should be individuals with knowledge of the topic being discussed and have an interest in the outcome. The number of participants should be appropriate for the technique selected. Be careful about the mix of people invited. For example, in the NGT, a cross-section of people from different constituencies can effectively participate, while Focus Groups work best if the participants share similar backgrounds. Avoid inviting supervisors and subordinates from the same organization to the same work group. They may not feel free to express their views in front of one another.
Page 59
Step 6. Conduct the process. On the planned schedule, the facilitator will conduct the group activity. Step 7. Analyze the results. Immediately after the groups work is completed, prepare a summary of the results, including any statements of consensus, findings, or recommendations generated by the group. Provide this summary and a thank you to the group participants. Step 8. Communicate the results. Provide the results to the decision-makers, along with other information being used in their deliberations.
Page 60
PHASE 7. ANALYSIS AND PRESENTATION

With the conclusion of Phases 5 and 6, you will have available all the information to complete the needs assessment. What remains to be done is to analyze the information to identify the specific training needs. You also should present this information to the vocational and technical education decision-makers. The specific analysis to be done will depend to a great extent on the information available from Phases 5 and 6. Regardless of the analytical method used, however, the analysis should lead to identification of relative need for various training programs, either in the form of lists of programs or lists of occupations in demand for which programs should be provided. The results of the needs assessment should be put into a written report, and an oral presentation should be developed for any appropriate audiences, such as the Stakeholder Group and the program decision-makers. The written report is essential so the results can be accessed by individuals who may not be available for presentations, as well as to provide a record to document decision-making.
Analysis and Presentation: Step by Step 1. 2. 3. 4. 5. Assemble the information. Decide on analysis method. Conduct the analysis. Prepare written report. Prepare and give oral presentations.
Analysis and Presentation: Step by Step

Step 1. Assemble the available information. Put the data and information from focus groups or other qualitative activities into a format that permits analysis. This might be in the form of databases that permit querying the data, spreadsheets for ranking and comparison, or other forms. Step 2. Decide on analysis method. You will need to choose the analysis methods that best fit the information available and address the policy questions. Examples of methods include comparisons, rankings, computing scores that represent several indicators, and writing narrative analyses for different occupations or programs. Step 3. Conduct the analysis. In this step, you will apply the analysis methods to the information, identify patterns or key points, and develop conclusions or recommendations.
Page 61
Step 4. Prepare written report. In this step, you will write the needs assessment report. The report should cover the following topics, at a minimum: ! A description of the policy context in which the needs assessment is being conducted and the vocational-technical education goals or drivers being addressed. The scope of vocational-technical education covered by the needs assessment. The target geographic area. A summary of the Economic Review. A summary of the methods used, such as surveys or group processes. The results, including the analysis leading up to the results and a list of the programs or occupations identified as training needs. Documentation about the data sources and methods used, perhaps shown as an appendix.
! ! ! ! ! !
TIP: Be sure the needs assessment report acknowledges the role and contributions of the Sponsor and the Stakeholder Group.
Step 5. Prepare and give presentations. In preparing the oral presentation, keep in mind that usually only the highlights can be covered in a presentation. There will not be time for covering all of the material contained in the written report. Also, be sure to tailor the presentation to your audience. How much do they already know about the needs assessment project? Do they want to know the details of the study or just the highlights and recommendations?
Page 62
PHASE 8. PUTTING NEEDS ASSESSMENT RESULTS TO WORK

Once decisions are made on the training programs to be offered, several additional activities are needed to make full use of the needs assessment results. These are directed toward providing information for developing the training curriculum, and keeping the curriculum up to date with developments in the work place and the labor market. For these programs to be effective, data about the programs must be collected, and career information must also be provided to current and prospective students.
Putting Needs Assessment Results to Work: Step by Step

Step 1. Develop and update curriculum. The needs assessment probably resulted in decisions about occupations or programs for which training will be offered. Translating a list of Putting Needs occupations or programs into training curriculum Assessment Results to requires activities to identify the specific skills and Work: knowledge that must be taught in the program to qualify Step by Step students for work in the occupation. Job analysis is the tool for identifying this skill and knowledge information. As discussed in the following section on needs assessment within a company, several job analysis methods are available. One of the most often used methods is DACUM.
1. Develop and update curriculum. 2. Monitor the labor market. 3. Provide career information.
4. Collect and analyze Once a program curriculum has been prepared and is put program information. into use, it is important to keep it up to date. Skill requirements can change rapidly, and the knowledge workers must have to perform their jobs may also change. It is critical, therefore, to keep employers involved in reviewing and advising on program curriculum. The creation of employer advisory committees, for example, will help tie curriculum to current practice in businesses that may hire program graduates. The References section notes resources about employer advisory committees.
Step 2. Monitor the labor market. The economy and the labor market are dynamic. Change is occurring all the time and can be abrupt and unexpected. Thus, it is important to monitor the labor market to spot changes that could affect decisions about vocational and technical education program offerings. These may be specific events or major changes in labor market trends or smaller changes that call for continuous adjustment of training investments.
Page 63
Step 3. Provide career information. In a vocational technical education system where student choice is the primary driver of decisions on program offerings, providing career information to students is vital. Students need to be know about the careers for which vocational-technical programs will prepare them, how well the work in these occupations fits their interests and goals, what the job and earnings prospects are like, and what additional education and training they may need. They need to be equipped to make informed choices. Even in systems where student choice is not the primary driver, career information plays an important role. Offering programs to prepare workers for demand occupations needs to be accompanied by providing information to current and prospective students about how these programs will prepare them for good careers that meet their personal interests and goals. Without such information, the programs may fail for lack of student enrollment. Step 4. Collect and analyze program information. Needs assessment is a continuing process that should include regular review of programs to track the level of student enrollment, student completion of the programs, and tracer studies to identify student outcomes. Such information also is critical for overall management of the vocational technical education system.
Page 64
TRAINING NEEDS ASSESSMENT WITHIN A COMPANY

Part of good human resource management in any business or organization is providing training for staff development. Providing the right training training that is needed and will benefit the company as well as the employees is key. Training must be tailored to the businesss strategy and fit within its organizational structure and work processes. Needs assessment is the foundation for successful company training. Needs assessment will identify which employees should receive training, the type of training they should get. The assessment will lead to development of appropriate training curricula. Doing solid needs assessment will help the company avoid wasting its resources money, time, and physical resources on training that is not appropriate. Company training as referred to here includes training the company provides to its employees directly, as well as employee training sponsored by the company but provided by others, such as schools, technical colleges, and private providers. In general, needs assessment is a process of identifying the skill and knowledge requirements of the work, assessing the skill and knowledge levels of the experienced employees or newly hired employees and developing training to address the gap between requirements and employee capabilities. The result should be a set of objectives describing the purpose of the training and the competencies employees should have when training is completed.
Reasons for Providing Company Training

The initial issue that must be examined in assessing training needs is, What is the nature of the business problem that training is to address? Companies provide employee training for a number of different reasons, and each reason can lead to a different type of needs assessment. Typical reasons for company training are discussed below. Training new employees in company-specific procedures, tools, and processes. New employees often need training in operating the specific equipment used by the company, in company procedures and standards, and in other company-specific topics they would not be expected to know prior to their employment. Training should be based on an analysis of the job content and the expected capabilities of new employees. Planning for new employee training must be coordinated with recruitment and hiring, so employees hired have the expected knowledge and skills. Training new and current employees to address basic or general skill deficiencies. Companies may find that the workers available in their labor markets do not possess basic skills (reading, mathematics) or general skills and knowledge. Ideally, potential employees gain basic and general skills and knowledge through their education experience. Where this does not happen, companies may decide to provide training. Assessing Vocational-Technical Training Needs Page 65
Training needs should be identified based on analysis of the basic and general skills and knowledge required by the jobs, and assessment of the skills of individual workers. Training to address performance problems of specific employees or groups of employees. Training sometimes is needed because employees are not correctly carrying out work tasks. When a performance problem is identified, it should be analyzed carefully to determine if it is caused by lack of job skill or knowledge. These causes can be addressed by training, while other causes, such as poorly designed work processes, are not training problems. Task analysis and individual worker skill assessments may be useful in determining what training is needed. Training current employees in use of new procedures, equipment, and processes. New ways of doing work, such as new computer software, new safety procedures, or a new machine tool, usually require employee training. Such training is designed based on the requirements of the new procedures, equipment, or processes. In this situation, task analysis may be used. Individual worker assessment probably is not relevant, as the workers would not be expected to have the newly required skills and knowledge. Training to develop current employees for promotion and advancement. Training for advancement may be more complex than training provided for other reasons. The requirements of jobs into which advancement is anticipated must be identified. Assessment of the ability of current employees to perform these jobs is complicated by the fact that they do not currently hold these jobs and have had no opportunity to demonstrate their abilities. Special assessment techniques such as assessment centers or tests for supervisory activities may be used.
Three Levels of Company Needs Assessment

Fisher, Schoenfeldt, and Shaw19 identify three levels of needs assessment: organizational analysis, task analysis, and individual assessment. Organizational analysis looks at training within the context of the organization, responding to question such as: ! ! ! ! ! How does the training fit in with the organizations future plans and goals? Where in the organization is training needed? Which units should be trained first? Will this training adversely affect people or units not being trained? Which training programs should have priority?
Job and task analysis is the process of identifying duties of the jobs to be involved in training, and the knowledge, skills, and abilities needed to perform these duties. A
Page 66
variety of task analysis methods are available, but task inventory and critical incident methods are particularly useful. Individual assessment determines which employees should be trained and their current levels of skill and knowledge. This may include use of assessment methods and should pay attention to the workers basic skills as well as the job-specific tasks. The workers may be unable to benefit from training for job-specific tasks if they do not have the necessary level of reading or math skills, for example. Once these three levels of assessment have been conducted, the results can be translated into training objectives for use in developing the training program.
Resources for Company Needs Assessment

Job and Task Analysis Methods. A number of job and task analysis methods are widely used in company needs assessments.20 Examples include: ! DACUM (Developing A CurriculUM) uses a focus group of highperforming incumbent workers in a facilitated storyboarding process to capture their observations on the major duties and related tasks in their occupation. The result is a detailed list of the duties and tasks that make up their job. As its name implies, DACUM is frequently used in curriculum development for training and education programs. Position Analysis Questionnaire (PAQ) is a structured worker-oriented task analysis instrument, using an extensive list of worker-oriented job elements. The PAQ reveals the types of interpersonal relationships required in task performance, which are often overlooked in task inventory analysis. Common-Metric Questionnaire is a structured job analysis instrument for describing work activities.
More information on these job analysis methods is provided in the References section. Individual Assessment Methods. A variety of tools are available for assessing individual workers performance and skills. Good practice calls for using an assessment tool designed to work with the method selected for task analysis, so valid comparisons may be made of the worker and the job requirements. These comparisons will guide the development of company training efforts. The developers of the PAQ and CommonMetric Questionnaire, as well as other task analysis methods, usually also have developed assessment tools.
Page 67
Page 68
REFERENCES
The following references include, where possible, Internet sites providing resources that may be more accessible than printed material. References are also categorized by topic. Classification Systems International Standard Classification of Occupations, 1988, International Labor Organization. More information is available at http://www.ilo.org/public/english/bureau/stat/class/isco.htm International Standard Industrial Classification, Rev. 3, United Nations Statistics Division, 1989. More information available at http://esa.un.org/unsd/cr/family2.asp?Cl=2 National Occupational Classification System, Human Resources Development Canada, 2000. More information is available at http://www.worklogic.com:81/noc/home.html North American Industrial Classification System, United States Office of Budget and Management, 1997. More information is available at http://www.census.gov/epcd/www/naics.html Standard Occupational Classification, 2000, U.S. Office of Management and Budget. See also Standard Occupational Classification (SOC) User Guide on-line at http://stats.bls.gov/soc/socguide.htm. Classification of Instructional Programs, U.S. Department of Education, National Center on Education Statistics, 2000. More information is available at http://nces.ed.gov/npec/papers/cipPreface.html International Standard Classification of Education, United Nations Statistics Division, 1997. More information is available at http://esa.un.org/unsd/cr/family2.asp?Cl=23 Company Needs Assessment A Practical Guide to Needs Assessment, Kavita Gupta, Jossey-Bass/Pfieffer, San Francisco, 1999. Training In Organizations: Needs Assessment, Development, And Evaluation, Irwin L. Goldstein, Brooks/Cole Pub. Co., Pacific Grove, California, 1993. Employer Advisory Committees Edison College Advisory Committee Handbook, http://www.edison.edu/aboutecc/academic_policies.htm
Page 69
Idaho State Board for Professional-Technical Education, Local Advisory Committee Member Handbook. Available at http://www.pte.state.id.us/document/public.htm Kerka, Sandra, Effective Advisory Committees, In Brief No. 17, National Dissemination Center for Career Technical Education, forthcoming 2002. To be available at http://www.nccte.org/publications/index.asp South Carolina Department of Education, Handbook for Advisory Groups in Career and Technology Education, 2000. Available at http://www.sde.state.sc.us/archive/educator/occed/handbook.htm Group Processes American Statistical Association, What are Focus Groups? brochure available at http://www.amstat.org/sections/srms/whatsurvey.html. Altschuld, James W.; and Witkin, Belle Ruth, Planning and Conducting Needs Assessments: A Practical Guide, Chapters 7-9, Sage Publications, Thousand Oaks, California, 1995. Edmunds, Holly, The Focus Group Research Handbook, NTC Business Books, Lincolnwood, Illinois, 1999. Healthcare Forum Foundation, Outcomes Toolkit Information, Delphi Technique, available at http://www.act-toolkit.com/toolkit/collect4.htm International Association of Facilitators, Virtual Library, Available at http://www.iaf-world.org/bibvirtIng.html Southern Cross University, Action Research Resources, Northern New South Wales, Australia, Available at http://www.scu.edu.au/schools/gcm/ar/arp/delphi.html Job Analysis Methods DACUM: Norton, Robert E., DACUM Handbook, Center on Education and Training for Employment, The Ohio State University, Columbus, Ohio, 1997. See also http://www.dacum.org/index.asp PAQ: The PAQ is available from PAQ Services, Inc., at http://www.paq.com/. There is a fee associated with use of PAQ. Common-Metric Questionnaire: This job analysis tool is available from Personnel Systems and Technology Corporation at http://www.pstc.com/index.htmlat. There is a fee associated with use of the Common-Metric Questionnaire.
Page 70
Needs Assessment and Labor Market Analysis Altschuld, James W.; and Witkin, Belle Ruth, Planning and Conducting Needs Assessments: A Practical Guide, Sage Publications, Thousand Oaks, California, 1995. Altschuld, James W., and Witkin, Belle Ruth, From needs assessment to action: transforming needs into solution strategies, Sage Publications, Thousand Oaks, California, c2000. Middleton, John; Ziderman, Adrian; and Van Adams, Arvil, Skills for Productivity: Vocational Education and Training in Developing Countries, published for the World Bank by Oxford University Press, 1993. Texas Occupational Information Coordinating Committee, Connecting the Dots: The Labor Market Information View of Workforce Development, Essays for the Practitioner, Austin, Texas, June 2000. Occupational and Career Information Resources U.S. Bureau of Labor Statistics, Occupational Outlook Handbook, on-line at http://stats.bls.gov/ocohome.htm?H2. U.S. Department of Labor, Employment and Training Administration, Occupational Information Network: O*NET, on-line at http://www.onetcenter.org/ and http://online.onetcenter.org/. The National Career Development Association web site at www.ncda.org provides a variety of resources on career information and counseling, including guidelines on developing career information resources. Statistical and Survey Methods American Statistical Association, Center for Statistics Education, web site at http://www.amstat.org/education/index.html Don A. Dillman, Mail and Internet Surveys: The Tailored Design Method, John Wiley & Sons, New York, 2nd edition, 2000. Don A. Dillman, Mail and Telephone Surveys: The Total Design Method, John Wiley & Sons, New York, 1978. U.S. Bureau of Labor Statistics, Occupational Employment Statistics Survey, see on-line information at http://stats.bls.gov/oeshome.htm?H4 University of British Columbia, Research Methods Resources on the WWW. Available at http://www.slais.ubc.ca/resources/research_methods/index.htm
Page 71
Page 72
ENDNOTES
Managing Vocational Training Systems: A Handbook for Senior Administrators, Vladimir Gasskov, International Labor Office, Geneva, 2000, Module 4.
2 3 1
Gasskov, p. 130.
For an example of labor force forecasts and a brief review of the methods used, see Howard N Fullerton, Jr., "The labor force: steady growth, changing composition," Monthly Labor Review, November 1999, pp. 19-32. Available on-line at http://stats.bls.gov/opub/mlr/archive.htm A convenient inventory of classification systems is maintained by the United Nations Statistics Division at http://www.un.org/depts/unsd/class/invent.htm In the U.S., these crosswalks are available from the National Crosswalk Service Center at http://www.state.ia.us/ncdc/ See for example R.A. Wilson, Forecasting Skills and Requirements at the Company and National Levels, pp. 561-609, in "Training in Europe (2nd report on Vocational Training Research in Europe 2000: Background Report, Volume 2) edited by P. Descy and M. Tessaring, CEDEFOP Reference Series, Luxembourg, Office for Official Publications of the European Communities, 2001; and Texas State Occupational Information Coordinating Committee, Connecting the Dots: The Labor Market Information View of Workforce Development, Essays for the Practitioner, Austin, Texas, June 2000. See http://www.onetcenter.org/ See for example Skill Needs in Britain, 1998, Department for Education and Employment, February 1999, at http://skillsbase.dfee.gov.uk/Reference/Reference.asp?sect=1&page=2, and Workforce Training Needs and Practices of Washington State Employers, Washington Training and Education Coordinating Board, 2000, at www.wa.gov/wtb This section is based on R.A. Wilson, op.cit.. Dillman, p. 27. Dillman, p. 14-23.
7 8
10 11 12
For more information, see the American Statistical Associations pamphlet What is a Margin of Error, on-line at http://www.amstat.org/sections/srms/whatsurvey.html.
This section is based on Planning and Conducting Needs Assessments: A Practical Guide, Belle Ruth Witkin and James W. Altschuld, Sage Publications, Thousand Oaks, California, 1995.
14 15
13
Dillman, page 199-200.
For more information, see the American Statistical Associations pamphlet Designing a Questionnaire, on-line at http://www.amstat.org/sections/srms/whatsurvey.html. For more information, see the American Statistical Associations pamphlet How to Conduct Pretesting, on-line at http://www.amstat.org/sections/srms/whatsurvey.html. Dillman, page 177. Witkin and Altschuld, page 155-159.
16
17 18 19
Cynthia D. Fisher, Lyle F. Schoenfeldt, and James B. Shaw, Human Resource Management, Houghton Mifflin Company, 1996, p. 362-364.
Page 73
20
For a brief review of task analysis methods, see Task Analysis Strategies and Practices, Bettina Lankard Brown, 1998, ERIC Clearinghouse Practice Application Brief, available on-line at http://ericacve.org/docs/taskanal.htm.
Page 74

Needs Assessment Guide

Uploaded by

Copyright:

Available Formats

Needs Assessment Guide

Uploaded by

Document Information

Copyright

Available Formats

Share this document

Share or Embed Document

Sharing Options

Did you find this document useful?

Is this content inappropriate?

Copyright:

Available Formats

Needs Assessment Guide

Uploaded by

Copyright:

Available Formats

Assessing Vocational-Technical Training Needs in the Eastern Caribbean Region

Assessing Vocational-Technical Training Needs

OVERVIEW OF THE NEEDS ASSESSMENT PROCESS PHASE 1. LAYING THE FOUNDATION

PHASE 2. ESTABLISHING THE POLICY FRAMEWORK

PHASE 3. ENGAGING KEY STAKEHOLDERS

PHASE 4. REVIEWING THE ECONOMY

PHASE 5. SELECTING TRAINING NEEDS INDICATORS

PHASE 6. GATHERING ADDITIONAL INFORMATION

PHASE 7. ANALYSIS AND PRESENTATION

PHASE 8. PUTTING NEEDS ASSESSMENT RESULTS TO WORK 63

TRAINING NEEDS ASSESSMENT WITHIN A COMPANY

Assessing Vocational-Technical Training Needs

Why Do Needs Assessment?

What is Needs Assessment?

Assessing Vocational-Technical Training Needs

Needs Assessment Within a Company

Assessing Vocational-Technical Training Needs

Assessing Vocational-Technical Training Needs

Assessing Vocational-Technical Training Needs

OVERVIEW OF THE NEEDS ASSESSMENT PROCESS

Assessing Vocational-Technical Training Needs

Assessing Vocational-Technical Training Needs

PHASE 1. LAYING THE FOUNDATION

Laying the Foundation: Step by Step

Assessing Vocational-Technical Training Needs

Prepare a Project Budget.

Assessing Vocational-Technical Training Needs

Assessing Vocational-Technical Training Needs

Assessing Vocational-Technical Training Needs

PHASE 2. ESTABLISHING THE POLICY FRAMEWORK

Assessing Vocational-Technical Training Needs

Establishing the Policy Framework: Step by Step

Assessing Vocational-Technical Training Needs

Assessing Vocational-Technical Training Needs

Assessing Vocational-Technical Training Needs

PHASE 3. ENGAGING KEY STAKEHOLDERS

Engaging Key Stakeholders: Step by Step

Assessing Vocational-Technical Training Needs

Engaging Key Stakeholders: Step by Step

Parent or student organizations.

Assessing Vocational-Technical Training Needs

Assessing Vocational-Technical Training Needs

Assessing Vocational-Technical Training Needs

Assessing Vocational-Technical Training Needs

Assessing Vocational-Technical Training Needs

Assessing Vocational-Technical Training Needs

PHASE 4. REVIEWING THE ECONOMY Purpose of the Economic Review

Content of Economic Review

Assessing Vocational-Technical Training Needs

Assessing Vocational-Technical Training Needs

Reviewing the Economy: Step by Step

Assessing Vocational-Technical Training Needs

Assessing Vocational-Technical Training Needs

PHASE 5. SELECTING TRAINING NEEDS INDICATORS

A Note about Industries and Occupations

Relating Training Programs and Occupations

Assessing Vocational-Technical Training Needs

Assessing Vocational-Technical Training Needs

Occupations and Skills

Types of Training Needs Indicators