A North American Study of the Entry-Level Veterinary Practitioner

October 2009 Conducted for the National Board of Veterinary Medical Examiners P.O. Box 1356 Bismarck, ND 58502 701-224-0332 www.nbvme.org
prepared by: Applied Measurement Professionals, Inc. 18000 W. 105th St. Olathe, KS 66061

Copyright 2010. PROPRIETARY. All rights reserved. No part of this publication may be reproduced or transmitted in any form or by any means, electronic or mechanical, including photocopy or recording, or any information and retrieval system, without permission in writing from the National Board of Veterinary Medical Examiners (NBVME).

EXECUTIVE SUMMARY
The job analysis described in this report was conducted in 2009 at the request of the National Board of Veterinary Medical Examiners (NBVME). The purpose of the study was to describe the job activities of entry-level veterinary practitioners in sufficient detail to provide evidence of validity for the North American Veterinary Licensing Examination (NAVLE). The NBVME Job Analysis Advisory Committee (AC) conducted the activities necessary to identify job activities and develop the test specifications for the entry-level veterinary practitioner. The AC represented varied species-oriented practices, national regions, and practice settings. All AC members were experts in the duties and activities associated with the profession. The study involved developing a job activity list and a list of clinical diagnoses arranged by species, and then combining them into a survey. The survey was distributed to potential respondents from the United States and Canada. Survey responses were then analyzed. The AC was responsible for the following activities regarding job analysis survey development: a. b. c. d. e. developing a sampling plan for the survey, identifying activities and diagnoses for the survey instrument, determining the survey rating scales, determining the relevant demographic variables of interest, and integrating activities and diagnoses, rating scales, and demographics into a survey instrument.

The draft job analysis survey was distributed to AC members. Based on comments, Applied Measurement Professionals, Inc. (AMP) project staff modified the survey for distribution to a random sample of 4200 veterinarians in the United States and 700 veterinarians in Canada, for a total of 4900 potential respondents. Surveys were distributed electronically to a sample of 4900 practitioners across a variety of specialties. Six hundred seventy five (13.78% return rate) subjects responded with surveys suitable for analysis. Responses to the demographic questions indicated that there were sufficient numbers from relevant groups for subsequent analyses. More than 95% of the respondents felt both the job activity list and the species-specific diagnoses lists adequately or completely addressed the responsibilities of the entry-level practitioner. A relatively large respondent group is associated with minimal rating error. The activity/diagnosis ratings and raters were reliable (consistent). The AC felt that relevant demographic subgroups were adequately represented. Therefore, the job analysis proceeded to the next phase. Survey data were presented to the AC, who subsequently developed and used exclusion decision rules to identify activities and diagnoses appropriate for the NAVLE. Proposed specifications for the examination were constructed from the remaining activities. The resulting examination matrices will be presented to the NBVME for guidance in assembling future NAVLE test forms.

A North American Study of the Entry-Level Veterinary Practitioner 2010 NBVME. All rights reserved.

Page 2

INTRODUCTION
The job analysis described in this report was conducted in 2009 at the request of the NBVME. The purpose of the job analysis was to describe the entry-level veterinary practitioners job in sufficient detail to provide substantial validity evidence for a North American licensing examination, and to ensure that the content of the examination was job-related. The NBVME appointed a Job Analysis Advisory Committee (AC) to assist in the preparation and review of the job analysis survey instrument. The AC developed a comprehensive inventory of activities that practitioners may perform by (1) reviewing the results of a previous job analysis conducted in 2003, and (2) conducting telephone interviews with licensed practitioners. The AC also developed a list of clinical diagnoses that may be addressed in practice. In addition, demographic variables and a rating scale were selected for use on the survey. After pilot testing, the job analysis survey was distributed to a random sample of 4900 veterinarians. Each potential respondent was sent an email that contained a link to the online survey. The returned surveys were analyzed to determine the significance of each activity and each diagnosis to the entry-level veterinary practitioners job. Job Analysis Survey (JAS) data were evaluated to determine the degree of consensus among veterinarians on critical aspects of the job. Data were specifically analyzed to answer the following questions: 1. Which activities are more significant to the entry-level veterinarian? 2. Which clinical diagnoses are more significant to the entry-level veterinarians practice? These questions helped identify the more significant activities and diagnoses from which the content of the examinations can be derived.

A North American Study of the Entry-Level Veterinary Practitioner 2010 NBVME. All rights reserved.

Page 3

METHODOLOGY
Forming the Advisory Committee
The Job Analysis Advisory Committee (AC) was consulted throughout the survey development stages to ensure that expert judgment was available to AMP staff. The responsibilities of the AC are listed below. The members of the AC were experienced veterinarians, all thoroughly familiar with the skills and activities of an entry-level practitioner.
Nicole Azene, DVM (University of Missouri 2005), MS, Baltimore, Maryland Dr. Azene is a postdoctoral fellow in laboratory animal medicine at Johns Hopkins University Terri Clark, DVM (Oregon State 1991), Corvallis, Oregon Dr. Clark is a faculty member at Oregon State University Kerry Ann Collins, DVM (Virginia Tech 2007), Brunswick, Maine Dr. Collins is a small animal practitioner Tom Graham, DVM (University of California Davis 1984), MPVM, Davis, California Dr. Graham is a dairy practitioner Tom Hairgrove, DVM (Texas A&M 1974), DABVP (bovine practice), College Station, Texas Dr. Hairgrove works for the Texas AgriLife Extension Service, and is a member of the NBVME Daphne Hall, DVM (Iowa State 1989), Duluth, Minnesota Dr. Hall is a small animal practitioner Jay Hedrick, DVM (Kansas State 1972), El Dorado, Kansas Dr. Hedrick is a small animal practitioner, and is a member of the NBVME Jessica J. Hudak, DVM (Texas A&M 2008), Charlottesville, Virginia Dr. Hudak is a small animal practitioner Susan Little, DVM (University of Guelph 1988), DABVP (feline practice), Ottawa, Ontario Dr. Little is a feline practitioner, and is a member of the NBVMEs Examination Development Advisory Board Loren Schultz, DVM (Kansas State 1997), MS, DACVPM, Columbia, Missouri Dr. Schulz is a faculty member at the University of Missouri Leah Steinberg, DVM (Tufts 2008), Westborough, Massachusetts Dr. Steinberg is an equine practitioner Tonya Stewart, DVM (North Carolina State 1991), Charlottetown, Prince Edward Island Dr. Stewart operates the Community Practice Service at the Atlantic Veterinary College Jack Wilson, DVM (University of Saskatchewan 1972), Calgary, Alberta Dr. Wilson is a retired small animal practitioner, and is a member of the Canadian National Examining Board

A North American Study of the Entry-Level Veterinary Practitioner 2010 NBVME. All rights reserved.

Page 4

Advisory Committee Responsibilities

1. 2. Provide AMP with current information about the job. Develop the Job Analysis Survey (JAS): a. develop a sampling plan for the survey, b. identify activities (i.e., activities, clinical diagnoses) for the survey instrument, c. determine the survey rating scales, d. determine the relevant demographic variables of interest, and e. integrate the definition, activities, rating scales, and demographics into a survey instrument. Review the final form of the JAS for completeness, relevance to the profession, appropriate language, and clear instructions. Interpret and review survey results, determine activity and diagnosis exclusion criteria, and recommend final examination specifications.

3. 4.

A significant investment of time by the AC ensured a successful job analysis study. We are grateful to each of these professionals for their guidance, expertise and devotion to this complex project.

Developing the Job Analysis Survey

Developing the Activity List With the assistance of AMP project staff, the AC drafted an inventory containing a comprehensive list of activities and clinical diagnoses. The lists were drafted from current examination blueprints, a previous entry-level veterinary practitioner job analysis, and telephone interviews with current practicing veterinarians. After a comprehensive review of all resource materials, a number of additional activities and diagnoses were incorporated into the final survey. Telephone interviews were also conducted with educators, key constituents, and practicing veterinarians throughout the United States and Canada. 40 veterinarians were included on the original list of individuals to be contacted, and of those, 15 were available to participate in a telephone interview. Those individuals interviewed were asked differing questions depending on whether they were practitioners, educators, or key constituents. The questions asked of each group of veterinarians are listed below, and Appendix A includes the full text of responses from the interviews. In summary, all responses were reviewed, and their implications for the wording of the existing and additional activities and diagnoses were considered in the development of the final survey.
Questions Asked of Each Group of Veterinarians Educator 1. What courses do you currently teach? 2. In what types of settings do you teach? (e.g., classrooms, laboratories, teaching hospital) 3. In your opinion, what are the most important activities that entry level veterinarians need to be able to perform in a competent manner? 4. In what research are you directly involved that relates to the practice of entry-level veterinary medicine? 5. The NAVLE is a comprehensive objective examination designed to help licensing boards ensure that veterinarians demonstrate a specified level of knowledge and skills before entering private clinical practice. What are your recommendations on how the NAVLE should be structured to best meet that objective?

A North American Study of the Entry-Level Veterinary Practitioner 2010 NBVME. All rights reserved.

Page 5

Key Constituent 1. In your opinion, what are the most important activities that entry level veterinarians need to be able to perform in a competent manner? 2. The NAVLE is a comprehensive objective examination designed to help licensing boards ensure that veterinarians demonstrate a specified level of knowledge and skills before entering private clinical practice. What are your recommendations on how the NAVLE should be structured to best meet that objective? 3. What is your professional affiliation (e.g., professional association, licensing board, school administration, former board member)? Practicing Veterinarian 1. What animal species does your practice involve? 2. In what types of settings do you practice? (e.g., clinic, hospital, ambulatory practice, house call practice, referral practice, institution) 3. What are you major responsibilities? 4. What activities do you spend most of your time doing? 5. What are the most important medical conditions you see?

The final survey document consisted of 49 activities presented in content order and 1153 clinical diagnoses presented in alphabetical order, organized by species. Survey respondents were allowed to suggest additional activities and diagnoses. The complete JAS and cover letter are available through NBVME upon request. Selecting Rating Scales The AC also assisted in the development of the rating scales used in the job analysis study. The scales were based on the scales used in the previous job analysis conducted in 2003. Separate scales were developed for the activity and diagnoses sections of the survey. A 5 point significance scale was selected by the AC to include on the survey to rate the activities. A 4 point significance scale, similar to the one used to rate activities, was developed for the diagnoses section. These scales were designed to identify the most significant activities to achieving entry-level practitioners job objectives. Such information was necessary to demonstrate that the examination will measure significant aspects of the job and cover appropriate content. The following scales were used:
In your working environment, considering both importance and frequency, how significant is this activity to competent and effective performance for the first-year practitioner with entry-level skills? 4 = Extremely Significant 3 = Quite Significant 2 = Somewhat Significant 1 = Not Significant 0 = Not Performed In your working environment, how significant is this clinical diagnosis or problem to competent and effective performance for the first-year practitioner with entry-level clinical skills? 4 = Extremely Significant 3 = Quite Significant 2 = Somewhat Significant 1 = Not Significant

A North American Study of the Entry-Level Veterinary Practitioner 2010 NBVME. All rights reserved.

Page 6

Selecting Background Information Questions This section was designed to gather information about the respondents demographic characteristics. Demographic questions were used to help the AC evaluate potential bias in the respondent group. Therefore, the following information about the survey respondents was available: Geographic region Practice emphasis Years of experience Professional role Educational degrees Specialty board certifications Gender Integrating the Definition, Activities, Rating Scales, and Demographics into a Survey Following the AC meeting, survey components were compiled into draft form. The draft survey was reviewed by the AC. The pilot job analysis survey was distributed to the AC members for review and comment. The purpose of the pilot study was to determine (1) if any important activities or diagnoses were missing from the survey, (2) if the directions were clear, and (3) if the rating scales were easy to use and understand. The NBVME Executive Director reviewed comments from the pilot study participants and made any necessary modifications to the survey prior to distribution. Sample Selection The AC developed a sampling plan in order to obtain information from a diverse group of veterinary practitioners throughout the United States and Canada. Email addresses for veterinarians in the United States were obtained from the American Veterinary Medical Association (AVMA) member database. A selection pool consisting of all veterinarians in private, academic, and government practice was created. 4200 email addresses were chosen randomly, to ensure a diverse and representative sample with regard to species focus, employment function, medical discipline, employer type, geographic location, age, school and year of graduation, and gender. Email addresses for Canadian veterinarians were obtained from the Canadian Veterinary Medical Association (CVMA). The selection pool included veterinarians from both English and French speaking databases. From the English speaking database, every 7th member was selected until 620 potential respondents were obtained. Every 4th member from the French database was selected until 80 potential respondents were obtained.

A North American Study of the Entry-Level Veterinary Practitioner 2010 NBVME. All rights reserved.

Page 7

RESULTS
Return Rate and Sample Size
Of the 4900 invitations emailed, 675 respondents completed the survey for a response rate of 13.78%. Though typical for an unsolicited survey, the response rate is likely underestimated because both the AVMA and CVMA did not track undeliverable or opt out messages. Therefore, it is not possible to calculate an exact response rate. Also, the AC suggested that because the survey was not translated from English to French, this may have contributed to the lower response rate from the Canadian population. A general approach was incorporated to evaluate the standard error of the ratings. An approximate standard error was calculated for the rating scale and determined to be .04, based on the 675 respondents. This indicates that ratings were relatively stable and reflective of the population of veterinary practitioners.

Activity and Species Rating Reliability Estimates

To find the extent to which activities and diagnoses were consistently rated within each survey section, a statistic known as coefficient alpha (Norusis, 1992, p. 204; Hopkins, Stanley & Hopkins, 1990, p. 133-134) was used. Coefficient alpha is an estimate of the amount of error reflected by the scores associated with the instrument. Higher estimate values (e.g., .90 or higher) reflect smaller amounts of error. To determine the extent to which the respondents were consistent in rating inventory activities and diagnoses, a statistic known as the intraclass correlation (Guilford, 1956) was used. Separate reliability estimates were calculated for the sections of the survey and are displayed below. Since a maximum reliability coefficient is represented by a value of 1.00, and the total reliability estimates for the whole activity list were . 99 (intraclass) and .957 (alpha), the respondents' activity ratings were considered statistically reliable. As can be seen, reliability estimates for clinical diagnoses are also statistically reliable. Based on these data, it is likely that a different sample from the same population would have produced similar activity ratings. Activities Reliability Estimates Content Category *Number of Number Respondents of Activities 619 478 431 12 25 12 Reliability Alpha 0.857 0.930 0.883 Intraclass 0.990 0.992 0.992

I. Data Gathering and Interpretation II. Health Maintenance and Problem Management III. Professional Behavior, Communication, and Practice Management Total

49

*Note: Respondent subsets of varying size are shown because complete data matrices were required to calculate reliability estimates and some respondents did not rate some activities; therefore, they are excluded from all analyses. When values are below 50, caution should be used when making interpretations.

A North American Study of the Entry-Level Veterinary Practitioner 2010 NBVME. All rights reserved.

Page 8

Species Reliability Estimates Species Category Canine Feline Pet Birds Other Small Animals: Excluding Birds Bovine Porcine Ovine/Caprine Cervidae Equine Camelidae Poultry Ratites *Number of Respondents 287 306 49 57 57 18 48 17 68 27 9 7 Number of Diagnoses 234 175 52 118 148 68 69 16 159 46 40 28 Reliability Alpha 0.990 0.989 0.979 0.993 0.988 0.980 0.982 0.858 0.990 0.961 0.992 0.974 Intraclass 0.995 0.995 0.951 0.930 0.968 0.818 0.955 0.706 0.971 0.904 0.602 0.689

*Note: Respondent subsets of varying size are shown because complete data matrices were required to calculate reliability estimates and some respondents did not rate some diagnoses; therefore, they are excluded from all analyses. When values are below 50, caution should be used when making interpretations.

Activity List Adequacy

After respondents finished rating the activity list section of the survey, they were asked to respond to the following question: how well did this section cover the critical activities for your role as a licensed veterinarian?. They could select one of the following responses: inadequately, adequately, or completely. One hundred and seventeen respondents (17.3% of total respondents) did not answer this question. Of 558 professionals responding to the question, 97.7% thought the survey completely or adequately described the activities performed by the licensed veterinarian. N 558 Completely 33.5% Adequately 64.2% Inadequately 2.3%

A North American Study of the Entry-Level Veterinary Practitioner 2010 NBVME. All rights reserved.

Page 9

Diagnosis List Adequacy

Respondents had an opportunity to rate how comprehensive they thought the clinical diagnosis list section of the survey was after each species section, and each respondent was given the choice of inadequately, adequately, and completely. Species Canine Feline Pet Birds Other Small Animals Bovine Porcine Ovine/Caprine Cervidae Equine Camelidae Poultry Rattites N 453 428 60 131 95 25 60 17 101 31 10 7 Completely 49.9% 47.0% 30.0% 25.2% 36.8% 16.0% 31.7% 5.9% 32.7% 19.4% 30.0% 14.3% Adequately 49.9% 53.0% 65.0% 71.0% 63.2% 76.0% 65.0% 82.4% 65.3% 71.0% 70.0% 85.7% 8.0% 3.3% 11.8% 2.0% 9.7% 5.0% 3.8% Inadequately 0.2%

Demographic Analyses
Background information was collected from the total respondent group. more detailed information. See Appendix B for

Respondents were distributed across eleven geographic regions of the United States and Canada. Practice areas included Canine (423 respondents), Feline (423), Exotic (99), Equine (85), Beef (66), Pet Bird (53), Small Ruminant (48), Dairy (41), Swine (16) and Poultry (3). Over 80% of the total sample indicated their practice emphasis as canine or feline (or both). The largest group of respondents (43.7%) had more than 20 years of experience. Other categories included 11-20 years (18.8%), 6-10 years (11.8%), 2-5 years (17.6%), and less than two years (8.1%). The majority of respondents, 43.8%, indicated their professional role as a practice owner. Other roles included associate/employee (40.1%), relief veterinarian (5.6%), other (4.3%), university (2.9%), intern/resident (1.9%), government (0.8%), and industry (0.6%).

A North American Study of the Entry-Level Veterinary Practitioner 2010 NBVME. All rights reserved.

Page 10

All respondents held a DVM, VMD, or equivalent educational degree (483, or 100% of the total sample). Forty-eight respondents also listed MS or equivalent, and 12 listed PhD or equivalent. Of the respondents who held additional Board Certification(s), seven of these were ACVS (25%), and seven were ACVIM (25%). Sixteen respondents listed eight other specialty boards. Respondents were fairly equally divided between males and females. Females (56.2%) had slightly more respondents than males (43.8%).

Mean Activity and Species Ratings

To determine which activities and diagnoses were more significant, descriptive data were calculated for each activity and diagnosis, and the AC used the results to determine which activities or diagnoses should remain on the final examination specifications. As noted in the survey instrument (available upon request), the significance scales had values ranging from 1 (Not significant) to 4 (Extremely significant). The scale used for the list of activities also included a value of 0 for "Not performed". Appendix C shows the recommendations from survey respondents for distribution of items on the examination by activities and species. The AC reviewed the data for each activity and diagnosis. They concluded that the ratings obtained from the job analysis survey were in agreement with their judgments about the job. Consequently, the AC also concluded that the job analysis survey data adequately defined the entry-level practitioners job in North America. Moreover, the AC judged the results sufficient for the purpose of delineating the structure and content of a North American licensing examination. It is critical that the test specifications reflect the responsibilities of those who might be eligible to take the examination. Therefore, the proposed test specifications and resulting examination content should include activities and diagnoses considered significant to the job by those for whom the examination was intended. To ensure this, the mean rating for each activity and diagnosis was evaluated for two survey respondent subgroups: geographic region and years of experience. The final information used by the committee to determine activity or diagnosis eligibility was respondent comments. After review of this information, the AC determined that there should be edits to the detailed content domain. Due to comments from survey respondents or from discussion of emerging fields, certain diagnoses were either merged together or added to the detailed content domain. The AC was encouraged to consider how best to limit the content eligible for the test specifications to only the broadly performed critical activities or diagnoses. Therefore, the AC adopted decision rules to identify activities or diagnoses eligible for assessment. Eligible activities and diagnoses were found sufficiently significant by the sample and resulting subgroups and met the threshold criteria. Appendix D lists all AC-approved diagnoses, grouped by animal species and organ system. The AC used data collected from the survey to develop recommended test specifications, listing the number of items to include in each major content area. The goal was to distribute items in accordance with observed working patterns across the various activity and animal species areas. These recommendations were presented to the NBVME for consideration in developing revised NAVLE test specifications. Appendix E lists the final NAVLE test specifications, by activities and species, approved by the NBVME.

A North American Study of the Entry-Level Veterinary Practitioner 2010 NBVME. All rights reserved.

Page 11

Cognitive Complexity
After the number of items in each content area and each species was determined, the next step involved determining the cognitive complexity for each activity. A complexity scale was designed to determine the cognitive level individual activities were typically performed. The information provided a basis for matching test item complexity to job complexity. The complexity scale is based on Blooms Taxonomy of Educational Objectives (1956, pp. 201-207) and is presented below. The AC assigned a cognitive rating to each activity using this scale. What typical level of cognition is necessary for veterinary practitioners to properly perform this activity? Rating Description 1 2 Recall - requires only the identification, recall, or recognition of isolated information, such as specific facts, generalizations, concepts, principles, or procedures. The information generally does not vary relative to the situation. Application - requires comprehension, interpretation, or manipulation of limited concepts or data, in which the response or outcome is situationally dependent, but not overly complex (e.g., application of knowledge which varies based on patient characteristics and environment). Activities that require candidates to recognize elements and relationships among data and to classify, explain, or differentiate are usually application level. Analysis/Evaluation - requires the integration or synthesis of a variety of concepts or elements to solve a specific problem situation (e.g., evaluating and rendering judgments on complex problems with many situational variables). each each level level

To determine the mean cognitive level (Recall, Application, and Analysis) assigned to major and sub-content activity category, a mean cognitive level rating was calculated for category. The mean cognitive levels were used to determine the suggested cognitive distribution of items on the exam. The following table displays the suggested cognitive distribution by percent of items on the exam based on the mean cognitive level ratings.

Cognitive Distribution of Items 1. If the mean cognitive level for a section is below 1.5, then 100% of items in that section are recall. 2. If the mean cognitive level for a section is between 1.5 and 2.1, then 40% of the items in that section are recall and 60% of the items are application. 3. If the mean cognitive level for a section is between 2.1 and 2.5, then 20% of the items in that section are recall, 60% of the items are application, and 20% are analysis. 4. If the mean cognitive level for a section is greater than 2.5, then 20% of the items in that section are recall, 20% are application, and 60% of the items are analysis. The AC reviewed the results, and based on their expert judgment, made minor changes that they felt better reflected current practice. The following tables display the recommendations from the survey respondents and the AC-adjusted recommendations for the NAVLE.

A North American Study of the Entry-Level Veterinary Practitioner 2010 NBVME. All rights reserved.

Page 12

Recommendations by Activity
Respondent AC # # Items Items 124 31 140 35 Cognitive Level Recall 30 7 Appl. 33 7 Analysis 77 21 Totals 140 35

Content Area (number of activities) I. Data Gathering and Interpretation (12) A. Obtain History, Perform Physical Examination, and Evaluate the Environment (3) B. Determine the status (normal/ abnormal) of the animal(s) and/or environment by (2) C. Record pertinent information in a legible and orderly system of medical records to promote retrieval and sharing of information (1) D. Develop a problem list, and a differential diagnosis list (5) E. Establish a working or final diagnosis or conclusion (1) II. Health Maintenance and Problem Management (24) A. Identify and Evaluate Prevention, Treatment, and Management Options (11) B. Implement Plan of Action (8) C. Assess Outcome (5) III. Professional Behavior, Communication, and Practice Management (11) Totals

21

24

14

24

10

11

11

52 10 112 52

59 11 140 64

12 2 27 12

12 2 28 13

35 7 85 39

59 11 140 64

37 23 64

47 29 20

9 6 4

9 6 12

29 17 4

47 29 20

300

300

61

73

166

300

A North American Study of the Entry-Level Veterinary Practitioner 2010 NBVME. All rights reserved.

Page 13

Recommendations by Species Species % Recommended # of Respondent AC Recommended by Respondents Recommended Items Test Specifications (300 Item Exam) Canine Feline Pet Birds Other Small Animals Bovine Porcine Ovine/Caprine Cervidae Equine Camelidae Poultry Rattites 24.69 21.43 3.55 4.16 12.83 6.23 4.68 1.90 13.76 2.10 3.37 1.31 74 64 11 13 38 19 14 6 41 6 10 4 68 68 12 14 40 14 19 6 45 8 6 0

A North American Study of the Entry-Level Veterinary Practitioner 2010 NBVME. All rights reserved.

Page 14

CONCLUSIONS
The job analysis described in this report was undertaken to provide evidence supporting the content validity of the North American Veterinary Licensing Examination (NAVLE). The study was conducted to determine and comprehensively describe the entry-level veterinary practitioners job, to evaluate this description through the ratings of job experts, and to define the content areas that should be assessed on the NAVLE. The NBVME formed a Job Analysis Advisory Committee (AC), who prepared comprehensive lists of activities describing the job and diagnoses treated on the job by species. A representative sample of practicing veterinarians completed the survey. The AC reviewed the survey results and used the survey activity ratings to provide a foundation for developing test specifications directly related to the important activities that practitioners perform. The AC developed a framework that can be used to establish test specifications using the results from this study, their collective professional judgment, and direction from AMP project staff. This framework outlines the content domain and the suggested distribution of items across critical content categories. The specifications may be used to guide test development and provide content-related evidence that examination scores relate to the job. This evidence may then be used to support valid inferences from examination scores that candidates are able to perform the job of a practicing veterinarian.

REFERENCES
Bloom, B. (Ed.) (1956). Taxonomy of Educational Objectives, Handbook I: The Cognitive Domain. New York: David McKay Company, Inc. Guilford, J. P. (1956). Fundamental Statistics in Psychology and Education. New York: McGraw Hill. Hopkins, K.D., Stanley, J.C., Hopkins, B.R. (1990). Educational and psychological measurement and evaluation, (7th edition). New Jersey: Prentice Hall. SPSS/PC+ professional statistics version 17.0. (2009). Chicago: SPSS Inc.

A North American Study of the Entry-Level Veterinary Practitioner 2010 NBVME. All rights reserved.

Page 15