OPI Testing at the Defense Language

Institute Foreign Language Center

Thomas S. Parry, Ph.D.
Proficiency Standards Division
Directorate of Testing
 Workshop Outline
Part I
• OPI program overview
• OPI tester certification
• Requirements for testers to remain in active status
• OPI tester decertification
• OPI relook and score resolution procedures
• Current and planned PSD projects and initiatives
Part II
• The OPI conducted at DLIFLC
• Sample OPI tasks at levels 2 and 3
 OPI Program Overview

• Program Mission: Provide appropriate interpretation and

uniform implementation of the Interagency Language
Roundtable Skill Level Descriptions for Speaking to
accomplish valid and reliable assessment of spoken
language proficiency at DLIFLC and at field sites

Accomplished through training, certifying and maintaining 
select DLIFLC faculty to serve as collateral duty OPI testers in 
languages taught at the Institute. In addition, PSD provides 
orientation and awareness training for all DLIFLC faculty in the 
ILR guidelines as well as specialized training on advanced‐level 
standards and associated assessment techniques
 OPI Program Overview

• Program Vision: Using the OPI, conduct consistently fair

and accurate assessment of oral language proficiency in
meeting all DLIFLC and field testing requirements

Fair assessment is accomplished through effective 
elicitation of appropriate proficiency‐based tasks
Accurate assessment is accomplished by assigning the 
appropriate rating based on proficiency 
 Languages Tested via the OPI

• Category I: (24 weeks) French, Italian, 

Portuguese, Spanish,
• Category II: (36 weeks) German, Indonesian 
Category III: (47 weeks) Dari, Hebrew, Hindi,  
Farsi, Kurdish, Russian, Serbian‐Croatian, 
Tagalog, Turkish, Urdu 
• Category IV: (63 weeks) Arabic MSA, Levantine, 
Iraqi, Chinese, Japanese, Korean, Pashto 
**********
• English: For job applicants
5
 Who Must Take the OPI?

• All DLI students must take an Oral Proficiency 
Interview (OPI) as part of the DLPT battery 
• Faculty applicants must take OPIs in both EN  
(minimum score L2) and TL (minimum score 
L3) to be prequalified for hiring
• Military personnel who must qualify for MOS 
readiness or who want to receive “Language 
Proficiency Pay” bonuses
The DLI Student Population

SIGINT: 85‐90%
Voice Interceptors 
Cryptology Linguists

HUMINT: 10‐15%
97E Interrogators
Foreign Area Officers
 Graduation Requirements

• All students must complete the Defense 
Language Proficiency Test (DLPT) battery 
in order to graduate 
– Listening: administered in lab (3 hrs.)
– Reading: administered in lab (3 hrs.) 
– Speaking: OPI (face‐to‐face or telephonic 30‐45 mins.) 
 Graduation Requirements

• Exit proficiency levels
– Basic Course – L2/L2/L1+ ‐> (L2+/L2+/L2)
– Intermediate – L2+/L2+/L2
– Advanced – L3/L3/L2+
 OPI Volume, Statistics and
Facts for FY 2014
• Number of DLIFLC OPIs conducted: 3,010
– Total students tested: 2760
– Total FPS tested: 133
• Number of certified DLIFLC OPI testers: 351
– Some testers are dual certified and test in more than
one language
• Tester attrition: 27
• Tester gain: 39
Total Number of DLIFLC OPIs
FY 2010-2014
Number of OPIs Conducted for
Category 4 (FY 2010-14)
Number of OPIs Conducted for
Category 1-3 (FY 2010-14)
Number of OPIs Conducted
by ACTFL/LTI FY10-14
 OPI Tester Certification

• Eligibility to serve as an OPI tester

– Demonstrated ability to multitask
– Effective organizational and teamwork skills
– Solid academic foundation in language/linguistics
– Has level 3 or higher proficiency in the language to be
tested
– Must not be in an administrative position
 OPI Tester Certification

• Three phases leading to tester certification:

– Phase 1: Screening and selection of prospective OPI
testers
– Phase 2: The OPI Tester Training and Certification
Workshop
– Phase 3: Post-workshop provisional certification
mentoring and training
 OPI Tester Certification

• Phase 1: Screening and selection of prospective

OPI testers
- Factors considered when screening prospective tester
candidates:
– Demonstrated ability to multitask
– Evidence of effective organizational and teamwork skills
– Currently not serving in an administrative position
– Has a solid academic foundation in language/linguistics
– Has speaking ILR level 3 or higher in the language to be tested
 OPI Tester Certification
• Phase 2: The OPI Certification Workshop
– Offered as needed based on demand for testers
– Conducted over a period of 12-15 days
– Generally conducted for specific languages but multi-
language workshops are also possible
– Admits 8-10 participants maximum per iteration
– Academically demanding and rigorous curriculum
• One week devoted to theoretical and foundational testing concepts
including the ILR
• Two weeks of practicum experiences
– Demands 100 percent attention and effort
– Required for provisional certification
 OPI Tester Certification

• OPI Tester Certification Workshop Outcomes

– Upon completion of the OPI Tester Certification
Workshop, participants receive one of the following
certificates depending on their status:
• Certificate of Provisional Certification, valid for one year from date
of workshop completion, for those who complete all workshop
requirements at standard
• Certificate of Course Completion for those who complete the
course but fail to fully meet all workshop requirements and do not
obtain provisional certification
• Certificate of Attendance for those who fail to complete the full
course and issued only for the dates of attendance
 OPI Tester Certification
• Phase 3: Post-workshop provisional certification
mentoring and training
– Normally 12 months from completion of initial tester
certification workshop and may be less
– Provisionally certified testers actively conduct OPIs and
receive extra attention in the first year of testing
– Provisional certification requirements:
• Successful elicitation and rating of 8-10 OPIs with close review
and monitoring by PSD staff
• Periodic one-on-one test review/mentoring sessions with PSD
staff (varies by language and need)
• Participation in at least one full cadre norming session
• Completion of assigned third ratings with PSD review
 Requirements for Testers to
Remain in Active Status

• All fully certified (journeyman) testers must meet

minimal annual requirements to retain their certified
status:
– Receive periodic test review and individualized training
– Meet minimal annual testing requirement
– Participate in special group norming sessions as required
– Complete annual OPI tester re-certification training
– Annually review and sign OPI Tester Code of Conduct
document
 Tester Maintenance and
Quality Control

Test Quality 
Control

Annual 
Test Review  OPI Tester 
Recertification 
Program Performance
Training
 Test Review Program

• Provides for one-on-one feedback sessions from

an OPI Education Specialist
• Sessions based on OPIs the tester recently
conducted
• Identifies strengths and weaknesses of testing
• Areas of improvement are clarified
• Sessions are scheduled up to four times per year
for each tester
• Sessions are approximately two hours in length
 Test Quality Control

• Up to 20% of all OPIs are randomly-selected for

third rating and analysis
• The analysis of OPIs is critical because it identifies
– areas of weakness to address in one-on-one Test
Review sessions
– trends in the testing cadre, by language, that can inform
the content of recertification and norming workshops
• The focus of third-rating analysis is on
– interview structure
– elicitation
– rating accuracy
 Annual Recertification Training

• Offered annually for each language-specific testing cadre

• Conducted for two or three days depending on need
• Admits 10-20 testers per iteration
• Number of iterations is dependent on size of the testing
cadre for a given language
• Emphasis is “hand’s-on” with practice OPIs and third rating
activities providing practice and evaluation of elicitation
techniques, appropriate test structure and rating accuracy
• Issues related to higher-level testing are also discussed
• Each workshop is tailored to to specific language groups
informed by the OPI quality control process
 Minimal Annual Testing
Requirement
• Minimal annual testing requirement:
– Designed to equitably distribute workload, prevent tester
burnout and enhance cadre norming in each language
– Applies specifically to testers in the highest enrollment/most-
frequently-tested DLIFLC language programs
– Does not apply to low-enrollment language programs
– Testers must test up to 70% of the average number of
testing events per tester in their language each year
– Testing events include elicited OPIs and third ratings
– Testing averages are revised each year by language
 The OPI Code of Conduct

• OPI testers sign a document and agree to:

– Adhere strictly to official testing procedures
– Use only the ILR descriptions for speaking and the rating factor
grid to rate the performance of any test
– Rate independently and not discuss with their partner
– Never discuss the content of an OPI with anyone at any time
– Never seek information pertaining to other OPIs
– Notify PSD of any attempt to solicit information about a test,
negotiate or influence ratings
– Consult with PSD about any questions or concerns
– Never bring a cell phone into the testing room
– Attend Test Review sessions as required
 OPI Tester Decertification

• Grounds for tester decertification include:

– Tester misconduct
– Consistent inability to conduct tests per established procedures
– Consistent inability to rate according to the ILR
– A pattern of justifiable examinee complaints or excessive retests
– Ongoing/repeated non-availability to test as required
– Demonstrated inability/unwillingness to cooperate with other
testers
– Unwillingness or inability to take feedback
– Reduction in testing requirements
– Active tester is moved into an administrative position
 OPI Tester Decertification

• Documentation for decision to decertify an OPI tester

– A decision to decertify an OPI tester is based on
documented evidence of patterns of difficulty in testing
and/or rating correctly, not on random or isolated errors
– Testers who experience difficulties in elicitation or rating
OPIs correctly are de-certified only if they fail to respond to
remedial training
– Testers who commit ethical or other serious conduct
violations may be de-certified immediately.
 OPI Tester Decertification

• Documentation for decision to decertify an OPI tester

– To ensure all decisions are made fairly and on the basis of
accurate information, the following documented aspects of
tester performance are carefully reviewed:
• OPI ratings
• Response to tester support sessions
• Response to annual re-certification/norming sessions
• Third/fourth ratings conducted by a given tester
• Third/fourth ratings conducted on a given tester’s OPIs
• Retests necessitated by a tester’s errors/misconduct
• Substantiated complaints by students/FPS applicants, etc.
 OPI Tester Decertification

• Temporary decertification and leaves of absence

– Occasionally, OPI testers may, with legitimate justification,
request a temporary, short-term leave of absence from
conducting OPIs
– The following are the most justifiable reasons to grant a
temporary leave of absence as an OPI tester:
• Temporary duty (TDY) assignment for three to six months duration
• Exigencies of mission requiring a temporary re-assignment, detail or
reprioritization of workload
• Extended absence from work due to a severe health problem,
critical family emergency and/or extended leave
 OPI Relook and Score
Resolution
Cases Situations Action
Splits across a major border 0+/1, 1+/2, 2+/3, 3+/4, 4+/5 Third rating
Splits in the following ranges 2/2+, 3/3+, 4/4+ Assign lower 
rating only
Graduation requirement  ‐ Any 1/1 Third rating
cases  ‐ Any 1/1+ split
‐ Intermediate course student lower than 2
‐ Advanced course student lower than 2+
‐ 97E student receiving lower than 2
Relooks ‐ If speaking score is higher than listening    PSD Dean will 
score by one skill level or more make decision 
‐ Written requests by examinee endorsed by  on whether to 
service unit commander third rate
Retests ‐ If testers failed to elicit a ratable sample PSD Dean will 
‐ If a test did not record and the recording is  make decision 
needed for resolution on retest
‐ Special problems on a case‐by‐case basis
 Current and Planned
PSD Projects

• Best practices in OPI dialect assessment

• Alternative methods of assessing speaking
proficiency at levels 3+ and higher
• Collaborative tester and tester trainer norming with
ACTFL
• Explanatory notes for the ILR speaking guidelines
• Produce all new set of training videos in English and
key languages
The OPI Conducted at DLIFLC
 Overview of the OPI

• General overview
• Student OPI testing at DLIFLC
• The OPI assessment process
• OPI rating criteria
 The OPI –General Overview

The OPI
• is a global test that goes beyond course content and 
measures ability to handle real‐world tasks by 
simulating them 
• measures language proficiency holistically by 
determining patterns of strengths and weaknesses
• is an interactive and adaptive test 
• is based on a set of assessment criteria and its 
structure is standardized
 The OPI –General Overview

The OPI
•is a valid and reliable assessment of spoken language 
ability when the following conditions are met:
- The sample obtained accurately reflects an examinee’s 
proficiency and is ratable
- The rating is based on the ILR criteria
 The Purpose of OPI Testing

• The twofold purpose of OPI testing is to find 
out what the examinee
- can do in the TL
- cannot do in the TL
 What the OPI Measures

• The OPI measures general spoken 
language proficiency, which includes 
- daily life transactions
- social interactions
- performance of work‐related and 
professional tasks
 What the OPI does
NOT Measure

• The OPI does NOT measure
– The ability to perform job‐specific tasks
– The specific knowledge of an individual
– Discrete points, language‐specific knowledge
– Achievement
– The education level of an individual
 OPI Test Modalities

• Face‐to‐face

• Telephone

• Video Teleconferencing
 OPI Content and Activities

• Each OPI consists of
– Conversation on a range of topics which are 
familiar and, in some cases, relatively 
unfamiliar to the examinee (according to 
appropriate ILR level descriptions)
– Role‐plays in the target‐culture setting which 
require language that cannot easily be elicited 
in a conversation
– The performance of specific tasks
 The OPI Structure
Four Phases and Three Perspectives

Four Phases Warm-Up Level Checks Probes Wind-Down

Three Iterative Process

Perspectives
Proves to examinee Proves to examinee Returns examinee to
Psychological Relaxes examinee what she can do what she cannot do level at which she is
comfortable

Reacquaints examinee Checks for tasks and Checks for tasks and Chance to check that
Linguistic with language, if content that examinee content which the iterative process
necessary handles with greatest examinee performs is complete
accuracy with least accuracy

Provides testers with Finds the examinee’s Finds level at which No new information
Evaluative preliminary indication speaking level examinee can no for evaluation
of level of examinee’s longer speak
speaking skills accurately
 The Core of the OPI
Required Tasks by Level
Level 0+ Level 1  Level 2  Level 3  Level 4  Level 5 

10 Subject Areas 1. Simple Short  1. Past Narration 1. Support Opinion 1. Support  All  L 4  Tasks  

Conversation Opinion + 
zBasic Objects      2. Present  2. Abstract Topic
zMonths 2. Role Play Narration 2. Abstract  1. High Level 
3. Hypotheses
zBasic Colors               Topic Colloquialisms
3. Examinee Ask  3. Future 
4. Role Play 
zTime
Qs  Narration 3. Hypotheses 2. (Information 
(Unfamiliar Situation)
zClothing                    
4. Description 4. Role Play  Pass) 
zWeather
zDate                           (Formal)
5. Instruction
zWeekdays 5. Role Play 
6. Reporting Facts
zFamily Members     (Informal )
zYear 7. Role Play +C 

Proficiency Standards Division DLIFLC
April, 2014  44
 OPI Assessment Criteria

• Task
- What can the person do in the language?
• Content/Context
- On which topics and in what settings?
• Accuracy 
- How well can they do it?
• Text Type
- How much language is produced and in what 
type?
 ILR Base Levels

Base Levels: 0 ‐ 5
• Examinees at a given base level 
perform ALL the tasks required by that 
level and minimally meet the criteria of 
that level
 ILR Plus Levels

Plus Levels: 0+ ‐ 4+
• Examinee performs with a significant 
increase in proficiency over the base level 
• Shows evidence of proficiency at the next 
higher base level, but cannot sustain 
proficiency at that level 
 Graphic Depiction of Base
Levels and Plus Levels

level 5 range

Level 4+ base level description 5

level 4 range

base level description 4

Level 3+

level 3 range
base level description 3
Level 2+
level 2 range
base level description 2
Level 1+ level 1
range
Level 0+ base level description 1
0
base level description 0
Increase in Breadth and Depth
across ILR Levels

Level 5 range
base level description 5

Level 4 range
base level description 4

Level 3 range
base level description 3
Level 2 
range
Level 1  base level description 2
range
base level description 1
0
base level description 0
Proficiency Standards Division DLIFLC
 Relationship Between the
ILR and ACTFL Scales

ACTFL uses the derived

academic scale shown
here as well as the ILR.
The OPI Assessment
Process

Tester 
assessment 

Examinee  The rate-as-you-

rating go process
 Example OPIs Levels 1-3

• OPI topic domains at each level
• The ILR Skill Level Descriptions ‐ Speaking
• OPI basic rating factors (Levels 0+ to 3)
‐ What a speaker can do 
‐ How well a speaker can do it
OPI Topic Domains at Each Level
 L5

L4

L3 

L2

L1: self, 
family, 
survival
 L5

L4

L3 

L2: limited 
work, concrete

L1: self, 
family, 
survival
 L5

L4

L3: societal issues, 
abstract

L2: limited 
work, concrete

L1:self, 
family, 
survival
 L5

L4: highly abstract, 
tailored language

L3: societal issues, 
abstract

L2: limited 
work, concrete

L1:self, 
family, 
survival
 L5: HAWENS

L4: highly abstract, 
tailored language

L3: societal issues, 
abstract

L2: limited 
work, concrete

L1: self, 
family, 
survival
The ILR Skill-Level Descriptions

Interagency Language Roundtable 
Language Skill Level Descriptions

http://www.govtilr.org/Skills/ILRscale2.htm
 OPI Basic Rating Factors

Delivery

Grammar Text Type

Examinee  Socio‐
Vocabulary performance linguistic
 Hypothesized Relative
Contribution Model
• Level 5: Delivery is that of a highly articulate, well‐
educated native speaker (all factors line up equally)
• Level 4: Emphasis is on sociolinguistic variation and 
appropriateness
• Level 3: Structures are controlled
• Level 2: Text type is minimally cohesive
• Level 1: Vocabulary is sufficient for simple conversations 
on familiar topics
• Level 0+: Word/phrases are limited to rehearsed 
utterances to satisfy immediate needs
 Level 0

• No Functional Ability in the language
– Occasional isolated words, but cannot 
use them in a functional manner 
– Unable to communicate with the 
language
 What a Level 0+ Can Do

A Level 0+ can
• communicate with rehearsed and 
memorized utterances
• cover limited topics on immediate 
survival needs including food, colors, 
numbers, clothing, weather, days of the 
week, etc.
 How Well a Level 0+ Speaks

A Level 0+ speaks with
• lists of simple fixed rehearsed words 
or phrases
• memorized utterances
• faulty stress, intonation and tone 
 What a Level 1 Can Do

A Level 1 can
• maintain very simple face‐to‐face 
conversations to satisfy simple daily 
needs
• ask for help and to verify 
comprehension of native speech 
• ask and answer simple questions
 Level 1 Tasks

• Simple short conversation
• Role play
• Examinee ask questions
 How Well a Level 1 Speaks

A Level 1 speaks with
• very limited vocabulary to survive 
immediate personal and accommodation 
needs including daily routine
• random or severely limited structural 
accuracy, time concepts vague
• great difficulty
• discrete sentences
 What a Level 2 Can Do

A Level 2 can
• satisfy routine social demands and 
limited work requirements
• describe people, places, and things
• narrate in present, past and future time 
frames
• give instructions or directions
• deal with non‐routine daily situations
 Level 2 Tasks

• Narration in the past
• Narration in the present
• Narration in the future
• Description
• Directions/Instructions
• Role play with complication
• Reporting facts
 How Well a Level 2 Speaks

A Level 2 speaks with
• sufficient vocabulary to talk about high‐
frequency concrete topics including current 
events
• simple structure and basic grammatical 
relations typically controlled (more often 
than not)
• confidence
• minimally cohesive utterances
 What a Level 3 Can Do

A Level 3 can 
• support opinions on practical, social and 
professional issues
• discuss societal issues, clarify points
• hypothesize or speculate on societal issues
• deal with unfamiliar situations and justify 
decisions
 Level 3 Tasks

• Support opinion
• Discuss abstract topics
• Hypothesize 
• Deal with unfamiliar situations
 How Well a Level 3 Speaks

A Level 3 speaks with
• broad vocabulary on practical, social and 
professional topics, conveying abstract concepts
• structural devices flexibly and elaborately but 
with noticeable imperfections
• extended discourse
And
• a level 3 speaks readily, fills pauses suitably and 
uses cultural references
 Q&A 
Discussion