ISSN 1813-1166. Proceedings of the NAU. 2010.

№3 149

PROFESSIONAL EDUCATION

UDC 811.111:378.261 (045)

Olena P. Petrashchuk, Doctor of Pedagogy, Prof.
TESTING LANGUAGES FOR SPECIFIC PURPOSES
National Aviation University
Е-mail: [email protected]
The article is about a problem of the English language for specific (professional) purposes.
Language for specific purposes is being analysed in comparison with the General Language. Oral
proficiency interview is being described as an effective tool of oral proficiency measurement.
Regarding high reliability of the test results, the ways to validate the oral proficiency interview as a
part of oral test in English for specific purposes aimed are proposed.
Розглянуто питання володіння англійською мовою для спеціальних цілей. Проаналізовано
особливості англійської для професійних цілей порівняно з розмовною англійською. Описано
характеристики усного професійного інтерв’ю як ефективного засобу визначення рівня
сформованості умінь говоріння. Запропоновано шляхи валідації усного інтерв’ю як частини
тесту з англійської.
Рассмотрены вопросы владения английским языком для специальных целей.
Проанализированы особенности английского языка для профессиональных целей в сравнении
с разговорным английским языком. Описаны характеристики устного профессионального
интервью как эффективного способа определения уровня сформированности умений говорения.
Предложены пути валидации устного интервью как части теста по английскому языку.

Statement of purpose The background knowledge concerns

specific communication situations and is
Language for specific purposes (LSP) is a
reflected in vocabulary (e.g., terminology),
complicated notion which might be defined at a communication intents and tasks (e.g., job
structural level, functional level and discoursal related communicative tasks), topics (e.g., job
level which complement each other [1,37] in related areas of language use), etc. Therefore
communication. It is known that traditionally communication in LSP is feasible for
LSP is considered the language used for work or participants with relevant background
academic purposes. If LSP is viewed as an knowledge required and identified by specific
approach not as a product [1,19] it is possible to communication purposes.
conclude that all specificity lies in the area of While testing the LSP one should take into
learner’s needs to use the language in some account various factors which might affect a test
target language communications. From this taker’s performance and, therefore, the
point of view any language used purposely in reliability of testing results.
specific situations of communication can be In this article some issues concerning testing
defined as LSP. LSP will be reviewed and some research results
Obviously that communication in LSP will and current tendencies in this area will be
be different from communication in general identified.
language (GL). Our teaching and testing Firstly, the difference between the LSP and the
experience show that the main difference GL tests will be considered. Then the research
between LSP and GL is background knowledge results in the area of assessment of spoken
which serves as a necessary prerequisite for language, and the oral proficiency interview (OPI)
successful learning and testing LSP. in particular, will be reviewed and analyzed.
© Olena P. Petrashchuk, 2010
150 ISSN 1813-1166. Proceedings of the NAU. 2010. №3

Review of research results Another challenge is the method of scoring

the test results which should be based on the
Douglas reports that there are two main
authentic (job related) criteria. For example, in
features that distinguish an LSP test from a GL
case with aviation personnel two raters should
test. One difference concerns authenticity of the
be involved: a linguistic specialist and an
test tasks, and the second one deals with
aviation specialist with advanced language
interaction between language knowledge and knowledge - to assess properly both language
subject matter knowledge. The test task and specific purpose content knowledge.
authenticity is identified by the degree of the According to language requirements of the
test task relevance to the target language use International Civil Aviation Organization
situation (situational authenticity) and the (ICAO) aviation personnel language ability
degree to which the test takers’ performance should be assessed in two skills only – listening
demonstrates use of language, strategic and comprehension and speaking. The ICAO rating
world knowledge involved by the test task scales include detailed descriptors of oral ability
(interactional authenticity) [2]. of the personnel. So, assessing speaking is
There are several challenges faced by the considered below.
LSP test designers. One is the degree of Speaking ability is widely believed to be the
influence the specific content knowledge has on most complicated among others to master and,
the performance of the test taker. therefore, the most difficult to assess [3]. Speaking
For instance, the proficiency LSP test might ability means the ability to interact successfully in
require more specific background knowledge the language, and this ability involves both
than a diagnostic LSP test. That is because the comprehension and production [4,113].
more authentic a test task is to be, the more N. Schmitt mentions that in addition to
specific purpose content it will require. producing language, spoken interaction also
Similarly, the higher level of performance to involves negotiating language under time
be measured, the more specific purpose content constraints. Participants of spoken interaction
will be reflected in the test task. are to adjust their utterances in accordance to
In order to properly evaluate how much their communicative intents.
specific purpose background knowledge is Among issues under consideration
needed one should clarify how specific is N. Schmitt indicates genres and generic
specific and what makes the text more or less structure of speaking, also keeping exchange,
specific. One of the assumptions might be that turn-taking (turn types) and topic management [5].
this is the level of contextualization that makes Speaking can be realized only by articulation
the text more or less specific [1; 2]. of sounds and tone. N. Schmitt points out on the
For instance, the Listening Comprehension importance of pronunciation, which is
(LC) test for aviation personnel is based on job ”responsible for ‘intelligibility’ - whether or not
related text and situations. Since the language we can get our message across”[2,212]. In
use at the workplace is closely connected with pronunciation he indicates key roles of
technical procedures, the authentic test task will stress/unstress and sound segments, tone
require extended specific purpose background units/chunking, prominence, turn-taking,
knowledge. introducing and ending topics, social meanings
While taking LC test the Aviation Academy and roles [5].
students (ab initio controllers) in comparison Therefore, speaking test is an assessment of
with professional controllers, often demonstrate the ability to speak the target language. The
much lower comprehension of the assessment of the candidate’s speaking ability
radiotelephony text, whereas they are usually can be carried out indirectly as a paper and
more successful in comprehending the pencil test, and directly by observing his/her
interaction performing within interactive test
professionally-oriented text which is beyond the
task [6,182].
technical procedures.
ISSN 1813-1166. Proceedings of the NAU. 2010. №3 151

Lots of researches have been done to find the Purpose of the work
best solutions for proper measurement of To make the test design appropriate to LSP
spoken language ability. Thus, the researchers targets one should focus on the test task format
focus on test-taker characteristics, types of test which is the best for job/professional related
tasks, test qualities, rating scales, effects of situations.
interlocution strategies, impact of The term task refers to ‘a type of test item
examiner/interlocutor’s and rater’s training, etc. involving complex performance in a test of
[7], scoring rubrics and rating criteria [3], test productive skills’, e.g., adopting a specified role
participants’ behaviour [8]. in a role play, describing a picture, etc. [6,196].
Regarding assessment of speaking ability a A test task is a means to elicit language
test taker should demonstrate an oral language performance which is feasible and expected,
performance within a construct to be measured. represents the real candidate’s language ability
This performance should reflect in maximum and ‘will be scored validly and reliably’ [4,113].
his/her real ability to speak in target language in To design a relevant test task a construct
real-life situations. The test results should be which is to be assessed should be identified. B.
reliably documented. Let’s have a look at some O’Sullivan reports about three groups of
of the crucial factors regarding the LSP oral test. ‘operations involved in the performance of a
A test taker is any person taking a test. There particular task’.
are other synonymic terms such as candidate, They are as follows:
examinee, testee [6,208] or interviewee in oral – Informational – providing personal/non-
tests. Any of his/her personal qualities, personal information, elaborating, expressing
opinions, comparing, complaining, speculating,
background knowledge, educational and
analyzing, making excuses, explaining,
working experience and other factors might
narrating, paraphrasing, summarizing,
affect the test taker’s performance. suggesting, expressing preferences;
L.Bachman describes four groups of test – Interactional – challenging,
taker’s characteristics which may affect the test agreeing/disagreeing, justifying/providing
performance namely personal characteristics, support, qualifying, asking for opinions,
topical knowledge, general level and profile of a persuading, asking for information,
test taker’s language ability and predictions conversational repair, negotiating meaning;
about test takers’ potential affective responses – Managing interaction – initiating,
to the test [3,129-130]. changing, reciprocating, deciding, terminating
B. O’Sallivan identifies three categories of [7,7].
test taker’s characteristics which might the best It should be noticed that all the above
way reflect characteristics of LSP test takers. mentioned operations fit well to the construct
They are as follows: physical (physiological), identified for the LSP test and the test for
psychological, experimental. Physical aviation personnel in particular.
characteristics cover age, sex and also ailments Hughes suggests to elicit a valid sample of
oral ability with appropriate techniques which
or disabilities, psychological characteristics –
can be presented in three general formats:
memory, personality, cognitive style, affective
interview, interaction with fellow candidates,
schemata, concentration, motivation and responses to audio- or video-recorded stimuli.
emotional state, experiental characteristics In case of testing aviation personnel all three
include education background, experience in formats might be appropriate and being used
taking tests, preparedness for the test, now in different Aviation English tests. But
communication experience [7]. specificity of ground-to-air communication
Regarding the LSP test one characteristic is between a pilot and a controller may identify an
lack – this is working/occupational/job oral interview format as the most relevant to the
related/professional experience of a candidate. real job situation.
© Olena P. Petrashchuk, 2010
152 ISSN 1813-1166. Proceedings of the NAU. 2010. №3

Hughes points out a drawback of the A. Lazarton reviews outcome-based

interview which is lack of possibility to elicit empirical studies of OPIs which have been
various styles of speech because of inequality of carried out in response to criticism of this test
participants (interlocutor is viewed as a seniour task type.
partner of communication). He stands ‘for Variety of issues related to the interview
introducing a variety of elicitation techniques procedure have been studied in the area of
into the interview situation [4,119]. a) proficiency interview guidelines;
Six types of the techniques have been b) validity of a construct in respect of
indicated: relationship between grammatical, discourse,
– questions and requests for information sociolinguistic and strategic components of
(e.g., requests for elaboration, appearing not to communicative competence;
understand, invitation to ask questions, c) consistency of relationship between direct
interpretation, abrupt change of topic); and semi-direct proficiency interviews’ ratings;
– pictures; d) impact of interviewers’ and raters’
– role play; behaviour, topics, speech styles on
– interpreting; eliciting/scoring performance in the face-to-face
– prepared monologue (presentation); proficiency interview, and many other issues
– reading aloud [4, 119-121]. [8,6-12].
The techniques may be well applied to The results reported have proved
testing aviation personnel which is done through appropriateness of an OPI to assess speaking. At
OPI. the same time and in respect of the LSP, special
Many researchers express criticism of the
attention should be paid to the following main
OPI. The criticisms are based on presumptions
factors which might significantly influence the
that proficiency as a unitary language ability
candidate’s scores on oral proficiency:
which is not similar to interactional competance,
– Interlocutor’s and rater’s reliability and
can not be assessed appropriately through the
proper preparedness;
OPI.
– Availability of task-specific rating scales
This testing procedure lies in a limited
domain of interaction and therefore is able to and their proper validation;
provide with a valid sample of overall language – Topics and speech styles;
oral ability [7,6]. “The general consensus – Direct or semi-direct format of the
nowadays is that the inherent inequality of the interview;
test event makes true conversation impossible” – Proper construct validation.
because of interlocutors having control over the Another approach to validate an OPI can be a
candidate’s discourse [7,8]. process-based research with the interview
In other words the speech sample we get and discourse analysis [8,12]. This method of the
assess through the OPI does not reflect a interview validation seems to be more relevant
candidate’s ability to interact in any untested to the LSP test and in particular to the test for
context within all communicative functions and aviation personnel.
variety of speech events. Therefore this is an Through the discourse analyses it might be
issue of the construct validity. possible to validate the construct which is
One of the ways to provide an appropriate aviation specific within radiotelephony
construct to be assessed as well as to increase communication.
reliability and validity of testing results is the Discourse competence is a component of
OPI validation. communicative competence “which is basically
Different approaches to validation of the OPI concerned with above-sentence-level cohesion
are described in the reference literature. and coherence” [6,47].
ISSN 1813-1166. Proceedings of the NAU. 2010. №3 153

While analyzing discourse one will have a – studies on comparison of test task formats,
look at real texts and study “the relationship e.g., direct or semi-direct interview,
between texts and the contexts in which they questioning, describing a picture, responding to
arise and operate” [5,56]. In other words the recorded stimulus, etc.
discourse analysis enables to deal with language All of the above mentioned approaches are in
in its social context, and in this respect it can compliance with what might be going on during
well serve for development of evaluation the oral interview of aviation personnel.
criteria of test performances as well as for Presumably the participants of the interview
validation of test tasks. might be two people interacting in the non
There are many approaches to discourse visual format via microphone and headset. One
analysis depending on their disciplinary origins. of the participants is a pilot and another one is
For instance, in respect of aviation area of an examiner/interlocutor pretending to be an air
communication two disciplines are worth traffic controller and vice versa. This kind of
mentioning – sociology and sociolinguistics. relationship and interaction format simulate real
Sociology is considered to make the major life situation on the job place.
contribution to analysis of spoken discourse The specificity of the interview is that the
from its conversational analysis. interlocutor should have some technical
Sociolinguistics approaches, namely background knowledge though he/she works
ethnography of speaking and interactional within interview scenario (structured and semi-
linguistics, enables to study speech events. structured parts of the OPI).
Conversation analysis is concerned with As it has been mentioned above one of the
dialogic, spoken discourse of an informal criticism of OPI was that the interview format
character. Rules and sequence of turn-taking are does not allow assessment of a candidate’s oral
a main focus of the analysis. language performance within all possible
Interactional linguistics is concerned with communicative functions, that the interview is
studying of how language creates effective restricted in speech events and makes
communication in a context. impossible to cover its variety in natural
Ethnographic approach deals with research of context.
speech events in their cultural and social In order to compensate this, the interview for
contexts [5,60-64]. aviation personnel might include different test
Discourse-based studies on spoken language task types including direct and semi-direct
assessment conducted the last decade have formats.
made it possible to get empirical results on a/ It should be noticed that in the case of
how the language proficiency interview is assessment oral ability of the aviation personnel
accomplished through discourse and b/ there is one peculiarity absent in many other
evidences of multidisciplinary nature of oral occupations.
assessment. This is that the speech of the interaction
Regarding assessment of pilot/air traffic participants is recorded in both test and non test
controller oral ability some of the discourse- situations. This condition is of great value since
based approaches seem to be relevant for one can have authentic recordings of the same
research aimed at validation of the OPI as well candidates from the job place and from the OPI
as rating procedures [8,14]. to serve the basis for discourse-based research.
They could be as follows: For any research on the LSP test data
– studies on the interview participants analysis and presentation are of great
behaviour; importance. The research on the OPI of aviation
– studies on extent of identity of a personnel can be carried out reflecting on
candidate’s behaviour in testing and real-life general analytic techniques suggested by
environment; A. Lazarton:

© Olena P. Petrashchuk, 2010

154 ISSN 1813-1166. Proceedings of the NAU. 2010. №3

1) using authentic, recorded data; It is evident that proper validation of the OPI
2) using ‘umotivated looking’ rather than will significantly influence on the quality of the
pre-stated research questions; LSP test. Due to quality testing a test taker will
3) employing the ‘turn’ as the unit of be provided with comfortable friendly
analysis; atmosphere during the interview which makes
4) analyzing single cases, deviant cases; possible to demonstrate the best oral language
5) disregarding ethnographic and demographic performance. In case with aviation personnel it
is a key issue because the testing is conducted
particulars of the context and participants;
with a purpose to obtain a speech sample
6) eschewing the coding and quantification measurable against descriptors of operational
of data [8,75]. level according to requirements of ICAO [9].
While analyzing interactive data the In turn it will contribute to possibility to
following analytic tools are proposed to be used manage language performance of a test taker
in five consequent steps [8,88-92]: and in this way to provide him/her with friendly
1) to select a sequence of research interest by test taking environment. In turn, it will
looking for identifiable boundaries; contribute into the reliability of the test results
2) to characterize the participants’ actions in
the sequence by answering the questions: “What References
is the participant doing in this turn?”1 1. Hutchinson T. English for Specific
3) to consider how packaging of actions (how Purposes. A learning-centred approach /
they are formed and delivered) provides for T. Hutchinson, A. Waters. – Cambridge: CUP,
certain understandings; 1987. – 240 p.
4) to consider how timing and turntaking 2. Douglas D. Language for specific purposes
provide for certain understandings of actions testing / D. Douglas // Language Testing and
and the matters talked about; Assessment. – 1997. – Vol. 7. – Р.111–119.
5) to consider how the ways the actions were 3. Bachman L. Language testing in practice /
accomplished suggest certain identities, roles, L. Bachman, A. Palmer. – Oxford: OUP, 1996. – 422 p.
and/or relationships for interactants. 4. Hughes A. Testing for language teachers. –
2nd ed. / A. Hughes. – Cambridge: CUP,
As mentioned above, the OPI for aviation
2003. – 185 p.
personnel may consists of various task types and 5. Schmitt N. An introduction to Applied
formats of interaction. Some of sections of the Linguistics / N. Schmitt. – London: Hodder
interview may include description of a picture Arnold, 2002. – 187 p.
or extend on questions printed on the card (or 6. Dictionary of language testing / M. Milanovic //
displayed on monitor). In these cases the Studies in Language Testing. – 1999. – # 7. – Р. 16–21.
candidate will be expected to demonstrate 7. O’Sallivan B. Notes on Assessing
monologic speech. The monologic data can be Speaking / B. O’Sallivan. – Cambridge:
analyzed through rhetorical, fuctional and UCLES. – 2008. – 106 p.
structural analysis [8,96-100]. 8. Lazarton A. A Qualitative Approach to the
Conclusion Validation of Oral Language Tests / A. Lazarton;
ed. M. Milanovic, C. Weir // Studies in
All the issues mentioned above are being Language Testing. – 2002. – # 14. – P. 42–46.
studied and researched because of their key role 9. Manual on the Implementation of ICAO
in making language ability measurement precise Language Proficiency Requirements // ICAO. –
and efficient. 1st ed. – 2004. – 180 p.

The editors received the article on 17 June 2010.

1
It should be noticed that in case with the Aviation English
the participants are to deal within mandatory
communicative functions prescribed by ICAO Doc # 9835,
app.B [9].
ISSN 1813-1166. Proceedings of the NAU. 2010. №3 155

© Olena P. Petrashchuk, 2010