CAF: Defining, Refining, and Differentiating

Constructs

A Research Summary
What is the difference between Accuracy *
? and Fluency
 Fluency: means being able to use language
.easily, in an effortless way

Accuracy: means doing something correctly or

.exactly, without making errors
:Introduction *
The notions of complexity, accuracy, and fluency (CAF) have been
employed in a number of studies on the acquisition and use of a
second language, although they do not constitute a theory or a
research program in themselves. They are dimensions for describing
language performance, most frequently used as dependent variables
to assess variation with respect to independent variables such as
.acquisitional level or task features
CAF measures can also be used to describe performance by native
speakers or first language learners. Each of these three constructs
has been operationalized in various ways and the articles in this
issue bear witness to this lively methodological debate. Researchers
agree on the usefulness and validity of the constructs, but they also
agree that they should be further specified and that they do not
.exhaust performance description
:DEFINING CONSTRUCTS *
Researchers frequently seek to identify the ‘best’ measures -
of language performance, and the answer is typically that the best
measures are those that most clearly show variance among
subjects, both over time and across tasks, correlating with other
equally varying proficiency measures (e.g. Wolfe-Quintero et al.
.1998)
,Research should be concerned with variations and differences -
but also with constants and similarities. If after an experimental
treatment two groups of subjects do not show any difference, then
this is not a nonresult, but a result just as interesting as their being
.different
Likewise, if a measure does not change over time, then this does not
make it a poor measure, but perhaps a measure pointing to a trait
that does not actually vary. For a measure to be valid, it does not
necessarily have to vary across subjects, but it must adequately
represent its underlying construct. If the trait related to that
construct does not change, then so will not the measure. This search
.for variation is manifest in the present issue as well
Norris and Ortega in their contribution seek to identify variables -
with greater ‘predictive value’ or that are ‘best indicators of
.differences’ among students at various proficiency levels

Norris and Ortega note that ‘researchers have not done sufficient -
thinking about what we are measuring or why’. In their paper they
provide an exemplary treatment of syntactic complexity, carefully
.pinpointing theoretical constructs and their operationalizations
:FLUENCY *

Fluency can be defined as ‘the capacity to produce speech at -

.normal rate and without interruption’ (Skehan, this issue)

Fluency is a multidimensional construct, in which sub-dimensions -

can be recognized, such as breakdown fluency, repair fluency,
speed fluency (Tavakoli and Skehan 2005). Once it is established
which of these sub-dimensions is at issue, it is in principle relatively
.transparent what is being measured
:ACCURACY *
Accuracy is perhaps the simplest and most internally coherent -
.construct, referring to the degree of conformity to certain norms

Despite the dubious validity of accuracy as a measure of language -

development— Norris and Ortega (2003: 737) suggest it may be
appropriate mainly in the description of advanced levels—it is at
least a well-defined and coherent notion. There is a risk, however,
of making it a spurious construct covering distinct dimensions, and
this happens when errors are classified according to their gravity.
For instance, some authors (e.g. Homburg 1984) propose to
calculate accuracy scores by giving errors different weights
.according to the extent they compromise communication
:COMPLEXITY *
Complexity is certainly the most problematic construct of the CAF -
triad because of its polysemous nature. The term complexity is
used to refer to the properties of both tasks and language
.performance

Complexity is still retains multiple meanings, because it can be -

.applied to different aspects of language and communication

A structurally complex rule requires many cognitive resources for -

.its production and that it is therefore acquired late
Development should be considered as a separate dimension from -
CAF. CAF refer to the properties of language performance as a
product, while development is a process, with its subdimensions
such as route and rate. CAF measures can empirically be related to
developmental dimensions, but the latter should not be analytically
.considered as the part of the former
:TASKS AND CAF: GENERAL AND SPECIFIC MEASURES *
The task may require learners to pay a high amount of attention to -
the content of their utterances which depletes the attention available
for the Formulator that is responsible for constructing more
.structurally complex utterances

Robinson’s Cognition Hypothesis makes the prediction that complex -

tasks will lead to more complex and accurate performance in general,
while Skehan maintains that ‘each influence is a task design feature’
.rather than a generalized effect
It is more logical to expect that a task requiring to talk about -
astronauts and their space adventures elicits more uncommon words
than one describing children playing with a ball in the park. In this
way, the relationship between task difficulty and linguistic complexity
becomes more reliable, but less theoretically
.Interesting

linguistic complexity grows when this is specifically required by the -

task and its goals, and not for the sake of it, as if learners aimed at
complexification by default, as if they were eager to produce a
subordinate clause or a rare lexical item whenever they have a pinch
.of free attentional resources
:ADEQUACY *
Adequacy represent the degree to which a learners’ performance is -
.more or less successful in achieving the task’s goals efficiently

As a descriptor of communicative success and efficiency, adequacy -

can be measured in several ways. In closed tasks with
correct/incorrect outcomes, it can be rated straightforwardly as the
ratio of correct items achieved. In open tasks with no predefined
correct answer, adequacy can be evaluated by means of qualitative
ratings, using predefined descriptor scales like the ones of the
common European Framework of Reference (Gilabert et al.
forthcoming). A second way in which adequacy should be considered
.in CAF research is in the interpretation of CAF measures themselves
Wolfe-Quintero et al. (1998: 2) write that ‘we are not interested in -
measuring the ability to ‘‘write well’’ in a second language . . . but in
measuring language development as it manifests in a written
modality’. It is certainly possible to measure anything in and for itself,
but one wonders how performance can be meaningfully interpreted
.without the reference to its quality and effectiveness
:CONCLUSIONS *
CAF measures are a good starting point for describing linguistic
performance and its multidimensionality. In order to apply them it is
first necessary to clearly define underlying constructs, so that each
.measure or groups of measures refer to a well-identifiable construct