Journal of Second Language Writing 36 (2017) 85–86

Contents lists available at ScienceDirect

Journal of Second Language Writing

journal homepage: www.elsevier.com/locate/jslw

Disciplinary Dialogues

An assessment perspective on argumentation in writing MARK

a,⁎ b
Lia Plakans , Atta Gebril
a
The University of Iowa, N259 Lindquist Center, Iowa City, IA, 52242, United States
b
The American University in Cairo, 2046 HUSS Building, The American University in Cairo, 11835, New Cairo, Egypt

Argumentative writing appears with regularity in assessments of academic English. Hirvela uses examples of writing sections from
standardized assessments (e.g.TOEFL, IELTS, and GRE) to spotlight the situation: “The fact that we rely so heavily on argumentative
writing to tell us how well students write academically highlights the importance of argumentation in SLW [second language
writing]”. For a language tester, selecting this task genre for performance assessment should entail careful consideration of empirical
evidence to inform test design, development, and use. However, research focused on argumentation in test tasks is scarce, confirming
Hirvela’s call for more empirical research that looks into argumentation. In this response, we explore four key aspects of assessment in
their relation to argumentation: 1) construct definition; 2) tasks; 3); test performance; and 4) scoring.
First, in Hirvela’s agenda, conceptualizing argumentation is described as “the theoretical domain through which we define and
understand it (argumentation)”. In language testing, this domain is referred to as a construct – what we use when designing tasks to
elicit the underlying abilities we want to measure. Although constructs have not been articulated that specifically underlie the
assessment of argumentation, Shaw and Weir (2007) detailed six phases of writing which they believe generally contribute to the
quality of writing, based on work with the Cambridge English assessments. These include: macro-planning, organization, micro-
planning, translation, monitoring, and revising. Most of these seem relevant to argumentation, for instance, macro-planning is where
test takers identify the writing genre, and translation is where they convert abstract thought into linguistic form. This example
suggests how argumentation might be situated in a construct based on the writing process; however, while we could map
argumentation into writing constructs in language assessment, an explicit construct of the ability to write argumentation in a second
language remains undefined.
Second, test developers consider test tasks. Hirvela described the TOEFL independent writing task that asks test takers to write an
essay expressing and supporting “your opinion about a topic or issue.” In effect, this equates to building an argument. Why is
argumentation used in this task? The TOEFL writing framework, a guiding document in the latest revision of the test, states that “in
most academic settings, writing represents a response to learning a set of materials, synthesizing information, or building an argument
[italics added for emphasis] from previously viewed resources” (Jamieson, Eignor, Grabe, & Kunnan, 2008, p.78). In other words,
argumentation was selected because it is seen as critical in academic environments (Chapelle, Enright, & Jamieson, 2008), which
parallels Hirvela’s explanation for its prevalence in SLW teaching.
Third, the actual writing or test performance by test takers on argumentative tasks should show how well they can write an
argument, or in Hirvela’s words, “tell us how well students write academically” (Hirvela). Do argumentative tasks elicit writing from
SLW students that includes features associated with this genre? A few studies have compared rhetorical features of writing across
score levels. Such analysis provides evidence of how well the test is measuring these features. Cumming et al. (2005) considered the
structure of writers’ arguments on TOEFL tasks by evaluating six elements of performance: claims, data, warrants, propositions,
oppositions, and responses to oppositions. Although the relationship between writer’s proficiency and argument structure was
complex, more proficient writers were found to employ higher quality claims and propositions in their arguments. More research in
this vein is needed to provide evidence on the usefulness of such tasks in assessing writers’ argumentative ability.
Fourth, one aspect of test development of particular importance in writing assessments is the design of scoring schemes–rubrics or
scales. According to Shaw and Falvey (2008), the “development of a scale and the descriptors for each scale level are of critical

⁎
Corresponding author.
E-mail addresses: [email protected] (L. Plakans), [email protected] (A. Gebril).

http://dx.doi.org/10.1016/j.jslw.2017.05.008

Available online 09 May 2017

1060-3743/ © 2017 Elsevier Inc. All rights reserved.
L. Plakans, A. Gebril Journal of Second Language Writing 36 (2017) 85–86

importance for the validity of the assessment” (p. 12). Do rating scales target argumentation in writing adequately? If we take the
IELTS scoring rubric for Writing Task 2, in which writers develop an argument or take a specific position, there are four criteria on the
scoring scale: Task fulfillment, cohesion and coherence, lexical resources, and grammatical range and accuracy. Argumentation could
be evaluated by most of these features. For example, task fulfillment taps into whether test takers successfully address the task by
presenting a well-developed position or argument. Mickan (2003) investigated what distinguishes intermediate and advanced writers
on this task, concluding that it was “consistent development of the argument” (p.142). Also looking at scoring, Zhao (2012)
investigated how written voice might be scored to provide a more specific criterion related to argumentation. Grounded in prior
research, the researcher detailed 11 elements of voice including hedges, attitude markers, and rhetorical questions. Using rater think
aloud data and ratings from 480 TOEFL essays, she piloted, analyzed, and refined the scale. These studies suggest viable directions for
future research scoring and argumentation.
Finally, we would like to note that the examples of assessment tasks mentioned by Hirvela and discussed so far in this response are
from large-scale standardized assessment. Assessment, however, is a tool employed daily by writing teachers, both implicitly and
explicitly (Crusan, 2010). Considering the close alignment between learning, teaching, and assessment, we propose that the latter be
integrated more throughout the agenda. Hirvela suggests investigating how teachers assess argumentative writing and operationalize
this construct in their classes. This area is of critical importance since anecdotal evidence shows that teachers tend to have different
conceptualizations of argumentation, potentially leading to problems with judging the quality of students’ argumentative writing.
Future research needs to pay adequate attention to classroom-based assessment issues by looking into how teachers and students
conceptualize argumentative tasks, and also into scoring procedures used for judging resulting performance. Viewing argumentation
in writing assessment from the lens of formative assessment or learner oriented language assessment (LOLA) holds promise
(Turner & Purpura, 2015).
In reviewing these four areas of written language assessment–construct, tasks, test performance, and scoring–we conclude that,
while language assessment in SLW readily employs argumentation as a task to directly assess writing performance, argumentative
writing may in fact suffer from a lack of theoretical and empirical attention in language assessment. Including assessment in Hirvela’s
agenda for SLW is important. Conversely, the field of language assessment would itself benefit from an articulated plan for research
and practice similar to Hirvela’s trifecta: 1) defining the construct of argumentation (Hirvela’s “conceptualization”); 2) investigating
how the construct is used in the design and development of tasks and scoring (Hirvela’s “pedagogy”); and 3) carrying out assessment
research (Hirvela’s “research”).

References

Chapelle, C., Enright, M., & Jamieson, J. (2008). Building a validity argument for the Test of English as a Foreign Language™. New York: Routledge.
Crusan, D. (2010). Assessment in the second language writing classroom. Ann Arbor: University of Michigan Press.
Cumming, A., Kantor, R., Baba, K., Erdosy, U., Eouanzoui, K., & James, M. (2005). Differences in written discourse in independent and integrated prototype tasks for
next generation TOEFL. Assessing Writing, 10, 5–43.
Jamieson, J. M., Eignor, D., Grabe, W., & Kunnan, A. J. (2008). Frameworks for a New TOEFL. In C. Chapelle, M. Enright, & J. Jamieson (Eds.), Building a validity
argument for the Test of English as a Foreign Language™ (pp. 55–96). New York: Routledge.
Mickan, P. (2003). ‘What's your score?’: An investigation into language descriptors for rating written performance. IELTS Research Reports, 5, 125–155.
Shaw, S., & Falvey, P. (2008). The IELTS writing assessment revision project: Towards a revised writing scale. Cambridge: University of Cambridge ESOL Examinations.
Shaw, S., & Weir, C. J. (2007). Examining writing: Research and practice in assessing second language writing. Studies in langauge testing, Vol. 26. Cambridge: UCLES/
Cambridge University Press.
Turner, C. E., & Purpura, J. E. (2015). Learning-oriented assessment in the classroom. In D. Tsagari, & J. Banerjee (Eds.), Handbook of second language assessment (pp.
255–274). Boston, MA: DeGruyter Mouto.
Zhao, C. G. (2012). Measuring authorial voice strength in L2 argumentative writing: The development and validation of an analytic rubric. Language Testing, 30,
201–230.

Lia Plakans is Associate Professor in the Department of Teaching & Learning, University of Iowa. She conducts research and teaches courses in second language
writing, reading, and assessment.

Atta Gebril is associate professor in the Department of Applied Linguistics, the American University in Cairo (AUC), Egypt. He obtained his PhD in Foreign
Language & ESL Education from the University of Iowa. His research interests include writing assessment, test validation, and reading and writing connections.

86