3 Varieties of Cloze Procedure
3 Varieties of Cloze Procedure
3 Varieties of Cloze Procedure
Introduction
Ever since 1953, when Taylor introduced the cloze procedure, it has been probably one
of the most talked about topics in the field of language testing and language teaching.
Many articles have been published mostly supporting and some criticizing the
usefulness of this procedure in language education. However, cloze has remained as
one of the valuable procedures for instructional purposes in general, and for testing and
measurement purposes in particular. (Interested readers are referred to Oller, 1979,
1980, 1983; Farhady, 1980, 1983; and Farhady, et.al., 1994 to name a few).
Despite the popularity of cloze procedure as a test, many language educators are not
well aware of its merits and demerits. Therefore, this paper is an attempt to provide the
reader, especially language teachers, with some useful information about the cloze test.
More specifically, the purposes of this paper are to (a) offer a clear definition of the
cloze test through delineating the steps for its development, (b) explain the scoring
methods of the cloze tests, (c) introduce the varieties of the cloze tests, and (d) offer
some practical classroom applications of such tests.
Cloze test is often defined as a text or a passage of appropriate length and difficulty
with every nth word deleted. The definition seems quite straightforward. However,
certain precautions should be taken regarding the concepts of appropriate length,
difficulty, and the value and the frequency of n. Each is briefly explained below.
The value of n refers to the distance between two deletions. That is, by assigning a
value to n, the number of words between two deletions is determined. The frequency of
n refers to the number of deletions in a text. Since the distance between the deletions
and the number of the deletions influence the major characteristics of the test, including
the length and the difficulty, they are explained in detail.
Regarding the distance between the deletions, suppose a passage is selected and n is set
to be 1. This means that the test developer should start deleting words one after the
other. In such a case, all the words will be deleted and no text will be left. Thus, n
cannot be taken to be one. In other words, the minimum value for n should be 2. That is,
one should start counting the first word and deleting every second word. The following
sample passage will be used throughout this paper to illustrate the points relevant to
different aspects of cloze tests.
Education has two goals. First, it provides means so that we may earn our living. Some
of these tools are basic tools, such as reading or writing. Others are technical skills, such
as typing, accounting, and data processing. Still others are highly professional skills
peculiar, for example, to medicine, to science, and to teaching. Secondly, it provides
217
Varieties of cloze procedure Hossein Farhady
experience so that we can learn how to live. Education provides knowledge of ideas for
using the past. It provides a sense of values for meeting the future.
Setting the value of n at 2, this sample passage will look like the following:
Education ___ two___. First, ___ provides ___ so ___ we ___ earn ___ living. ___ of ___ tools
___ basic ___, such ___ reading ___ writing. ___ are ___ skills, ___ as ___, accounting, ___ .
Still ___ are ___ professional ___ peculiar, ___ example, ___ medicine, ___ science, ___ to ___.
___, it ___ experience ___ that ___ can ___ how ___ live. ___ provides ___ of ___ for ___ the
____. It ___ a ___ of ___ for ___ the ___.
In this example, every other word is deleted, i.e., n = 2. This kind of test is difficult
because it requires the test taker to provide one word for every single word given in the
text. The value of n, however, can be taken to be any number. Of course, the larger the
value of n, the more distant the blanks will be from one another. Further, as the distance
among the blanks increases, more context will be provided which makes the test easier
because the test taker is required to provide one word after reading a larger amount of
the text. Although values of 5, 6, 7, and 8 have been often used in constructing a cloze
passage, the most frequently used value for n has been 7. That is, every seventh word in
the passage is deleted.
It should be mentioned that taking this number as a deletion rate is not a strict law. It is
more or less conventionalized. Otherwise, cloze passages with 5th, 6th, and 8th word
deletion are empirically shown to have similar characteristics. Therefore, teachers
should not limit themselves to only 7th word deletion procedure. Based on the nature of
the passage and the purpose of the cloze, they should use other deletion rates as well.
Another point related to n is its frequency that determines the number of deletions.
Generally speaking, since every deletion is considered to be an item, the more the
number of deletions, the greater the number of items, and thus the longer the test.
However, an excessive number of deletions would lead to an unreasonably long test.
Therefore, the length of the text, and thus the number of deletions should be kept within
a manageable range. Regarding the number of deletions, some researchers consider a
maximum of 50 blanks as an acceptable number of deletions. Others are satisfied with
20 to 25 blanks (Rand, 1978; Cohen, 1980). However, most researchers agree that the
optimal number of deletions in a cloze test is around 30 to 35. When the distance among
the deletions and the number of deletions are decided upon, the length of the passage
can be determined as well. For example, with 30 deletions and every 7th word deleted,
the length of the passage would be approximately 210 words.
Regarding the length of the passage, however, one more point should be mentioned.
Research on reading comprehension indicates that the structure of a text is an effective
factor in processing the materials to be read. More specifically, research shows that two
sentences in the paragraph carry the bulk of information. One of these sentences is the
topic sentence that carries the main idea of the paragraph. The other sentence is the
concluding sentence that provides a summary of the paragraph. Many scholars believe
that these two sentences should not be included in the deletion procedure because
deletion in these sentences might distort the reader’s natural processing of language.
Therefore, these two sentences are left intact. These sentences would normally have 20
to 30 words. Adding the number of words in these two sentences, the length of the
passage would be about 230 to 240 words long. When the number of deletions, the
218
Varieties of cloze procedure Hossein Farhady
distance among deletions, and consequently the length of the passage are determined, a
clear definition of cloze can be provided as follows: cloze is a passage of around 240
words with every 7th word deleted providing that the first and the last sentences
are left intact. The only criterion still to be decided upon is the appropriateness of a
passage in terms of its difficulty.
This formula will result in a particular number that is an indication of the readability
level of the text. In order to clarify the use of this formula, an example may be helpful.
Suppose a teacher intends to develop a cloze passage for a particular group of students
(in this case, third graders at high school). A common procedure to be taken is as
follows. First, three to five passages are selected from the students’ textbook. Second,
the above-mentioned readability formula is applied to the passages and the readability
of each passage is determined. Suppose further that the readability indexes of the three
passages are 14, 13, and 15 respectively. The average readability of these passages,
then, is 14. Thus, this number would serve as a criterion for the selection of a new
passage for the cloze procedure for the third graders. Finally, a new unseen passage is
selected. In order to select a passage, certain criteria should be taken into account. Of
course, the best criterion is the teacher’s judgment. When the passage is judged
appropriate by the teacher, as the second criterion, the formula is applied in order to see
whether its readability approximates the average readability of the students’ textbook.
So, two criteria are utilized to select a new passage: teacher’s judgment and the average
readability of the passages in the students’ textbook.
When the passage is selected and the first and the last sentences are left intact, then
every 5th, 6th, or 7th word is deleted. After deleting the words on the basis of a certain
ratio, the passage can be administered as a test or assigned as a practice activity.
219
Varieties of cloze procedure Hossein Farhady
Education has two goals. First, it PROVIDES means so that we may earn OUR living. Some
1 2
of these tools are main tools, such as reading or writing. There are technical skills, such as
3 4
typing, management, and data processing. Still others are HIGHLY professional skills
5 6
peculiar for example, to speaking, to science, and to teaching. Secondly, education provides
7 8
experience so that we can understand how to live. Education provides a knowledge of ideas
9 10
for using the past. IT provides a sense of values for MEETING the future.
11 12
In this passage, blanks 1, 2, 6, 11, and 12 (written in capital letters) are filled with the
words that correspond to the originally deleted words. Therefore, they are given credit.
That is, the student’s score on this test will be 5 out of 12. Since the other words
provided by the student do not correspond to the originally deleted words, they would
not be given any credit.
It should be mentioned that each method has its own advantages and disadvantages.
The advantage of EWM is that it is objective and easy to apply. That is, no scorer
judgment is needed in order to score the test. However, it has the disadvantage of
lowering the students’ scores. On the other hand, AWM increases the students’ scores
but it is somehow subjective and difficult to apply. The problem is that determining the
words which might be acceptable for a particular blank is both subjective and beyond
the ability of most EFL teachers. In other words, most EFL teachers would not feel
comfortable with deciding on all appropriate and acceptable words for a deletion.
220
Varieties of cloze procedure Hossein Farhady
the subjectivity of scoring would be fairly removed. In the process of pretesting, a list of
acceptable words provided by the native speakers will be prepared for each blank. In
this manner, the scorers will check the words produced by nonnative speakers with the
words provided by the native speakers. If nonnative speakers produced a word which is
provided by the native speakers, they would receive credit for that item. In spite of this
advantage of the AWM, access to educated native speakers is not always possible in
EFL situations. Therefore, most practitioners recommend EWM of scoring for EFL
situations and AWM for ESL situations where native speakers are available.
Up to this point, procedures for the development and scoring a standard cloze test have
been discussed. The remaining of this paper will offer some useful varieties of cloze
and present their applications to classroom situations.
The standard cloze test is defined as an appropriate passage of around 250 words with
every 7th word deleted. According to the psycholinguistic classification of the test
forms from the test taker perspective (Farhady, et al., 1994), the standard cloze test
would be categorized as comprehension/production type test in written mode. Since
research on cloze tests has demonstrated highly acceptable psychometric
characteristics, i.e., reliability and validity, scholars have conducted research to develop
different versions of the cloze procedure. Through one line of research, scholars
attempted to develop varieties of cloze on the basis of the psycholinguistic processes
going on in the minds of the test takers. Instead of having comprehension and
production activities that are required by the standard cloze, they tried to construct
cloze tests which required only comprehension. Thus, the production part, i.e., finding
out a suitable word for each blank and writing it, was eliminated. Removing the
production part from the requirements of a cloze test led to the development of
multiple-choice type cloze tests. Through another line of research, testers concentrated
on the modality of the cloze test, i.e., the mode of language in which cloze is presented.
Contrary to the standard cloze in which the language is in written form, scholars
developed oral cloze tests. Through the third line of research, the test developers
investigated the possible varieties due to the content, form, and the number of blanks in
the cloze test. Each of these lines of research led to the development of varieties of cloze
to be discussed below.
Education has two goals. First, it (provides/gives/offers) means so that we may earn
(our/the/some) living. Some of these tools are (basic/main/major) tools, such as reading or writing.
(There/Some/Others) are technical skills, such as (accounting/management/engineering) and data
221
Varieties of cloze procedure Hossein Farhady
processing. Still others are (all/highly/more) professional skills peculiar, for example, to
(speaking/medicine/reading), to science, and to teaching. Secondly, (it/education/they) provides
experience so that we can (know/learn/understand) how to live. Education provides
(background/knowledge/science) of ideas for using the past. (It/Education/Science) provides a
sense of values for (forming/knowing/meeting) the future.
The advantage of the multiple-choice cloze over the standard cloze is that the task of
comprehension is easier than that of production. Therefore, a multiple-choice cloze
would be easier, and thus, students’ will obtain scores that are higher than on an
open-ended form. Of course, in a norm-referenced situation, the increase at the level of
the scores would not influence the ranking of the students, i.e., all testees will score
higher on the multiple-choice form than they will on the open-ended form. In a
criterion-referenced situation, however, where there exists a predetermined criterion for
the students to meet, low scores would hurt those at the borderline. This shortcoming
can be overcome by developing certain adjustment formulas to equate the scores
obtained from multiple choice and open-ended forms.
Each of the three major types of cloze tests, i.e., standard, multiple-choice, and oral has
many varieties which can be used both for teaching and testing purposes. Below is a
brief description of these varieties that can be applied to each type.
This form of the cloze test is adapted from the traditional fill-in-the-blank type
activities. The difference is that traditional type activities were often sentence based,
whereas a cloze activity involves a passage with every nth word deleted. The deleted
words are listed and the testees are required to fill in the blanks with the appropriate
words from the list. The following is an example.
222
Varieties of cloze procedure Hossein Farhady
Education has two purposes. First, education provides tools so that ____ can earn a living. Some
of ____ tools are basic tools, such as ____ and writing. Others are technical skills, ____ as
typing, accounting, and data processing. ____ others are highly professional skills peculiar, ____
example, to medicine, to law and ____ teaching. Secondly, education provides experience so
____ we can learn how to live. ____ provides a background of ideas for ____ the past. It provides
a sense of values for meeting the future.
In this form, the testees should read the text and understand the context. Then based on
the context, they should select and write the appropriate words in each blank.
In the standard cloze procedure, when a value is assigned to n, the deletion procedure
follows through the text. That is every nth word is deleted. This method, in which the
value of n is fixed and constant, is called the fixed-ratio deletion. Sometimes, the
testers, specially the teachers, are interested in deleting a particular set of words to
check the students' ability in using these words. In such cases, the value of n cannot be
fixed because the blank may correspond to a word that should not be deleted. Therefore,
teachers use random ratio deletion procedure. That is, n is not fixed and it is
determined by the test developer prefers to delete a particular word. In this version,
which is most useful for instructional purposes, a particular group of words is deleted.
For instance, the teacher might be interested to see the extent to which the students have
command over the use of prepositions, articles, verb forms, adverbs, adjectives, etc.
Thus, instead of deleting every 7th word, or whatever it might be, one or two of the
above-mentioned classes of words are deleted. In such cloze activities, known as cloze
tasks rather than cloze tests, certain useful diagnostic pieces of information can be
obtained. The passage given as the example will have the following form when all the
propositions are deleted.
Education has two purposes. First, education provides tools so that we can earn a living. Some
____ these tools are basic tools, such as reading and writing. Others are technical skills, such as
typing, accounting, and data processing. Still others are highly professional skills peculiar, ____
example, ____ medicine, ____ law, and ____ teaching. Secondly, education provides experience
so that we can learn how ____ live. It provides a background ____ ideas ____ using the past. It
provides a sense ____ values ____ meeting the future.
Although this type of cloze does not usually serve as a testing device, it eliminates one
of the major shortcomings attributed to cloze tasks that they do not provide diagnostic
information. Utilizing this version would enable the teachers to have a quite reasonable
idea of the students’ strengths and weaknesses on a predetermined element of language.
Through the findings from these tasks, teachers can plan remedial work to alleviate the
weak points of the students in different areas.
223
Varieties of cloze procedure Hossein Farhady
constructions, a modal such as will, would, etc. and auxiliary verbs such as BE or
HAVE are present. Considering the various forms of these modals and auxiliary verbs,
cloze tests can be constructed to check the students’ ability using the chunk of language
rather than just a word.
3. Catered technique
In some versions of cloze, to facilitate the testees’ task, some pieces of information
related to the deleted words are given to provide a cue. Depending on the kind and
extent of cueing, certain varieties have been developed. In its simplest form, it
resembles a spelling test known as the letter cloze. In this version, one or more letters of
the word in the blank are left out and the testees are required to fill in the blanks with the
missing letters. Consider the example passage in which two letters are deleted from
each word.
Education has two purposes. First, education provides tools so that w__ can earn a living. Some
of t__es__ tools are basic tools, such as r__ad__ng and writing. Others are technical skills
s__c__ as typing, accounting, and data processing. S__m__ others are highly professional skills
peculiar, f__ __ example, to medicine, to law, and t__ teaching. Secondly, education provides
experience so t__ __t we can learn how to live. E__ provides a background of ideas for u__in__
the past. It provides a sense of values for meeting the future.
This form can be made more difficult by reducing the cue. In its most difficult form,
only the initial letter of each word is provided and the rest of the word is to be provided
by the testees. In this form, the example passage would have the following form.
Education has two purposes. First, education provides tools so that w___ can earn a living. Some
of t___ tools are basic tools, such as r___ and writing. Others are technical skills, s___ as typing,
accounting, and data processing. S___ others are highly professional skills peculiar, f___
example, to medicine, to law, and t___ teaching. Secondly, education provides experience so
t___ we can learn how to live. I___ provides a background of ideas for u___ the past. It provides
a sense of values for meeting the future.
This version is often used to check and improve the spelling ability of the students.
These varieties can be utilized at different levels with varying degrees of difficulties.
Another version of cloze that is also based on the rate of the deletion of the words is
called random deletion procedure. In this version, there is neither fixed ratio deletion
nor deleting a particular class of words in the mind. Words are deleted on the basis of
pure randomization. To construct such a test, all the words in the passage are numbered
and every number is written on a piece of paper. These pieces of papers that carry the
numbers corresponding to the words in the passage are located in a hat. Then the
numbers are drawn out of the hat one by one. The word that corresponds to every
number is deleted from the passage. Although random deletion procedure conforms to
the principles of randomization in research and removes some of the criticisms made on
the selection of an arbitrary number for n, it has a drawback. Since words are deleted on
224
Varieties of cloze procedure Hossein Farhady
the basis of the randomly drawn numbers, there is a possibility of having two or more
adjacent words deleted. This would complicate the task of the test takers. Of course, this
problem can be alleviated by drawing the numbers and deleting the words providing
that no adjacent words would be deleted.
A different variety of cloze test is known as the cloze-elide test. In this type, instead of
deleting a set of words from the passage, a certain number of words are added to the
text. The testees are required to read the text and identify the extra or redundant words
and cross them out. In fact, in the cloze-elide test, the processing of the text is somewhat
opposite to that of the standard cloze test. In the cloze test, the testees should read the
text and add some words; whereas in the cloze-elide test, the testees should read the text
and delete some words. Since the cloze-elide test is different from the varieties of the
cloze test mentioned here, certain points should be taken into account in developing and
using a cloze-elide test.
The first point is the selection of the passage. In selecting the passage, all the rules and
regulations regarding the difficulty and the length, discussed before, should be
observed. That is, the selected passage should have appropriate length and difficulty in
respect to the expected group of testees. The second point is determining the locations
where the extra words should be inserted. The most appropriate method of inserting the
words in the text is the random insertion procedure. That is, the words in the passage
should be numbered. Then the numbers should be randomly selected and the redundant
word should be inserted after or before the word that corresponds to the randomly
selected number. The third point is the selection of the words to be inserted in the
passage. Again, a good procedure is selecting certain words from the dictionary on a
random basis. Of course, some scholars believe that the inserted word should be similar
to the adjacent words. However, the difference between inserting words selected
randomly and the words inserted logically needs empirical evidence.
In the sample passage presented before, there are 12 blanks. Therefore, twelve words
should be selected randomly. Following a random insertion procedure, the passage
would look like the following.
Education has two goals. First, it provides very means so that we may earn the our living. Some
of these tools are actual main tools, such as reading or writing. Others may are technical skills,
such as typing, quite accounting, and data processing. Still others are would highly professional
skills peculiar, for example, to such medicine, to science, and to teaching. Secondly, it is
provides experience so that we can learn the how to live. Education provides knowledge that of
ideas for using the past. It after provides a sense of values for the meeting the future.
In this passage, underlined words are inserted. They are redundant and the examinees
should detect them and cross them out. Of course, they are underlined here to inform the
reader. Otherwise, in the original test, the inserted words are written in exactly the same
form as the other words in the text.
In scoring the cloze-elide test, the words that are detected and crossed out correctly will
be given credit. Sometimes, testees cross out the words that are not redundant and are
parts of the text. In such cases, the testees are penalized and one point is deducted for
any error.
225
Varieties of cloze procedure Hossein Farhady
The cloze-elide test as another variety of cloze is an interesting cognitive exercise and it
can be used both as a test and as a class activity.
Scholars believe that a cloze test measures three types of knowledge: linguistic
knowledge, textual knowledge, and knowledge of the world. In a standard cloze, almost
all these types of knowledge are somehow tested (Cohen, 1980). That is, the test takers
should utilize their linguistic knowledge to read the text and understand it. They should
use their textual knowledge to determine the logical relationship between the sentences
and determine the references made in the text. Finally, they should use their knowledge
of the world to map their linguistic and textual knowledge to the context of the text.
However, different versions of the cloze test may focus on utilizing different types of
knowledge. For instance, when a certain class of words such as articles or prepositions
is deleted, the focus might be more on the linguistic knowledge rather than on the other
types of knowledge. Further, when only cohesive ties, such as coordinators, or
transitional expressions are deleted, textual knowledge might be tested more than other
types of knowledge. Thus, depending on the focus of the course and the purpose of the
test, the examiners would utilize one or the other version of the cloze test.
With the varieties presented here, teachers, educators, and testers have access to a
versatile measurement device that can be used for many purposes. The following are
but a few suggestions on the applications of the cloze test.
1. Teachers can use the letter cloze test to improve students' spelling ability at the
elementary levels without much burden on the students. The common procedure to
improve the spelling ability of the learners is to have them copy the words or sentences
for a few times. Copying, although sometimes useful, is often a mechanical activity that
does not stimulate students’ minds, and thus, the outcome of learning is not satisfactory.
Cloze, on the other hand, activates the learners’ minds and leads to a higher
achievement.
2. Teachers at intermediate and advanced levels can use cloze as a cognitive activity. Of
course, some of the students may resist doing the tasks because cloze type activities
may seem nontraditional. However, with a little care and attention, cloze would change
to an enjoyable problem solving activity in almost all classes and at different levels.
3. Educators can utilize varieties of cloze in the learners’ mother tongue. Research has
shown that cloze is a useful technique in improving students’ command in their mother
tongue as well.
4. Teachers can use varieties of oral cloze to improve students’ speaking and listening
ability. Oral cloze is a useful technique to help students practice their production
through cognitive activities.
226
Varieties of cloze procedure Hossein Farhady
Of course, these suggestions are made to provide guidelines for teachers. Most teachers
would employ their creativity in utilizing varieties of cloze in different situations at
different levels. It is hoped that teachers in general and language teachers in particular
will find the cloze procedure quite useful and effective.
*This is a revised version of the paper printed in Roshd Foreign Language Teaching Journal (1996). 12
(44). Tehran, Iran.
227
Varieties of cloze procedure Hossein Farhady
228
Varieties of cloze procedure Hossein Farhady
Bibliography
Cohen, A. (1980). Testing language ability in classroom. Massachusetts: Newbury
House Publishers, Inc.
_____ (1983). New Directions for ESL proficiency testing. In J. W. Jr. Oller (ed.),
Issues in language testing research. Massachusetts: Newbury House Publishers,
Inc.
Farhady, H., Jafarpour, A. & Birjandi, P. (1994). Testing Language Skills: From theory
to practice. Tehran: SAMT Publications.
Oller, J. W. Jr. (1979). Language tests at school. London: Longman Group, Ltd.
Rand, E. (1978). The effects of test length and scoring method on the precision of the
cloze test scores. Workpapers in TESL. UCLA.
Taylor, W.L.(1953). Cloze procedure:a new tool for measuring readability. Journalism
Quarterly 30, 415-433.
229