See discussions, stats, and author profiles for this publication at: https://www.researchgate.

net/publication/329000142

Criteria for the Construction of Tests for Language Assessment and Evaluation

Conference Paper · October 2015

CITATIONS READS

4 2,735

2 authors:

Vanya Ivanova Todorka Terzieva

University of Plovdiv Paisii Hilendarski Plovdiv University "Paisii Hilendarski"
62 PUBLICATIONS   75 CITATIONS    72 PUBLICATIONS   187 CITATIONS   

SEE PROFILE SEE PROFILE

Some of the authors of this publication are also working on these related projects:

NPD-NI15-FMI View project

PhD research View project

All content following this page was uploaded by Vanya Ivanova on 22 November 2018.

The user has requested enhancement of the downloaded file.

 Criteria for the Construction of Tests for
Language Assessment and Evaluation

Vanya Ivanova and Todorka Terzieva

Plovdiv University “Paisii Hilendarski”, Faculty of Mathematics and Informatics

[email protected], [email protected]

Abstract. The purpose of this article is to devise criteria for construct-

ing tests which will promote the development of English language skills
and higher-level cognitive thinking in students. One of the major prob-
lems of both the theory and practice of didactic testing is determining the
objectives and tasks of the educational work, the achievement of which is
diagnosed with tests. Defining the objectives is an important stage of the
overall planning, conducting, and result evaluation of education. In this
paper we propose a set of criteria for evaluating the learning outcomes
based on Bloom’s taxonomy. According to Bloom, there are six main
levels in the cognitive domain that are hierarchically arranged on the
principle “from simple to complex”: knowledge, comprehension, applica-
tion, analysis, synthesis, and evaluation. These levels can be considered
as varying in degree of difficulty – learning at the higher levels depends
on having attained prerequisite knowledge and skills at the lower levels.
For our teaching purposes we used Bloom’s taxonomy to devise five ba-
sic criteria for evaluating the learning outcomes. For each criterion we
defined several types of test questions. We also offer some practical ex-
amples based on the New Headway Academic Skills Level 2 Student’s
Book.
The future work of the authors includes using the test questions in prac-
tice to assess the students’ knowledge and skills after each unit studied
in class and evaluate the progress that the learners have made by com-
paring their results from different tests composed by using the suggested
criteria.

Keywords: test construction, assessment, evaluation, criteria, Bloom’s

taxonomy

1 Introduction
The principles and techniques for test construction have been much discussed
and examined. Tests are widely used to assess language knowledge and skills
acquired by learners and to evaluate the effectiveness of foreign language teaching
and learning. When planning the construction of a test one needs to outline the
subject-matter content of the test, identify the learning outcomes to be measured
by the test, prepare a table of specifications, and choose appropriate type(s) of
test items for evaluation of the learning outcomes as summarized in the table of
 Criteria for the Construction of Tests 59

specifications [4]. One of the major problems of both the theory and practice of
didactic testing is determining the objectives and tasks of the educational work,
the achievement of which is diagnosed with tests. Defining the objectives is an
important stage of the overall planning, conducting, and result evaluation of
the education. The concretization of the objectives is called operationalization,
which is achieved through the respective approaches and methods or by using
the existing taxonomies [12].
Nowadays the focus is on students’ ability to use their knowledge and skills
in order to meet real-life challenges, rather than merely on the extent to which
they have mastered the studied material [2]. This orientation reflects a change
in the goals and objectives of curricula, which are increasingly concerned with
what students can do with the things they learn at school or university, and not
merely whether they can reproduce what they have learned [11].
The purpose of this article is to devise criteria for constructing tests which will
promote the development of English language skills and higher-level cognitive
thinking in students. In this paper we propose a set of criteria for evaluating the
learning outcomes based on Bloom’s taxonomy.
The task of constructing a scheme for structuring the educational objectives
was undertaken for the first time in the USA. In 1956 Benjamin Bloom published
Taxonomy of the Educational Objectives for Cognitive Activities, which proved
to be extremely valuable for the diagnostics of the results from the educational
work [3]. Since its creation, Bloom’s taxonomy has been used for planning the
process of education and evaluating the results from it. It has also served as a
reliable tool for the experimental verification of new learning material. Bloom’s is
not only the most common and widespread taxonomy but also the most modified.
It was developed to advise instructors how to classify educational objectives.
This theory is based on the idea that the learning objectives and outcomes in
education are not identical. For example, memorizing scientific facts, however
important they may be, is placed at a lower level than the ability to analyze or
evaluate.
According to Bloom, there are six main levels in the cognitive domain that
are hierarchically arranged on the principle “from simple to complex”. These
levels can be considered as varying degrees of difficulty – learning at the higher
levels depends on having attained prerequisite knowledge and skills at the lower
levels. This approach enables measurability of students’ knowledge, skills, and
attitudes. Bloom offered six levels of learning objectives – knowledge, compre-
hension, application, analysis, synthesis, and evaluation.
Since the creation of this six-level description of thinking it has repeatedly
been revised and applied under various conditions. Bloom’s taxonomy is also
used for classification purposes in the education of English.
For our teaching purposes we have also considered the changes in the taxon-
omy made by Anderson, a former student of Bloom’s, and Krathwohl, some of
which are [1], [10]:

• changing the names in the six categories from noun to verb forms
• creating a processes and levels of knowledge matrix
60 Vanya Ivanova, Todorka Terzieva

Although the taxonomy of Anderson and Krathwohl is not the only possible
way to classify the levels of thinking, it has a clear structure, facilitates the
organization process of the intellectual development education, starting with the
initial stage of mastering techniques for thinking activity, a transition towards
intellectual operations at a higher level and adopting habits for highly-organized
thinking. The cognitive objectives of the extended taxonomy have universal
nature and could be applied to the education of English.
In the Bulgarian education the cognitive domain of Bloom’s taxonomy oc-
cupies a special place as it has become the foundation for the establishment of
standards, which are associated with the national curriculum.

2 Criteria for the Construction of Tests

In our practice we use tests not only to measure the language acquisition of
students at the Faculty of Mathematics and Informatics at Plovdiv University,
but we also integrate them in the learning process in the form of computer self-
study tests to boost its efficiency. The essence of these practice tests – their
construction, application, evaluation, and achieved results, is explained in [5],
[6], [7], [8], and [9]. In a nutshell, they are devised by the teacher on the basis of
the study material and are practiced online every week in the learners’ own time
and place. The grades from the self-study tests constitute part of the students’
final grades in English.
For our teaching purposes, we used Bloom’s taxonomy to devise five basic
criteria for evaluating the learning outcomes from the English language training.
The first criterion is connected with remembering. It is meant to check
students’ knowledge of words and grammatical constructions. The test-takers
are required to recognize, recall, and reproduce lexical and grammatical units
which have been studied during the academic classes.
The second criterion reviews the students’ comprehension of lexical units.
Test tasks check their understanding of the meaning of words. Students need to
be able to paraphrase words, recognize synonyms and antonyms of words and
expressions, etc.
The third criterion concerns the application of lexical and grammatical
units in practice. Students are instructed to detect and correct different types of
mistakes in a given context.
The fourth criterion is related to analyzing the use of lexical and grammati-
cal units in a context. Students are required to relate their theoretical knowledge
to practice, compare and distinguish various options, and select the most appro-
priate ones for specific cases.
The fifth criterion combines evaluating and creating a text. Students are
expected to organize their ideas into a comprehensible text to summarize their
view points and evaluate other people’s opinions and texts.
 Criteria for the Construction of Tests 61

2.1 Criteria for Evaluating the Learning Outcomes

Below we have defined these five criteria and for each one of them we have offered
several types of test questions with varying degrees of difficulty.

Criterion I. Reproduction of information. 1.Test questions for filling in a

missing word in order to:
a. test the acquisition of studied lexical items.
b. test the acquisition of studied grammatical items.

Criterion II. Understanding the meaning of a word, expression or a

phraseological unit and finding a match. 1. Multiple choice or short-answer
test questions for translating or defining words or phrases.
2. Matching tasks to match words to their meanings.
3. Multiple choice or short-answer test questions for finding/supplying synonyms
or antonyms of words.

Criterion III. Detection and correction of errors in various contexts.

1. True or false test questions:
a. for finding or correcting spelling errors;
b. for finding or correcting grammar errors.
2. Multiple choice test questions:
a. for finding or correcting spelling errors;
b. for finding or correcting grammar errors.

Criterion IV. Analysis of the use of words or expressions and selection

of an appropriate grammatical form of verbs in a context. 1. Multiple
choice test questions to select a sentence which most accurately explains the
meaning of a word, an expression, or phraseological unit used in a given context.
2. Multiple choice test questions to select an appropriate grammatical form of
verbs in a given context.
3. Short-answer open test questions to add the most appropriate lexical or gram-
matical unit in a given context.

Criterion V. Text creation 1. Writing tasks to ask or answer questions.

2. Writing tasks to compose sentences with words or phrases from a text in order
to illustrate their meaning.
3. Long-answer open questions to write an essay on a given topic.
To illustrate each of the proposed criteria, we will present a few test tasks
as an example based on Unit 1 from New Headway Academic Skills Level 2
Student’s Book (Reading, Writing, and Study Skills). The correct options in the
test tasks are given in bold. As the tests on all units are going to be used online,
the questions and answer options are automatically shuffled by the software
and the place of the correct options in the paper version of the tests is of no
importance.
62 Vanya Ivanova, Todorka Terzieva

2.2 Examples of Test Questions Organized by Criteria:

Criterion I. Reproduction of information.

I. Complete the sentences with the correct word(s).

1. I am really looking forward . . . . . . . . . . . . you again!
a. to seeing
b. to see
c. seeing
d. at seeing
2. People do not make natural . . . . . . . . . . . . ; they gather them from the earth.
a. beaches
b. assembly
c. destination
d. resources

Criterion II. Understanding the meaning of a word, expression, or a

phraseological unit.

I. Match the word with its definition.

3. skimming
a. quick reading to get the general idea of a text
b. quick reading to find particular pieces of information
c. slow reading for details including note making and highlighting important
points
d. reading for pleasure
4. skyscraper
a. a tool that has a small handle and a metal or plastic blade and can be used
for cleaning a particular surface
b. a very tall building usually found in cities
c. a four-engined, double-decked passenger airplane
d. small military airplane that is made to fight other airplanes

II. Find a synonym of the word (a word with a similar meaning).

5. appropriate
a. applicable
b. irrelevant
c. abundant
d. redundant

III. Find an antonym of the word (a word with the opposite meaning).
6. summarize
a. expand
b. outline
c. review
d. calculate
 Criteria for the Construction of Tests 63

Criterion III. Detection and correction of errors in various situations.

I. Choose the part of the sentence which contains a spelling or grammar mistake.

7. I am really exciting about coming to Bulgaria for the first time to do a course
in national folklore.
a. really exciting
b. about coming to
c. to do a course
d. in national folklore

II. Mark the sentences as True if they are free of lexical or grammatical errors,
or False if you have found a mistake.
8. I read the article twice to find relevant information for my project.
a. True
b. False
9. Although she can speak five languages but she doesn’t have a job.
a. True
b. False

Criterion IV. Analysis of the meaning and finding a match.

I. Complete the sentences with the correct word(s).

10. At the moment I . . . . . . . . . on a project about space.
a. am working
b. work
c. have worked
d. was working

II. Choose the sentence below which explains the meaning of the given one.
11. Correct punctuation is very important for students in literature.
a. It is essential that students in literature use commas, hyphens,
capital letters, etc. in their right places.
b. It is very important for students in literature never to be late.
c. It is crucial that students in literature say the words they use clearly and
correctly.
d. It is essential that students in literature do not hit each other.

Criterion V. Text creation 12. Imagine you are going to stay at an old
classmate’s house for a weekend to celebrate his/her birthday. Write an informal
email of about 100 words. Say that you accept the offer. Tell them briefly
about what you are doing nowadays. Give him/her information about
your arrival. Ask for information that you would like to have. (6 points)
64 Vanya Ivanova, Todorka Terzieva

2.3 Grading
Every test question except for the tasks from Criterion 5, when answered cor-
rectly, adds one point to the final score, while the long-answer open questions
give 6 points each. The total grades correspond to academic grades in the fol-
lowing manner:
18 – 24 p. (50–69%) – Satisfactory 3
25 – 27 p. (70–79%) – Good 4
28 – 31 p. (80–89%) – Very good 5
32 – 36 p. (90–100%) – Excellent 6

2.4 Future Work

The future work of the authors includes using the test questions in practice with
students from the Faculty of Mathematics and Informatics at Plovdiv University
to check the learners’ knowledge and skills after each unit studied in class and
evaluating the progress that they have made by comparing their results from
different tests composed by using the suggested criteria.

Acknowledgements

The study has been partially supported by the Scientific Research Fund at Plov-
div University “Paisii Hilendarski”, project NI15-FMI-004.

References
1. Anderson, L., Krathwohl, D.: Bloom’s Taxonomy Revised,
http://thesecondprinciple.com/teaching-essentials/
beyond-bloom-cognitive-taxonomy-revised/, last visited on 23.11.2015
(2013)
2. Angelova, E., Rahnev, A.: Boosting Teaching and Learning Efficiency in Training
Teachers of Information Technology, Scientific Works, Plovdiv University, vol. 36,
book 3, Mathematics, 5–18, (2009)
3. Bloom, B.: Taxonomy of Educational Objectives, Longman, (1956)
4. Devine, M., Yaghlian, N.: Test Construction Manual, Center for Teaching
Excellence, http://www.cte.cornell.edu/documents/Test%20Construction%
20Manual.pdf, last visited on 23.11.2015
5. Ivanova, V.: Construction and Evaluation of Achievement Tests in English, Na-
tional Conference “Education in the Information Society”, May 2011, ISSN 1314-
0752, pp. 276-285
6. Ivanova, V., Voynikova, D.: Intensification of the Processes of Testing and Evalua-
tion with the Help of IT by Integrating Self-Study Tests in Teaching and Learning
English, Medical University “Prof. Dr. Paraskev Stoyanov”, Varna, 2015, ISBN
978-954-449-770-5, 263–270 (in Bulgarian) (2015)
7. Ivanova, V.: Raising the Achievement of Students in English via Self-Study Tests,
University of National and World Economy, Sofia, 8 October 2015, (to print), (in
Bulgarian)
 Criteria for the Construction of Tests 65

8. Ivanova, V.: Use of Achievement Tests in the English Classes at the Distributed E-
Learning Centre, Proceedings of the International Conference FROM DELC TO
VELSPACE, Plovdiv, 26–28 March 2014, Third Millennium Media Publications
ISBN: 0-9545660-2-5, 167–174.
9. Ivanova, V., Stoyanova-Doycheva, .: Intensification of the Learning Process through
Self-Study Tests in Blended Learning, International Research Conference “Knowl-
edge – Traditions, Innovations, Perspectives”, Burgas Free University, v. 4, 2013,
http://ojs.bfu.bg/index.php/knk/article/view/118/139
10. Krathwohl, D.: Revising Bloom’s Taxonomy, 2002, http://www.unco.edu/cetl/
sir/stating{\_}outcome/documents/Krathwohl.pdf, last visited on 23.11.2015.
11. OECD, Learning for Tomorrow’s World – First Results from PISA 2003, Pro-
gramme for International Student Assessment, France, 2004, ISBN 92-64-00724-5
– No. 53799 2004
12. Terzieva, T.: Some criteria and indicators for diagnosis of the forming of algorithmic
thinking in Computer Science, Scientific Works, Plovdiv University, vol. 38, Book
3, 2011, Mathematics

View publication stats