CHAPTER 4

ADMINISTERING, ANALYZING, AND IMPROVING TESTS

Key Terms

item analysis
difficulty index
discrimination index
measures of attractiveness
positive discrimination
negative discrimination
zero discrimination
miskeyed item
guessing item
ambiguous item

Learning Objectives

At the end of this chapter, the students should be able to:

1. Define the basic concepts regarding item analysis;

2. Identify the steps in improving test items;
3. Solve difficulty index and discrimination index;
4. Identify the level of difficulty of an item;
5. Perform item analysis properly and correctly;
6. Identify the item to be rejected, revised, or retained; and
7. Interpret the results of item analysis.
INTRODUCTION

One of the most important functions of a teacher is to assess the

performance of the students. This is a very complicated task because you will
consider many activities such as the timing of the assessment process, the format
of the assessment tools and the duration of the assessment procedures.

After designing the assessment tools, package the test, administer the test
to the students, check the test papers, score and then record them. Return the test
papers and then give feedback to the students regarding the result of the test.

PACKAGING AND REPRODUCING TEST ITEMS

Assuming that you have already assembled the test, you write the
instructional objectives, prepare the table of specification, and write the test
items that match with the instructional objectives, the next thing to do is to
package the test and reproduce it as discussed in the previous chapter.

1. Put the items with the same format together.

2. Arrange the test items from easy to difficult.
3. Give proper spacing for each item for easy reading.
4. Keep questions and options in the same page.
5. Place the illustrations near the options.
6. Check the key answer.
7. Check the direction of the test.
8. Provide space for name, date and score.
9. Proofread the test.
10. Reproduce the test.

ADMINISTERING THE EXAMINATION

After constructing the test items and putting them in order, the next step is
to administer the test to the students. The administration procedures greatly
affect the performance of the students in the test. The test administration does
not simply means giving the test questions to the students ad collecting the test
papers after the given time. Below are the guidelines in administering the test
before, during and after the test.

Guidelines Before Administering Examinations

1. Try to induce positive test-taking attitude.

2. Inform the students about the purpose of the test.
3. Give oral directions as early as possible before distributing the tests.
 4. Give test-taking hints about guessing, skipping, and the like, are strictly
prohibited.
5. Inform the students about the length of time allowed for the test. If
possible, write on the board the time in which they must be finished with
answering the test. Give the students a warning before the end of the time
limit.
6. Tell the students how to signal or call your attention if they have a question.
7. Tell the students what to do with their papers when they are done
answering the test (how papers are to collected).
8. Tell the students what to do when they are done with the test, particularly
if they are to go on to another activity (also write these directions on the
chalkboard so they can refer to them).
9. Rotate the method of distributing papers so you don’t always start from
the left or the front row.
10. Make sure the room is well lighted and has a comfortable temperature.
11. Remind students to put their names on their papers (and where to do so).
12. If the test has more than one page. Have each student checked to see that all
pages are there.

Guidelines During the Examination

1. Do not give instructions or avoid talking while examination is going on to

minimize interruptions and distractions.
2. Avoid giving hints.
3. Monitor to check student progress and discourage cheating.
4. Give time warning if students are not pacing their work appropriately.
5. Make a note of any questions students ask during the test so that items can
be revised for future use.
6. Test papers must be collected uniformly to save time and to avoid test papers
to be misplaced.
Guideline After the Examination

After the examination, the next activity that the teacher needs to do is to
score the test papers, record the result of the examination; return the test papers and
last to discuss the test items in the class so that you can analyze and improve the test
items for future use.

1. Grade the papers (and add comments if you can); do test analysis (see the
module on test analysis) after scoring and before returning papers to
students if at all possible. If it is impossible to do your test analysis before
returning the papers, be sure to do it at another time. It is important to do
both the evaluation of your students and the improvement of your tests.
2. If you are recording grades or scores, record them in pencil in your class
 record before returning the papers. If there are errors/ adjustments in
grading they (grades) are easier to change when recorded in pencil.
3. Return papers in a timely manner.
4. Discuss test items with the students. If students have questions, agree to
look over their papers again, as well as the papers of others who have the
same question. It is usually better not to agree to make changes in grades
on the spur of the moment while discussing the tests with the students but
to give yourself time to consider what action you want to take. The test
analysis may have already alerted you to a problem with a particular
question that is common to several students, and you may already have
made a decision regarding, that question (to disregard the question and
reduce the highest possible score according, to give all students credit for
that question, among others).

ANALYZING THE TEST

After administering and scoring the test, the teacher should also analyze
the quality of each item in the test. Through this you can identify the item that is
good, item that needs improvement or items to be removed from the test. But
when do we consider that the test is good? How do we evaluate the quality of
each item in the test? Why is it necessary to evaluate each item in the test? Lewis
Aiken (1997) an author or psychological and educational measurement pointed
out that a “postmortem” is just as necessary in classroom assessment as it is in
medicine.

In this section, we shall introduce the technique to help teachers

determine the quality of a test item known as item analysis. One of the purposes
of item analysis is to improve the quality of the assessment tools. Through this
process, we can identify the item that is to be retained, revised or rejected and
also the content of the lesson that is mastered or not.

There are two kinds of item analysis, quantitative item analysis and
qualitative item analysis (Kubiszyn and Borich, 2007).
Item Analysis

Item analysis is a process of examining the student’ response to individual

item in the test. It consists of different procedures for assessing the quality of the
test items given to the students. Through the use of item analysis we can identify
which of the given are good and defective test items. Good items are to be
retained and defective items are to be improved, to be revised or to be rejected.
Uses of Item Analysis

1. Item analysis data provide a basis for efficient class discussion of the
test results.
2. Item analysis data provide a basis for remedial work.
3. Item analysis data provide a basis for general improvement of
classroom instruction.
4. Item analysis data provide a basis for increased skills in test construction.
5. Item analysis procedures provides a basis for constructing test bank.

Types of Quantitative Item Analysis

There are three common types of quantitative item analysis which provide
teachers with three different types of information about individual test items.
These are difficulty index, discrimination index, and response options analysis.

1. Difficulty Index
It refers to the proportion of the number of students in the upper
and lower groups who answered an item correctly. The larger the
proportion, the more students, who have learned the subject is measured
by the item. To compute the difficulty index of an item, use the formula:
n
D F= , where
N
DF = difficulty index
n = number of the students selecting item correctly in the upper
group and in the lower group.
N = total number of students who answered the test

Level of Difficulty

To determine the level of difficulty of an item, find first the difficulty

index using the formula and identify the level of difficulty using, the range given
below.

Index Range Difficulty Level

0.00 – 0.20 Very Difficult
0.21 – 0.40 Difficult
0.41 – 0.60 Average/ Moderately
Difficult
0.61 – 0.80
Easy
0.81 – 1.00
Very Easy
 The higher the value of the index of difficulty, the easier the item is. Hence, more
students got the correct answer and more students mastered the content
measured by that item.

2. Discrimination Index
The power of the item to discriminate the students between those
who scored high and those who scored low in the overall test. In other
words, it is the power of the item to discriminate the students who know
the lesson and those who do not know the lesson.
It also refers to the number of students in the upper group who got
an item correctly minus the number of students in the power group who
got an item correctly. Divide the difference the difference by either the
number of the students in the upper group or number of students in the
lower group or get the higher number if they are not equal.
Discrimination index is the basis of measuring the validity of an item.
This index can be interpreted as an indication of the extent to which overall
knowledge of the content area or mastery of the skills is related to the
response on an item.

Types of Discrimination Test

There are three kinds of discrimination index: p0ositive discrimination,

negative discrimination and zero discrimination.

1. Positive discrimination happens when more students in the uppe group

got the item correctly than those students in the lower group.
2. Negative discrimination occurs when more students in the lower group
got the item correctly than the students in the upper group.
3. Zero discrimination happens when a number of students in the upper
group and lower who answer the test correctly are equal, hence, the test
item cannot distinguish the students who performed in the overall test and
the students whose performance are very poor.

Level of Discrimination
 Ebel and Frisbie (1986) as cited by Hetzel (1997) recommended the use of
Level of Discrimination of an Item for easier interpretation.
Index Range Discrimination Level
0.19 and Poor item, should be eliminated or need to be revised
below
0.20 – 0.29 Marginal item, needs some revision
0.30 – 0.39 Reasonably good item but possibly for improvement
0.40 and Very good item
above

Discrimination Index Formula

CUG −C LG
D I= , where
D

DI = discrimination index value

CUG = number of the students selecting the correct answer in the upper

group

CLG = number of the students selecting the correct answer in the lower

group

D = number of students in either the lower group or upper group.

Note: Consider the higher number in case the sizes in upper and lower group
are not equal

Steps in Solving Difficulty Index and Discrimination Index

1. Arrange the scores from higher to lowest.

2. Separate the scores into upper group and lower group. There are different
methods to do this: (a) if a class consists of 30 students who takes an
exam, arrange their scores from highest to lowest, then divide them into
two groups. The highest score belongs to the upper group. The lowest
score belongs to the lower group. (b) Other literatures suggested to use
27%, 30%, or 33% of the students for the upper group and lower group.
However, in the Licensure Examination for Teachers (LET) the test
developers always used 27% of the students who participated in the
examination for the upper and lower groups.
3. Count the number of those who chose the alternatives in the upper and
 lower group for each item and record the information using the template:
Options A B C D E
Upper Group
Lower Group

Note: Put asterisk for the correct answer.

4. Compute the value of the difficulty index and the discrimination index
ans also the analysis of each response in the distracters.
5. Make an analysis for each item.

Checklist for Discrimination Index

It is very important to determine whether the test item will be retained
revised or rejected. Using the Discrimination Index we can identify the nonperforming
question items; just always remember that they seldom indicate what is the
problem. Use the given checklist below:

Yes No
1. Does the key discriminate positively?
2. Does the incorrect options discriminate negatively?
If the answer to questions 1 and 2 are both YES, retain the item.

If the answers to questions 1 and 2 are either YES or NO, revise the item.

If the answer to questions 1 and 2 are both NO, eliminate or reject the item.

3. Analysis or Response Options

Aside from identifying the difficulty index and discrimination index, another
way to evaluate the performance of the entire test item is through the
analysis of the response options. It is very important to examine the
performance of each option in a multiple-choice item. Through this, you
can determine whether the distracters or incorrect answer. The
attractiveness of the incorrect options is determined when more students
in the lower group than in the upper group choose it. Analyzing the
incorrect options allows the teachers to improve the test items so that it can
be used again in the future.

Distracter Analysis

1. Distracter
 Distracter is the term used for the incorrect options in the muliple-choice
type of test while the correct answer represents the key. It is very
important for the test writer to know if the distracters are effective or good
distracters. Using quantitative item analysis we can determine if the
options are good or if the distracters are effective.

Item analysis can identify non-performing test items, but this item seldom
indicates the error or the problem in the given item. There are factors to
be considered why students failed to get the correct answer in the given
question.

a. It is not taught in the class properly.

b. It is ambiguous.
c. The correct answer is not in the given options.
d. It has more than one correct answer.
e. It contains grammatical clues to mislead the students.
f. The student is not aware of the content.
g. The student were confused by the logic of the question because it has
double negatives.
h. The student failed to study the lesson.
2. Miskeyed item
The test item is a potential miskey if there are more students from the
upper group who choose the incorrect options than the key.
3. Guessing item
Students from the upper group have equal spread of choices among the
given alternatives. Students from the upper group guess their answers
because of the following reasons:
a. The content of the test is not discussed in the class or in the text.
b. The test item is very difficult.
c. The question is trivial.
4. Ambiguous item
This happens when more students from the upper group choose equally an
incorrect option and the keyed answer.

Qualitative Item Analysis

Qualitative item analysis (Zurawski, R.M) is a process in which the

teacher or expert carefully proofreads the test before it is administered, to check
if there are typographical errors, to avoid grammatical clues that may lead to
giving away the correct answer, and to ensure that the level of reading materials
is appropriate. These procedures can also include small group discussions on the
quality of the examination and its items, with examinees that have already took
the test. According to Cohen, Swerdlik, and Smith (1992) as cited by Zurawski,
students who took the examination are asked to express verbally their experience
in answering each item in the examination. This procedure can help the teacher
in determining whether the test takers misunderstood a certain item, and it can
help also in determining why they misunderstood a certain item.

IMPROVING TEST ITEMS

As presented in the introduction of this chapter, item analysis enables the

teachers to improve and enhance their skills in writing test items. To improve
multiple- choice test item we shall consider the stem of the item, the distracters
and the key answer.

How to Improve the Test Item

Consider the following examples in analyzing the test item and some
notes on how to improve the item based from the results of items analysis.

Example 1. A class is composed of 40 students. Divide the group into two.

Option B is the correct answer. Based from the given data on the table, as a teacher,
what would you do with the test item.
Option A B* C D E
Upper 3 10 4 0 3
Group
Lower 4 4 8 0 4
Group

1. Compute the difficulty

index. n = 10 + 4 = 14
N = 40
D n
F=¿ ¿
N
D 14
F=¿ ¿
40
D F=¿0.35 ∨35 % ¿

2. Compute the discrimination

index. CUG = 10
CLG = 4
D = 20
CUG −C LG
D I=
D
 10− 4
D I=
20
6
D I=
20
D I =0.30∨30 %

3. Make an analysis about the level of difficulty, discrimination and distracters.

a. Only 35% of the examines got the answer correctly, hence, the item is
difficult.
b. More students from the upper group got the answer correctly, hence, it
has a positive discrimination.
c. Retain options A, C, and E because most of the students who did not
perform well in the overall examination selected it. Those options attract
most students from the lower group.
4. Conclusion: Retain the test item but change option D, make it more
realistic to make it effective for the upper and lower groups. At least 5%
of the examinees choose the incorrect option.

Example 2.A class is composed of 50 students. Use 27% to get the upper and the
lower groups. Analysis the item given the following results. Option D is the
correct answer. What will you do with the test item?

Option A B C D* E
Upper Group 3 1 2 6 2
(27%)
Lower Group 5 0 4 4 1
(27%)
1. Compute the difficulty
index n = 6 +4 = 10
N = 28
D n
F=¿ ¿
N
D 10
F=¿ ¿
28
D F=¿0.36 ∨36 % ¿

2. Compute the discrimination

index. CUG = 6
CLG = 4
D = 14
 CUG −C LG
D I=
D
6 −4
D I=
14
2
D I=
14
D I =0.14∨14 %

3. Make an analysis
a. Only 36% of the examinees got the answer correctly, hence, the item is
difficult.
b. More students from the upper group got the answer correctly, hence, it
has a positive discrimination.
c. Modify options and B and E because more students from the upper
group chose them compare with the lower group, hence, they are not
effective distracters because most of the students who performed well
in the overall examination selected them as their answer.
d. Retain options A and C because most of the students who did not perform
well in the overall examination selected them as the correct answer.
Hence, options A and C are effective distracters.
4. Conclusion: Revised the item by modifying options B and E.

Example 3.A class is composed of 50 students. Use 27% to get the upper and the
lower groups. Analyze the item given the following results. Option E is the
correct answer. What will you do with the test item?

Option A B C D E*
Upper Group 2 3 2 2 5
(27%)
Lower Group 2 2 1 1 8
(27%)
1. Compute the difficulty
index: n = 5 + 8 = 13
N = 28
D n
F=¿ ¿
N
D 13
F=¿ ¿
28
D F=¿0.46 ∨46 % ¿

2. Compute the discrimination

index. CUG = 5
CLG = 8
D = 14
 CUG −C LG
D I=
D
5− 8
D I=
14
−3
D I=
14
D I =− 0.21∨−21 %

3. Make an analysis.
a. 46% of the students got the answer to test item correctly, hence, the
test item is moderately difficult.
b. More students from the lower group got the item correctly; therefore, it
is a negative discrimination. The discrimination index is -21%.
c. No need to analyze the distracters because the item discriminates negatively.
d. Modify all the distracters because they are not effective. Most of the
students in the upper group chose the incorrect options. The options
are effective if most of the students in the lower group chose the
incorrect options.
4. Conclusion: Reject the item because it has a negative discrimination index.

Example 4.Potential Miskeyed Item. Make an item analysis about the table below.

What will you do with the test that is a potential miskeyed item?

Option A* B C D E
Upper Group 1 2 3 10 4
Lower Group 3 4 4 4 5

1. Compute the difficulty

index: n = 1 + 3 = 4
N = 40
D n
F=¿ ¿
N
D 4
F=¿ ¿
40
D F=¿0.10 ∨10 % ¿

2. Compute the discrimination

index. CUG = 1
CLG = 3
 D = 20
CUG −C LG
D I=
D
1− 3
D I=
20
−2
D I=
20
D I =− 0.10∨−10 %
3. Make an analysis.
a. More students from the upper group choose option D than option A,
even though option A is supposedly the correct answer.
b. Most likely the teacher has written the wrong answer key.
c. The teacher checks and finds out that he/she did not miskey the
answer that he/ she though is the correct answer.
d. If the teacher, miskeyed it, he/ she must check and retally the scores
of the students’ test papers before giving them back.
e. If option A is really the correct answer, revise to weaken option D,
distracters are not supposed to draw more attention than the keyed
answer.
f. Only 10% of the students got the answer to the test item correctly, hence,
the test item is very difficult.
g. More students from the lower group got the item correctly,
therefore a negative discrimination resulted. The discrimination index
is -10%.
h. No need to analyze the distracters because the test item is very
difficult and discriminates negatively.
4. Conclusion: Reject the item because it is very difficult and has a
negative discrimination.

Example 5.Ambiguous Item.Below is the result of item analysis of a test with an

ambiguous test item. What can you say about the item? Are you going to retain,
revise or reject it?

Option A B C D E*
Upper Group 7 1 1 2 8
Lower Group 6 2 3 3 6

1. Compute the difficulty

index: n = 8 + 6 = 14
N = 39
 D n
F=¿ ¿
N
D 14
F=¿ ¿
39
D F=¿0.36 ∨36 % ¿

2. Compute the discrimination

index. CUG = 8
CLG = 6
D = 20
CUG −C LG
D I=
D
8 −6
D I=
20
2
D I=
20
D I =0.10∨10 %
3. Make an analysis.
a. Only 36% of the students got the answer to the test item correctly, hence,
the test item is difficult.
b. More students from the upper group got the item correctly, hence, it
discriminates positely. The discrimination index is 10%.
c. About equal numbers of top students went for option A and option E,
this implies that they could not tell which is the correct answer. The
students do not know the content of the test, thus, a reteach is needed.
4. Conclusion: revise the test item because it is ambiguous.

Example 6.Guessing Item.Below is the result of item analysis for a test with students’
answers mostly based on a guess. Are you going to reject, revise or retain the test
item?

Option A B C* D E
Upper Group 4 3 4 3 6
Lower Group 3 4 3 4 5

1. Compute the difficulty

index: n = 4 + 3 = 7
N = 39
D n
F=¿ ¿
N
D 7
F=¿ ¿
39
D F=¿0.18 ∨18 % ¿
 2. Compute the discrimination
index. CUG = 4
CLG = 3
D = 20
CUG −C LG
D I=
D
4−3
D I=
20
1
D I=
20
D I =0.05∨5 %

3. Make an analysis.
a. Only 18% of the students got the answer to the test item correctly, hence,
the test item is very difficult.
b. More students from the upper group got the correct answer to the test
item; therefore, the test item is a positive discrimination. The
discrimination index is 5%.
c. Students respond about equally to all alternatives, an indication that
they are quessing.
Three possibilities why student guesses the answer on a test item:
 The content of the test item had not yet been discussed in
the class because the test is designed in advanced;
 Test items were badly written that students have no idea
what the question is really about; and
 Test items were very difficult as shown from the difficulty
index and low discrimination index.
4. Conclusion: Reject the item because it is very difficult; reteach the material
to the class.

Example 7.Guessing Item.The table below shows an item analysis of a test item
with ineffective distracters. What can you conclude about the test item?

Option A B C* D E
Upper Group 5 3 9 0 3
Lower Group 6 4 6 0 4

1. Compute the difficulty

index: n = 9 + 6 = 15
N = 40
D n
F=¿ ¿
N
 D 15
F=¿ ¿
40
D F=¿0.38 ∨38 % ¿

2. Compute the discrimination

index. CUG = 9
CLG = 6
D = 20
CUG −C LG
D I=
D
9 −6
D I=
20
3
D I=
20
D I =0.15∨15 %

3. Make an analysis.
a. Only 38% of the students got the answer to the test item correctly, hence,
the test item is difficult.
b. More students from the upper group answered the test item correctly;
as a result, the test got a positive discrimination. The discrimination
index is 15%.
c. Options A, B and E are attractive distracters.
d. Option D is ineffective, therefore, change it with more realistic one.
4. Conclusion: Revise the item by changing option D.
Chapter Exercises

Answer the following questions properly.

Discuss the different guidelines in packaging and administering the test.

1. What are the steps in item analysis?
2. What are the two kinds of item analysis?
3. Differentiate quantitative item analysis from qualitative item analysis. What are the advantages of
a quantitative item analysis over qualitative item analysis?
4. Why is item analysis necessary in the assessment process?
5. Differentiate difficulty index from discrimination index.
6. How do you determine the difficulty level of a test item?
7. How would you determine the discrimination level of a test item?
8. How do you determine whether the test item is ambiguous, miskey or guessing item?
9. What is the importance of analyzing the options in each item in a multiple-choice test?
10. When do we retain, revise, or reject an item?
11. When do we consider that a distracter is plausible?
12. When do we say that a distracter is effective and attractive?
13. Compute the difficulty and discrimination index of a test item number 6 admin- istered to 40
students in Statistics class. Twenty-seven percent (27%) of the stu- dents belong to the upper
group and 27% belongs to the lower group. There are 5 students from the upper group who got
the item correctly and 9 from the lower group got the item right.
a. Compute the difficulty index.
b. What is the difficulty level?
c. Compute the discrimination index.
d. What is the discrimination level?
e. Is item number 6 a good item?
f. What will you do with item number 6?
14. A 25-item multiple-choice test in Elementary Algebra with four options was re- corded below.
Listed were a number of students in the lower and upper groups who answered A, B, C and D.
The letter or option with an asterisk is the correct answer.
Item 10 A B C D*
Upper Group (27%) 4 5 2 9
Lower Group (27%) 6 4 5 5

Based from the given table answer the following:

a. Give the difficulty index.
b. What is the level of difficulty?
c. Indicate the discrimination index.
d. What is the discrimination level?
e. Which group got more correct answer?
f. Which of the option is the most effective?
g. Which of the option(s) is/are ineffective?
h. Is item 10 a positive discrimination or a negative discrimination?
i. What can you say about options A and C? Are they effective distracters? How about option B?
What will you do about option B?