Group 3

Administering, Analysing, and Improving Test

Topic; PACKAGING AND REPRODUCING TEST ITEMS

Assuming that we already assembled the test, the next thing to do is to package the test and
reproduce it.
The following are the steps on how to package and reproduce the test.
1. Put the items with the same format together.
2. Arrange the test items from easy to difficult.
3. Give proper spacing for each item for easy reading.
4. Keep questions and options in the same page.
5. Place the illustration near the options.
6. Check the key answer.
7. Check the direction of the test.
8. Provide space for name, date and score.
9. Proofread the test.
10. Reproduce the test.

Topic: Administering the Examination

Guidelines before Administering Examination
1. Try to induce positive test- taking attitude.
2. Inform the students about the purpose of the test.
3. Give oral direction as early as possible before distributing the test.
4. Give test- taking hints about guessing, skipping, and the like, are strictly prohibited.
5. Inform the students about the length of time allowed for the test.
6. Tell the students how to signal or call your attention if they have a question.
7. Tell the students what to do with their papers when they are done answering the test.
8. Tell the students what to do when they are done in the test particularly, if they are to go on to
another activity.
9. Rotate the method of distributing papers so you don’t always start from the left or the front row.
10. Make sure the room is well lighted and has a comfortable temperature.
11. Reminds students to put their names on their papers.
12. If the test has more than one page, have each student checked to see that all pages are there.
Guidelines during Examination

1. Do not give instructions or avoid talking while examination is going on to minimize interruptions and
distractions.

2. Avoid giving hints.

3. Monitor to check student progress and discourage cheating.

4. Give time warnings if students are not pacing their appropriately.

5. Make a note of any questions students ask during the test so that items can be revised for future use.

6. Test papers must be collected uniformly to save time and to avoid test papers to be misplaced.

Guidelines after the Examination

After the examination, the next activity that the teacher needed to do is to score the test papers,
record the result of the examination, return the test papers and last to discuss the test items in the class
so that you can analyze and improve the test items for future use.
 1. Grade the papers (and add comments if you can); do test analysis (see the module on test analysis)
after scoring and before returning papers to students if at all possible. If it is impossible to do your test
analysis before returning papers, be sure to do it at another time. It is important to do both the evaluation
of your students and the improvement of your tests.

2. If you are recording grades or scores, record them in pencil in your class record before returning the
papers. If there are errors/adjustments in grading, they (grades) are easier to change when recorded in
pencil.

3. Return papers in a timely manner.

4. Discuss test items with the students. If students have questions, agree to look over their papers again,
as well as the papers of others who have the same question. It is usually better not to agree to make
change in grades on the spur of the moment while discussing the test with the students but to give
yourself time to consider what action you want to take. The test analysis may have already alerted you
to a problem with a particular question that is common to several students, and you must already have
made a decision regarding that question (to disregard the question and reduce the highest possible
score accordingly, to give all students credit for that question, among others)

Analyzing the Test

After Administering and scoring the test, the teacher should also analyze the quality of each item
in the test. Through this you can identify the item that is good, item that needs improvement or items to
be removed from the test.

There are two kinds of item analysis, quantitative item analysis and qualitative item analysis (kubiszyn
and Borich, 2007).

Item Analysis

Item Analysis is a process of examining the student's response to individual item in the test. It consists of
different procedures for assessing the quality of the test items given to the students. Through the use of
item analysis we can identify which of the given are good and defective test items. Good items are to be
retained and defective items are to be improved, to be revised or to be rejected.

Uses of Item Analysis

1. Item analysis data provide a basis for efficient class discussion of the test results.
2. Item analysis data provide a basis for remedial work.
3. Item analysis data provide a basis for general improvement of classroom instruction.
4. Item analysis data provide a basis for increased skills in test construction.
5. Item analysis procedures provide a basis for constructing test bank.
Quantitative Item Analysis

 It deals with and helps mostly with ideas and development of test items.
It considers scores in how they resulted as of the order and matters of questions that decided the
marks of the score.
 Identify problem areas, strength and weaknesses in structural itemized test. In the textbook it
states that “ As mentioned quantitative item analysis is a technique that will enable us to assess
the quality or utility of an item”p234
 It is mainly objective and applicable to exams that are multiple choices or answers that are
already defined to be chosen from.
 Quantitative item analysis is useful but limited. It points out items that have problems but does not
tell us what the problems are. It is possible that an item that fails to measure or match an
instructional objective could have an acceptable difficulty level.
 Displays the correlation of each item with every other item.
 Provides important information for increasing the test’s internal consistency.

Types of Quantitative Item Analysis

 There are 3 common types of quantitative item analysis which provides teachers with the different
types of information about individual test items. These are difficulty index, discrimination index, and
response option analysis.

1. Difficulty Index
 It refers to the proportion of the number of students in the Upper Group and Lower Group,
who answered an item correctly. The larger the proportion the more students who have
learned.
 The proportion of test takers who answer an item correctly
 Examine the test item whether it is too easy (very easy) or too difficult (very difficult)
Formula :

Df= n/N , where Df= difficulty index,

n=number of students selecting the item correct, N= total number of students who
answered the test.

Level of Difficulty

 To determine the level of difficulty of an item, find first the difficulty index using the formula and
identify the level of difficulty using the range below.

Level of the Difficulty of an Item

Index Range Difficulty Level

0.00 -0.20 Very difficult
0.21- 0.40 Difficult
0.41- 0.60 Average/moderately difficult

0.61- 0.80 Easy

0.81- 1.00 Very easy
*take note: the higher the value of the index difficulty the easier the item. It means more students
mastered the content.

Example:

Imagine a classroom of 40 students who took a test which included the item below. Use 27% to
get the upper and lower group. Analyze the item . Option D is the correct answer.

1. Who wrote the Great Gatsby?

a. Faulkner b. Hemingway c. Steinbeck d. *Fitzgerald e. J.k Rowling
Options A B C * D E
Upper Group 3 4 0 10 3
(27%)
Lower Group 4 8 0 4 4
(27%)

Compute the difficulty index

n = 10+4 = 14

N= 40

Df = n / N

Df = 14/40

Df = 0.35 or 35 %
Therefore, only 35% of the examinees got the correct answer, hence the item is difficult.

Example 2 ( I will let my classmate solve it)

Analyze the data below

Options A B C D *E
Upper 2 3 2 2 5
group
(27%)
Lower 2 2 1 1 8
group
(27%)

Compute the difficulty index

n = 5 + 8 = 13

N = 28

Df = n / N

Df = 13 / 28

Df = 0.46 or 46%

Therefore, 46% of the students got the correct answer, hence the item is moderately difficult

2. Discrimination Index
 The power of the item to discriminate the students between those who scored high and those who
scored low in the overall test. In other words, it is the power of an item who know the lesson and
does not know the lesson.
 It also refers to the number of students in the upper group who got an item correct minus the
number of students in the lower group who got an item correctly. Divide the difference by either
the number of students in the upper group or number of students in the lower group or get the
higher number if they are not equal.
 It is the bases in measuring the validity of an item.

Types of Discrimination Index

There are three kinds of discrimination index: positive discrimination, negative discrimination,
zero discrimination.
1. Positive discrimination
 Happens when more students in the upper group got the item correctly than those students in
lower group.
2. Negative discrimination
 Occurs when more students in the lower group got the item correctly than the students in upper
group.
3. Zero discrimination
 Happens when a number of students in the upper group and lower group who answers the item
correctly are equal, hence, the test item cannot distinguish the students who performed the
overall test and the student whose performance are very poor.
Level of Discrimination

Ebel and Frisbie (1986) as cited by Hetzel (1997) recommended the use of Level of Discrimination of an
item for easier interpretation.
Index Range Discrimination Level
0.19 and below Poor item, should be eliminated or need
to be revised

0.20 to 0.29 Marginal item, needs some revision

0.30 to 0.39 Reasonably good item but passively for

improvement
0.40 and above Very good item

Discrimination Index Formula

Di = Cug – Clg / D , where

Di = discrimination index value

Cug = number of students selecting the correct answer in the upper group

Clg = number of students selecting the correct answer in the lower group

D = number of students in either the lower group or upper group

Note: consider the higher number in case the sizes in upper or lower group are not equal.

Example

Imagine a classroom of 40 students who took a test which included the item below. Use 27% to
get the upper and lower group. Analyze the item . Option D is the correct answer.

1. Who wrote the Great Gatsby?

a. Faulkner b. Hemingway c. Steinbeck d. *Fitzgerald e. J.k Rowling

Options A B C * D E
Upper Group 3 4 0 10 3
(27%)
Lower Group 4 8 0 4 4
(27%)

Compute the discrimination index

Cug = 10
Clg = 4
D = 20
Di = Cug – Clg / D
Di = 10 – 4 / 20
= 6/ 20
= 0.30 or 30%

Therefore, more students from the upper group got the answer correctly, hence, it has a positive
discrimination.

Example 2 ( I will let my classmate solve it)

Analyze the data below

Options A B C D *E
Upper 2 3 2 2 5
group
(27%)
Lower 2 2 1 1 8
group
(27%)

Compute the discrimination index

Cug = 5

Clg = 8

D= 14

Di = Cug – Clg / D

Di = 5 – 8 / 14

= -3 / 14

= - 0.21 or – 21 %

Therefore, more students in the lower group got the item correctly. Therefore it is a negative
discrimination.

Steps in Solving Difficulty index and Discrimination Index

1. Arrange the scores from highest to lowest.

2. Separate the scores into upper group and lower group. There are different methods to do
this (a) if a class consists of 30 students who take an exam, arrange their scores from
highest to lowest, then divide them into two groups. The highest scores belongs to the
upper group. The lowest score belongs to the lower group. (b) Other literatures suggested
to use 27% , 30%, or 33% of the students for the upper group and lower group. However,
in the Licensure Examination for Teachers (LET ) the test developers always used 27% of
the students who participated in the examination for the upper and lower group.
3. Count the number of those who choose the alternatives in the upper group and lower
group for each item and record the information using the template:
Options A B C D E
Upper group
Lower group
Note: Put asterisk to the correct answer
4. Compute the value of the difficulty index and the discrimination index and also the
analysis of each response in distracters.
5. Make an analysis for each item.
Checklist for Discrimination Index

It is very important to determine whether the test item will be retained, revised or rejected. Using the
discrimination index we can identify the non- performing question items, just always remember that they
seldom indicate what is the problem. Use the given checklist below.

Yes No
1.Does the key
discriminate positively?
2 .Does the incorrect
options discriminate
negatively?

If the answers to questions 1 and 2 are both YES, retain the item.

If the answers to question 1 and 2 are either YES or NO, revised the item.

If the answers to question 1 and 2 are both NO, eliminate or reject the item.
Distracter Analysis

1. Distracter

Distracter is the term used for the incorrect options in the multiple-choice type of test while the
correct answer represents the key. It is very important for the test writer to know if the distracters are
effective or good distracters. Using quantitative item analysis we can determine if the options are good
or if the distracters are effective.

There are factors to be considered why students failed to get the correct answer in the given
question.

a. It is not taught in the class properly.

b. It is ambiguous.
c. The correct answer in not in the given options.
d. It has more than once correct answer.
e. It contains grammatical clues to mislead the students.
f. The student is not aware of the content.
g. The students were confused by the logic of the question because it has double negatives.
h. The student failed to study the lesson.

2. Miskeyed item

The test item is a potential miskey if there are more students from the upper group who choose
the incorrect options than the key.

3. Guessing item

Students from the upper group have equal spread of choices among the given alternatives. Students
from the upper group guess their answers because of the following reasons:

a. The content of the test is not discussed in the class or in the text.
b. The test item is very difficult.
c. The question is trivial.

4. Ambiguous item

This happens when more students from the upper group choose equally an incorrect option and
the keyed answer.

Quantitative Item Analysis

Are measurement used for evaluation the effectiveness of test structures, performance
assessments, and essay questions, and they judge the legitimacy of academic structures and
comprehension expectations requirements to the methods in which they are being tested. Through these
types of analysis, it is determined whether or not method of tests and their results are meeting the
requirements, and assures the students are staying on the same page for the academic spread sheets
as a whole. With either of these methods the qualities of test and measurements are evaluated, in
regards to how they engage the materials, and the scores that students achieve, to help to define
changes that are necessary within the methods.

IMPROVING TEST ITEMS

Item analysis enables the teachers to improve and enhance their skills in writing test items. To
improve multiple choice items we shall consider the stem of item, the distracters and key answer.

How to improve the test item

 Consider the following examples in analyzing the test item and some notes on how to improve the
item based from the results of items analysis.

Example;1

A class of is composed of 40 students. Divide the group into two. Option B is the correct answer.
Based from the given data on the table , as a teacher, what would do with the test item?

Options A B* c D E
Upper group 3 10 4 0 3
Lower group 4 4 8 0 4

1. Compute the difficulty index.

A= 10+4=14
N= 40

Df= n/N
Df= 14/40
Df= 0.35 or 35%

2. Compute the Discrimination index

CuG= 10 D1= CuG - ClG

CuG= 4 D
D= 20
𝟏𝟎−𝟒
= 𝟐𝟎

𝟔
= 𝟐𝟎
= 0.30 or 30%
3. Make an analysis about the level of difficulty, discrimination and distracters.

a. Only 35% Of the examinees got the answer correct hence, the item is difficult.
b. More students from the other group got the answer correctly, hence it has a positive discrimination
c. Retain option A,C and E because most of the students who did not perform well in the overall
examination . Those options attract most students from the lower group.

4. CONCLUSION

Retain the test item, but change option D, make it more realistic to make it effective for the
upper group and lower groups. At least 5 % of the examinees choose the incorrect option.

Example 2: A class is composed of 50 students. Used 27% to get the upper and lower group.
Analyze the item given the following result. Option D is the correct answer. What will you do to
the test item?

Options A B c D* E
Upper group 3 1 2 6 2
Lower group 5 0 4 4 1

Difficulty index=0.36 OR 36%

Discrimination index= 0.14 or 14%

Analysis

a. Only 36% of the examinees got the answer correctly, hence the item is difficult
 b. More students from the upper group got the correct answer, hence it has positive
discrimination.
c. Modify option B and E because more students from the upper group chose them
compare with lower group, hence they are not effective distracters because most of the
students who performed well in the overall examination selected them as their answer.
d. Retain options A and C because most of the students who did not perform well in the
overall examination selected them as the correct answer. Hence, options A and C are
effective distracters.

CONCLUSION

Revised the item by modifying options B and E .

Example 3: Potential Miskeyed items

Make an item analysis about the table below. What will you do with the test that is potential
miskeyed item?

Options A* B c D E
Upper group 1 2 3 30 4
Lower group 3 4 4 4 5

Difficulty index=0.10 or 10%

Discrimination index = - 0.10 or -10%

Analysis
a. More students from the upper group choose option D than option A even though
Option A is supposedly the correct answer.
b. Most likely the teacher has written the wrong answer key.
c. The teacher checks and find out that he/she did not miskey the answer that he/she
thought is the correct answer.
d. If the teacher miskeyed it, he/she must check and retally the scores of the students test
papers before giving them back.
e. If option A is really the correct answer. revise to weaken option D ,distracters are not
supposed to draw more attention than the keyed answer.
f. Only 10% of the students got the answer to the test item correctly hence, the test is
very difficult
g. More students in the lower group got the item correctly, therefore a negative
discrimination resulted. The discrimination index is .10
h. No need to analyze the distracters because the test item is very difficult and
discriminates negatively.

Conclusion

Reject the item because it is very difficult and has negative discrimination

Example 4: AMBIGUOUS ITEM

Below is the result of item analysis of a test with ambiguous item. What can you say about the
item? Are you going to retain, revise or reject it?

Options A B C D E*
Upper group 7 1 1 2 8
Lower group 6 2 3 3 6
Difficulty index= .36 or 36%

Discrimination index= .10 or 10 %

Analysis

a. Only 36% of the students got the answer to the test item correctly. Hence, the test item is
difficult.
b. More students from the upper group got the item correctly, hence it discriminates positively.
The discrimination index is 10%
c. About equal number of top students went for option A and Option E, this implies that they
could not tell which is the correct answer. The students do not know the content of the test,
thus, a reteach is needed.

Conclusion

Revise the test item because it is ambiguous.

Example 5: Guessing item

Below is the result of an item analysis for a test item with students answer mostly based on a
guess. Are you going to reject, revise or retain the test item?

Options A B C* D E
Upper group 4 3 4 3 6
Lower group 3 4 3 4 5

Difficulty index=.18 or 18%

Discrimination index=.05 0r 5%

Analysis

a. Only 18% of the students get the answer to the test item correctly hence, the item is very
difficult
b. More students from the upper group get the correct answer to the test item. Therefore, the test
is has positive discrimination. the discrimination index is 5%.
c. Student’s responds about equally to all alternatives an indication that they are guessing.

Possibilities why students guesses the answer on the test item

● The content of the test item has not been yet discussed in the class because the test is
designed in advance.
● Test item were badly written that students have no idea what the question really all about;
and
● test item were very difficult as shown from the difficulty index and low discrimination index.
● if the test item is well-written but too difficult, reteach the material to the class.

Conclusion
 Reject the test item because it is very difficult and the discrimination index is very poor
and options A and B are not effective distracters.

Example 6: Ineffective distracters

Options A B C* D E
Upper group 5 3 9 0 3
Lower group 6 4 6 8 4

Difficulty index=.38 or 38%

Discrimination index= .15 or 15%

Analysis

a. Only 38% 0f the students got the answer correctly to the test item hence, the test item is
difficult
b. More students from the upper group got the correct answer ; as a result, the test got a
positive discrimination 15%
c. Options A, B and E are attractive and effective distracters.
d. Option D is ineffective; therefore, change it with more realistic one.

Conclusion

Revise the item by changing option D.