Module 4 PDF

Download as pdf or txt
Download as pdf or txt
You are on page 1of 13

Module 4 Administering, Analyzing, and Improving Tests

Lesson Outline:
Define the basic concepts regarding item analysis;
Identify the steps in improving test items;
Solve difficulty index and discrimination index;
Identify the level of difficulty of an item;
Perform item analysis properly and correctly;
Identify the item to be rejected, revised, or retained; and
Interpret the results of item analysis.

I. Introduction
One of the most important functions of a teacher is to assess the performance of the students.
It is a very complicated task because one will have to consider many activities: Timing of the
assessment process
Format of the assessment tools
Duration of the assessment procedures

Packaging and Reproducing Test Items


1. Put the items with the same format together.
2. Arrange the test items from easy to difficult.
3. Give proper spacing for each item for easy reading.
4. Keep questions and options in one page.
5. Place the illustrations near the options.
6. Check the answer key.
7. Check the direction of the test.
8. Provide space for name, date and score.
9. Proofread the test.
10. Reproduce the test.

Administering the Examination


Comes after constructing the test items and putting them in order. The administering
procedures greatly affect the performance of the students in the test.
The test administration does not simply mean giving the test questions to the students
and collecting the test papers after the given time.

Guidelines Before Administering Examinations.


1. Try to induce positive test-taking attitude.
2. Inform the students about purpose of the test.
3. Give oral directions as early as possible before distributing the tests. 4. Give
test-taking hinds about guessing, skipping, and the like, are strictly prohibited.
5. Inform the students about the length of time allowed fir the test. If possible, write on
the board the time in which they must be finished with answering the test. Give the
students a warning before the end of the time limit.
6. Tell the students how to signal or call your attention if they have a question. 7. Tell the
students what to do with their papers when they are done answering the test (how papers
are to be collected).
8. Tell the students what to do when they are done with the test, particularly if they are to
go to another activity (also write thee directions on the chalkboard so they can refer to
them).
9.
left or the front row.
10. Make sure the room is well lighted and has a comfortable temperature. 11. Remind
students to put their names on their papers (and where to do so). 12. If the test has more
than one page, have each student checked to see that all pages are there.

Guidelines During the Examination


1. Do not give instructions or avoid talking while examination is going on to minimize
interruptions and distractions.
2. Avoid giving hints.
3.
4. Give time warnings if students are not pacing their work appropriately. 5. Make a note
of any questions students ask during the test so that items can be revised for future use.
6. Test papers must be collected uniformly to save time and to avoid test papers to be
misplaced.

Guidelines After the Examination


Grade the papers (and add comments if you can); do test analysis (see module on test
analysis); after scoring and before returning papers to students if at all possible. If it is possible
to do your test analysis before returning the papers, be sure to do it at another time.
It is important to do both the evaluation of your students and the improvement of your tests.
If you are recording grades or scores, record them in pencil in your class record before returning
the papers.
If there are errors/adjustments in grading, they (grades) are easier to change when recorded
in pencil.
Return papers in timely manner.
Discuss test items with the students.
If students have questions, agree to look over their papers again.
It is usually better not to agree to make changes in grades on the spur of the moment
while discussing the tests with the students but to give yourself time to consider what
action you want to take.
The test analysis may have already alerted you to a problem with a particular question
that is common to several students, and you may already have made a decision
regarding the question
Analyzing the Test
The teacher should always analyze the quality of each item in the test after
administering and scoring it.
With this, you can identify the item that is good, item that needs improvement or items to
be removed from the test.
According to Lewis Aiken (1997), an author of psychological and educational

assessment as it is in medicine.
Item Analysis
o The technique to help teachers determine the quality of a test item o One of the
purpose is to improve the quality of the assessment tools. o Through this, one can
identify the item that is to be retained, revised or rejected, and the content of the
lesson that is mastered or not.
o Two kinds:
Quantitative Item Analysis
Qualitative Item Analysis
Item Analysis

It consists of different procedures for assessing the quality of the test items given to the
students.
Through the use of this, one can identify which of the given are good and defective test
items.
o Good Test Items: Retained
o Defective Test Items: To be improved; to be revised; or to be rejected Uses of

Item Analysis

1. Item analysis data provide a basis for efficient class discussion of the test results.
2. Item analysis data provide basis for remedial work.
3. Item analysis data provide a basis for general improvement of classroom instruction.
4. Item analysis data provide a basis for increased skills in test construction. 5. Item
analysis procedures provide a basis for constructing test bank.

Types of Quantitative Item Analysis

Difficulty Index
Refers to the proportion of the number of students in the upper and lower groups who
answered an item correctly.
The larger the portion, the more students, who have learned the subject measured by
the item.

Formula:

D_f=n/N Df = difficulty index


n = No. of the students selecting
item correctly in the upper and lower group
N = total no. of students who
answered the test

Level of Difficulty of an Item


Index Range Difficulty Level
0.00-0.20 Very Difficult
0.21-0.40 Difficult
0.41-0.60 Average/Moderately Difficult
0.61-0.80 Easy
0.81-1.00 Very Easy

NOTE: The higher the value of the index of difficulty, the easier the item is. Hence, more
students got the correct answer and the more students mastered the content measured
by the item.

Discrimination Index
The power of the item to discriminate the students between those who scored high and
those who scored low in the overall test.
It is the power of the item to discriminate the students who know the lesson and those
who do not know the lesson.
Refers to the number of students in the upper group who got an item correctly minus the
number of students in the lower group who got an item correctly. Divide the difference by
either the number of the students in the upper group or in the lower group or get the higher
number if they are not equal.
It is the basis of measuring the validity of an item.
It can be interpreted as an indication of the extent to which overall knowledge of the content area or
mastery of the skills is related to the response on an item.

Types of Discrimination Index


Positive Discrimination
Happens when more students in the upper group got the item correctly than those students in the
lower group
Negative Discrimination
Occurs when more students in the lower group got the item correctly than the students in the
upper group.
Zero Discrimination
Happens when the number of students in the upper and lower group who answer the test are
equal.
Hence, the test item cannot distinguish the students who performed in the overall test and the
students whose performance are very poor.

Level of Discrimination (Ebel & Frisbie, 1986; Hetzel, 1997)


Index Range Discrimination Level
0.19 and below Poor item, should be eliminated or need to be revised 0.20-0.29
Marginal item, needs some revision
0.30-0.39 Reasonably good item but possibly for improvement 0.40 and
above Very good item

Discrimination Index Formula


D_I=(C_UG-C_LG)/D
DI = discrimination index value
CUG = no. of students selecting the
correct answer in the upper group
CLG = no. of students selecting the
correct answer in the lower group
D = no. of students in either the
lower group or upper group

NOTE:
Consider the higher no. in case the sizes in upper and lower group are not equal. Steps in
Solving Difficulty Index and Discrimination Index
Arrange the scores from highest to lowest.
Separate the scores into upper group and lower group. There are different methods to do this:
If a class consists of 30 students who takes an exam, arrange their scores from highest to lowest,
then divide them into two groups. The highest score belongs to the upper group. The lowest score
belongs to the lower group.
Other literatures suggested to use 27%, 30%, or 33% of the students for the upper group and
lower group. However, in the licensure Examinations for Teachers (LET), the test developers
always used 27% of the students who participated in the examination for the upper and lower
groups.
Count the number of those who chose the alternatives in the upper and lower group for each item
and record the information using the template:

Options A B C D E
Upper Group
Lower Group

Note: Put asterisk for the correct answer


Compute the value of the difficulty index and the discrimination index and also the analysis of each
response in the distracters.
Make an analysis for each item.

Checklist for Discrimination Index


Yes No
Does the key discriminate positively?
Does the incorrect options discriminate negatively?

NOTE:
If the answers to question 1 and 2 are both YES, retain the item. If the answers to question
1 and 2 are either YES or NO, revise the item. If the answers to question 1 and 2 are both
NO, eliminate or reject the item.

Analysis for Response Questions


Another way to evaluate the performance of the entire test. It is very important to examine the
performance of each option in a multiple-choice item.
It can be used to determine whether the distracters or incorrect options are effective or attractive to
those who do not know the correct answer. The attractiveness of the incorrect options is determined
when more students in the lower group than in the upper group choose it.
Analyzing the incorrect options allows the teachers to improve the test items so that it can be used
again in the future.
Distracter Analysis
Distracter
The term used for the incorrect options in the multiple-choice type of test while the correct answer
represents the key.
It is very important for the test writer to know if the distracters are effective or good distracters.
Item analysis can identify non-performing test items, but this test seldom indicates the error or the
problem in the given item.

Factors to Consider Why Students Fail to Get the Correct Answer


It is not taught in the class properly.
It is ambiguous.
The correct answer is not in the given options.
It has more than one correct answer.
It contains grammatical clues to mislead the students.
The student is not aware of the content.
The students were confused by the logic of the question because it has double negatives.
The student failed to study the lesson.

Miskeyed Item
The test item is a potential miskey if there are more students from the upper group who choose the
incorrect options than the key.

Guessing Item
Students from the upper group have equal spread of choices among the given alternatives.
Students from the upper group guess their answers because of: The content of
the test is not discussed in the class or in the text. The test item is very difficult.
The question is trivial.

Ambiguous Item
This happens when more students from the upper group choose equally an incorrect option
and the keyed answer.
Qualitative Item Analysis
A process in which the teacher or expert carefully proofreads the test before it is administered
to:
Check if there are typographical errors
Avoid grammatical clues that may lead to giving away the correct answer Ensure that the
level of reading materials are appropriate
These procedures can also include small group discussions on the quality of the
examination and its items, with examinees that have already took the test. According to Cohen,
Swerdlik and Smith (1992), students who took the examinations are asked to express verbally their
experience in answering each item in the examination.
This can help the teacher in determining whether the test takers misunderstood a certain item.
It can also help in determining why they misunderstood a certain item.

Improving Test Items


To improve multiple-choice test item, one shall consider the stem of the item, the distracters and
the key answer.

How to Improve the Test Item


Example 1: A class is composed of 40 students. Divide the group into two. Option B is the correct
answer. Based from the given data on the table, as a teacher, what would you do with the test
item?
Options A B* C D E
Upper Group 3 10 4 0 3
Lower Group 4 4 8 0 4

Compute for the difficulty index.


n = 10 + 4 = 14 N = 40
D_f=n/N
D_f=14/40
D_f=0.35 or 35%

Compute for the discrimination index.


CUG = 10 CLG = 4 D = 20
D_I=(C_UG-C_LG)/D
=(10-4)/20
=6/20
=0.30 or 30%

Make an analysis about the level of difficulty, discrimination and distracters. Only 35% of the
examinees got the answer correctly, hence, the item is difficult. More students from the upper
group got the answer correctly. Hence it has a positive discrimination.
Retain options A, C, and E because most of the students who did not perform well in the overall
examination selected it. Those options attract most students from the lower group.
Conclusion: Retain the test item but change option D, make it more realistic to make it
effective for the upper and lower groups. At least 5% of the examinees choose the incorrect option.

Example 2: A class is composed of 50 students. Use 27% to get the upper and the lower groups.
Analyze the item given the following results. Option D is the correct answer. What will you do with
the test item?

Options A B C D* E
Upper Group 3 1 2 6 2
Lower Group 5 0 4 4 1

Compute for the difficulty index.


n = 6 + 4 = 10 N = 28
D_f=n/N
D_f=10/28
D_f=0.36 or 36%

Compute for the discrimination index.


CUG = 6 CLG = 4 D = 14
D_I=(C_UG-C_LG)/D
=(6-4)/14
=2/14
=0.14 or 14%

Make an analysis about the level of difficulty, discrimination and distracters. Only 36% of the
examinees got the answer correctly, hence, the item is difficult. More students from the upper
group got the answer correctly. Hence it has a positive discrimination.
Modify options B and E because more students from the upper group chose them compare with the
lower group, hence, they are not effective distracters because most of the students who performed
well in the overall examination selected them as their answers. Retain options A and C because
most of the students who did not perform well in the overall examination selected it. Hence, options A
and C are effective distracters. Conclusion: Revised the item by modifying options B and E.

Example 3: A class is composed of 50 students. Use 27% to get the upper and lower groups.
Analyze the item given the following results. Option E is the correct answer. What will you do with
the test item?

Options A B C D E*
Upper Group 2 3 2 2 5
Lower Group 2 2 1 1 8
Compute for the difficulty index.
n = 5 + 8 = 13 N = 28
D_f=n/N
D_f=13/28
D_f=0.46 or 46%

Compute for the discrimination index.


CUG = 5 CLG = 8 D = 14
D_I=(C_UG-C_LG)/D
=(5-8)/14
=(-3)/14
=-0.21 or-21%

Make an analysis about the level of difficulty, discrimination and distracters. a. 46% of
the students got the answer to the test item correctly, hence, the test item is moderately
difficult.
More students from the lower group got the answer correctly, therefore, it is a negative
discrimination. The discrimination index is -21%.
No need to analyze the distracters because the item discriminates negatively. Modify
all the distracters because they are not effective. Most of the students in the upper
group chose the incorrect options. The options are effective if most of the students in
the lower group chose the incorrect options.
Conclusion: Reject the item because it has a negative discrimination index.

Example 4: Potential Miskeyed Item. Make an item analysis about the table below. What
will you do with the test that is a potential miskeyed item?

Options A* B C D E
Upper Group 1 2 3 10 4
Lower Group 3 4 4 4 5

Compute for the difficulty index.


n = 1 + 3 = 4 N = 40
D_f=n/N
D_f=4/40
D_f=0.10 or 10%

Compute for the discrimination index.


CUG = 1 CLG = 3 D = 20
D_I=(C_UG-C_LG)/D
=(1-3)/20
=(-2)/20
=-0.10 or-10%

Make an analysis about the level of difficulty, discrimination and distracters. More students
from the upper group chose option D tan option A, even though option A is supposedly the
correct answer.
Most likely the teacher has written the wrong answer key. The teacher checks and finds out that
he/she did not miskey the answer that he/she thought is the correct answer.
If the te
test papers before giving them back.
If option A is really the correct answer, revise to weaken option D, distracters are not supposed to
draw more attention than the keyed answer.
Only 10% of the students got the answer to the test item correctly, hence the test item is very
difficult.
More students from the lower group got the item correctly, therefore, a negative discrimination
resulted. The discrimination index is -10%.
No need to analyze the distracters because the test is very difficult and discriminates
negatively.
Conclusion: Reject the item because it has a negative discrimination index.
Example 5: Ambiguous item. Below is the result of item analysis of a test with an ambiguous test
item. What can you say about the item? Are you going to retain, revise or reject it?

Options A B C D E*
Upper Group 7 1 1 2 8
Lower Group 6 2 3 3 6

Compute for the difficulty index.


n = 8 + 6 = 14 N = 39
D_f=n/N
D_f=14/39
D_f=0.36 or 36%

Compute for the discrimination index.

CUG = 8 CLG = 6 D = 20
D_I=(C_UG-C_LG)/D
=(8-6)/20
=2/20
=0.10 or 10%

Make an analysis.
Only 36% of the students got the answer to the test item correctly, hence, the test item is difficult.
More students from the upper group got the item correctly, hence, it discriminates positively. The
discrimination index is 10%.
About equal numbers of top students went for the option A and option E, this implies that they
could not tell which is the correct answer. The students do not know the content of the test, this, a
reteach is needed.
Conclusion: Revise the test item because it is ambiguous.

Example 6: Guessing item. Below is the result of an item analysis for a test item with swers mostly
based on a guess. Are you going to reject, revise or retain the test item?

Options A B C* D E
Upper Group 4 3 4 3 6
Lower Group 3 4 3 4 5

Compute for the difficulty index.


n = 4 + 3 = 7 N = 39
D_f=n/N =7/39
D_f=0.18 or 18%

Compute for the discrimination index.

CUG = 4 CLG = 3 D = 20
D_I=(C_UG-C_LG)/D
=(4-3)/20
=1/20
=0.05 or 5%

Make an analysis.
Only 18% of the students got the answer to the test item correctly, hence, the test item is very
difficult.
More students from the upper group got the correct answer to the test item; therefore, the test item
is a positive discrimination. The discrimination index is 5%. Students respond equally to all
alternatives, an indication that they are guessing. Three possibilities why student guesses the
answer on the test item: The content of the test item has not yet been discussed in the class
because the test is designed in advance;
Test items were badly written that students have no idea what the question is really all about;
and
Test items were very difficult as shown from the difficulty index and low discrimination
index.
If the test item is well-written but too difficult, reteach the material to the class. Conclusion: Reject
the item because it is very difficult and the discrimination index is very poor, and options A and B
are not effective distracters.

Example 7: The table below shows an item analysis of a test item with ineffective
distracters. What can you conclude about the test item?

Options A B C* D E
Upper Group 5 3 9 0 3
Lower Group 6 4 6 0 4

Compute for the difficulty index.


n = 9 + 6 = 15 N = 40
D_f=n/N
D_f=15/40
D_f=0.38 or 38%

Compute for the discrimination index.


CUG = 9 CLG = 6 D = 20
D_I=(C_UG-C_LG)/D
=(9-6)/20
=3/20
=0.15 or 15%

Make an analysis.

Only 38% of the students got the answer to the test item correctly, hence, the test item is difficult.
More students from the upper group answered the test item correctly; as a result, the test got a
positive discrimination. The discrimination is 15%.
Options A, B, and E are attractive and effective distracters.

Option D is ineffective; therefore change it with more realistic one. Conclusion:


Revise the item by changing option D.
Chapter Exercises:
Discuss the different guidelines in packaging and administering the test. What are the
steps in item analysis?
Why is test analysis necessary in the assessment process?

You might also like