I.

3 Item analysis


Activity 3.21: Brain storming Questions
When you observe students achievement on a test you see high , medium and low
achievers ; does the teacher prepare such tests intentionally to identify students
ability level? Should tests include items that have equal difficulty?

Item analysis is the process of studying the characteristics of a test items based on data obtained
from examinees. It helps to answer questions related to difficulty of each item discrimination
power of the item between higher and lower achievers and effectiveness of each distracter to
attract students.

More specifically, the reasons for analyzing test items are the following:

1. Identify content that has not been adequately covered and should be re-taught
2. Provide feedback to students
3. Determine if any items need to be revised in the event they are to be used again or
become part of an item file or bank
4. Identify items that may not have functioned as they were intended
Direct the teacher's attention to individual student weaknesses. The three most common ways
of reporting item analysis are the item difficulty , the item discrimination and the distracter
analysis. Once the item analysis information is available, an item review is often conducted.
In the following sections you are going to learn the frequencies used to analyze responses to
test items.

3.5.1 Item difficulty level and item discrimination power

Item difficulty level index

Item difficulty index is the proportion of examinees who answered the item correctly. If scores
from all students in a group are included the difficulty index is simply the total percent correct.
When there is a sufficient number of scores available (i.e., 100 or more) difficulty indexes are
calculated using scores from the top and bottom 27 percent of the group.

1
procedures in Item analysis

The following steps are used to analyze item difficulty level and discrimination power of an
item.

1. Rank the test papers in order from the highest to the lowest score
2. Select 27 percent of the test papers from the highest total score and another 27 percent of
the of the test papers from lowest total scores( take half of total student as HSG and other
half as LSG if the number of students below 100)
3. For each test item, tabulate the number of students in the upper & lower groups who selected
each option
4. Compute the difficulty of each item (% of students who got the right item) and the item
discrimination power (D)

Item difficulty index can be calculated using the following formula:

p❑ x
¿ ()

Where, HSG = High Scoring Groups

LSG = Low Scoring Groups
N= the total number of HSG and LSG

The difficulty indexes can range between 0 to 100 % or 0.00 to 1.00 and are usually expressed
as a percentage. A higher value indicates that a greater proportion of examinees responded to the
item correctly, and it was thus an easier item. The average difficulty of a test is the average of the
individual item difficulties. An average difficultylevel of .60 or 60% is ideal. For example: If
250 students answered item no. 1 correctly and 12 students answered incorrectly, the difficulty
level of the item would be 250/262 or .95.or 95 %So this item is easy item.

In the example below, five true-false questions were part of a larger test administered to a class
of 20 students. For each question, the number of students answering correctly was determined,
and then converted to the percentage of students answering correctly.

Question Correct responses Item difficulty Interpretation and decision

1 20 100% or (20/20)x100 Very easy then reject

2
 2 16 80% or (16/20)x100 Very easy then reject
3 7 35% or (7/20)x100 Moderate and accepted
4 12 60% or 12/20)x100 Moderate and accepted
5 18 90% or (18/20)x100 Very easy then reject


Activity 3.22: Calculate the item difficulty level for the following four-option multiple
choice item.

(The sign (*) shows the correct answer).

Response Options
Groups
A B C D* Total
High Scorers 0 1 1 8 10
Low Scorers 1 1 5 3 10
Total 1 2 6 11 20

Item difficulty interpretation

P-Value Percent Range Interpretation

Less or = 25% 0-25% Difficult
between 25% & 75% 26%-75% Average
> or = 75% 75%-100% Easy

Item discrimination index (D)

Item discrimination index is a numerical indicator that enables us to determine whether the
question discriminates appropriately between lower scoring and higher scoring students. When
students who earn high scores are compared with those who earn low scores, we would expect to
find more students in the high scoring group answering a question correctly than students from
the low scoring group. In the case of very difficult items as well as very easy items or the

3
numbers of correct answers might be equal for the two groups the discrimination index of the
item will be a minimum.

Item discrimination index can be calculated using the following formula:

HSG−LSG
D= 1
( HSG+ LSG)
2

Where, HSG = High Scoring Groups

LSG = Low Scoring Groups

For example assume that you have a total of 16 students. For question 1, all 8 in the high
scoring group answered correctly, while only 4 in the low scoring group did so. Thus success in
the HSG – Success in the LSG (8 - 4) = +4. The last step is to divide the +4 by half of the total
4
number of both groups (16). Thus, will give us +.5, which is the D-value.
1/2(16)

Question HSG LSG Difference D value Interpretation and Decision

1 8 4 8–4=4 .5 Very good and accepted
2 6 1 6-1=5 0.62 Very good and accepted
3 4 8 4-8=-4 -0.5 Poor item and rejected

The item discrimination index can vary from -1.00 to +1.00. A negative discrimination index
(between -1.00 and zero) results when more students in the low group answered correctly than
students in the high group. A discrimination index of zero means equal numbers of high and low
students answered correctly, so the item did not discriminate between groups. A positive index
occurs when more students in the high group answer correctly than the low group. If the students
in the class are fairly homogeneous in ability and achievement, their test performance is also
likely to be similar, resulting in little discrimination between high and low groups.

Questions that have an item difficulty index of 1.00 orbelow 0.00 need not be included when
calculating item discrimination indices. An item difficulty of 1.00 indicates that everyone in the
upper group answered and the lower group lost the answer. while 0.00 means both groups
got the equal answer.

4
Item discrimination interpretation
D-Value Strength Decision
above.40 Very good Accepted
Between .3 to .39 good Accepted
Between .2 to .29 week--- Need modification
Below 0.2 very week Rejected

 For a small group of students, an index of discrimination for an item that exceeds 0.20 is
considered satisfactory.
 For larger groups, the index should be higher because more difference between groups
would be expected. The guidelines for an acceptable level of discrimination depend upon
item difficulty.
 For very easy or very difficult items, low discrimination levels would be expected; most
students, regardless of ability, would get the item correct or incorrect as the case may be.
For items with a difficulty level of about 70 percent, the discrimination should be at least
0.30.
 When an item is discriminating negatively, overall the most knowledgeable examinees
are getting the item wrong and the least knowledgeable examinees are getting the item
right.
 A negative discrimination index may indicate that the item is measuring something other
than what the rest of the test is measuring. More often, it is a sign that the item has not
been answered.

3.5.2 Distracter Analysis

One important element in the quality of a multiple choice item is the quality of the item’s
distracters. However, neither the item difficulty nor the item discrimination index considers the
performance of the incorrect response options, or distracters. A distracter analysis evaluates the
effectiveness of the distracters in each item by comparing the number of students in the upper
and lower groups who selected each incorrect alternative

Remember:-A good distracter will attract more students from the lower group than the upper
group on the other hand poor distracter does the opposite one. If a distracter is poor, more
examinee in the upper group will select the alternative other than the correct answer

5
(distractors). In fact, the presence of one or more plausible distracters in a multiple choice item
can make the item artificially far easier than it ought to be. Let us try to explain this using the
following table as an example that shows the responses of eight students to five multiple-choice
questions.

For item 1, the good distractor is “ B” for item 2 the good distractor is “A” the poor
destructor is “B” For item 3 the good distractor is “ C” and the poor distractors are
“B” and “D”

Item A B C D
1 HG 5** 1 1 1
LG 2 4 1 1
2 HG 0 2 6** 0
LG 3 1 2 1
3 HG 2** 2 2 2
LG 2 0 5 1
**Denotes Correct Answer.


Activity 3.23: Individual work
Use the information given in the table below:

Alternatives
Omitted
Group A B C D
Item 15 HG 2 6 18 3 1
LG 6 2 12 9 1
 Calculate and interpret the level of difficulty level each item
 Calculate and interpret discrimination index of each item
 Analyze the effectiveness of each distracter in each of the items and decide which
distracters served the purpose which is not.
2. Take corrected exam papers of 1 section from your college and by taking 10 multiple choice
questions:
1. Calculate the difficulty level of each item
2. Calculate the discrimination power of each item
3. Analyze the plausibility of the distracters

6
4. Present your work in the form of a report.