INSTRUMENT ANALYSIS TECHNIQUES: ITEM ANALYSIS

By Group 6
Nurani Apriliana (2304020074)
Miftahudin (2304020078)

MATHEMATICS EDUCATION
FACULTY OF MATHEMATICS AND NATURAL SCIENCE UNIVERSITAS
NEGERI SEMARANG
2023/2024
 PREFACE
We express our gratitude to Allah SWT, because thanks to His grace and guidance we were
able to prepare this paper well and finish it on time. We have entitled this paper
“INSTRUMENT ANALYSIS TECHNIQUES: ITEM ANALYSIS”.

The aim of preparing this paper is to fulfill the lecture assignments of the lecturers.
Apart from that, this paper also aims to provide additional insight for us as writers and for
readers. Especially in terms of the benefits of implementing group guidance as an effort to
increase student learning motivation.

We as authors do not forget to thank Dr. Masrukan, M.Si. and Vena Agustina, S.Pd.,
M.Pd. as a teaching lecturer. We would like to thank the other students who have supported
the preparation of this paper.

Finally, we realize that this paper is still not completely perfect. Therefore, we are open
to criticism and suggestions that can build our abilities, so that in the next assignment we can
write papers even better. Hopefully this paper is useful for us and the readers.

Semarang, 14 October 2024

Writer
 TABEL OF CONTENT

PREFACE ............................................................................................................................. 2
TABEL OF CONTENT ......................................................................................................... 3
CHAPTER 1.......................................................................................................................... 4
1.1 Background .................................................................................................................. 4
1.2 Problem Formulation .................................................................................................... 4
1.3 Purpose ........................................................................................................................ 4
CHAPTER II......................................................................................................................... 5
2.1 Concept of Item Analysis ............................................................................................... 5
2.2 The Concept of Discrimination and Level of Difficulty ................................................... 6
2.3 Calculating the Score of Discrimination Index and Level of Difficulty ............................. 7
2.4 Item Analysis Software................................................................................................ 10
CHAPTER III ..................................................................................................................... 12
REFERENCES .................................................................................................................... 13
 CHAPTER 1
INTRODUCTION

1.1 Background
In the learning process, there are several steps that must be passed. One of them is the
learning evaluation step. Evaluation or commonly called a test is the step after the lesson has
been explained. Test is an important instrument to evaluate student achievement. Tests are used
to measure the achievement of learning objectives, as well as to determine the level of students'
understanding of the material that has been taught.
The stage after the test is done is to analyze the test results. This analysis is used to
evaluate the overall performance of the students and to evaluate the effectiveness of the test
instrument used. Through this analysis, teachers can find out whether the test is right on target
in achieving the required learning outcomes. Also, analyzing test results helps identify things
that need to be improved. Things that need to be improved such as question preparation or
material delivery.
One of the methods used in analyzing test results is item analysis. The item analysis is
used to evaluate the quality of each item used in the test. The process of item analysis activities
is quite complicated and requires high concentration and accuracy when entering data. Item
analysis is important to identify questions by calculating several aspects, namely item validity,
item reliability, level of difficulty, discrimination index, and the effectiveness of examiners.
The most commonly analyzed aspects are level of difficulty and discrimination index. Analysis
of the level of difficulty of the question aims to be able to distinguish questions in the easy,
medium, and difficult categories. While the analysis of discrimination index is to examine
whether the question has the ability to distinguish students who fall into the category of high
ability and low ability.

1.2 Problem Formulation

a. What is the concept of item analysis?
b. What is the concept of discrimination and level of difficulty?
c. How to calculate the score of discrimination index and level of difficulty?
d. How to use item analysis software?

1.3 Purpose
a. To describe the concept of item analysis.
b. To describe the concept of discrimination and level of difficulty.
c. Calculating the score of discrimination index and level of difficulty.
d. To use item analysis software.
 CHAPTER II
DISCUSSION

2.1 Concept of Item Analysis

Item analysis is an assessment of test questions in order to obtain a set of questions that
have adequate quality (Sudjana, 2014). Meanwhile, according to Arikunto (2012), item
analysis is a systematic procedure, which will provide very specific information on the test
items that we compile. The purpose of item analysis, among others, is diagnostic information
to examine the lessons of the class and its learning failure, and then to provide better learning
guidelines. Moreover, the answers to individual questions and the correction of questions based
on these answers are the basis for preparing better tests for the following year. Item analysis
can also improve item quality, teacher competence and student learning outcomes.
Item analysis is categorized into two types, which are qualitative item analysis and
quantitative item analysis.
1. Qualitative
There are several techniques used to analyze qualitatively, which are moderator
techniques and panel techniques. The moderator technique is done by discussing each
item together with several experts such as teachers who teach the material, material
experts, compilers or curriculum developers. The panel technique is a technique for
reviewing question items based on the rules for writing question items. The rules of the
question include material, construction, language, correctness of the answer key or
scoring guidelines.
2. Quantitative
There are two types of quantitative item analysis, which are classical and
modern.
a. Classical
Analyzing items through a classical method is the process of examining
items based on student answers with the purpose of improving the quality of
relevant items (Nur Aziza & Dzhalila, 2018). One of the strengths of classical
item analysis is that it is cost effective, so it can be done routinely on a daily
basis. Also, this analysis can be done quickly using scientific calculator tools
and computer programmes. There are various computer programmes available
at affordable prices to conduct item analysis. Some of the programmes
commonly used for this purpose are Microsoft Excel, SPSS, and Iteman. The
aspect that needs to be considered in classical item analysis is that each item is
examined in terms of item difficulty and discrimination.
b. Modern
Modern item analysis is the analysis of question items by using Item
Response Theory (IRT). IRT models can be used to create adaptive tests,
evaluate test-taker skills and examine the complexity of test items
(discrimination, difficulty, and guessing). The advantages of using IRT in item
analysis are able to show errors in every measurement made by each item, the
 test item parameters and ability parameters have an invariance property and it
has many other advantages over classical method (Eleje et al., 2018).

2.2 The Concept of Discrimination and Level of Difficulty

1) Level of difficulty
The index of difficulty is between 0.00 - 1.00. This difficulty index shows the
level of difficulty of the question, so that a question with an index of 0.00 indicates that
the question is too difficult, on the contrary index 1.0 indicates that the question is too
easy.
The item difficulty score of multiple choice questions can be obtained using the
following formula:
𝛴𝐵
𝑇𝑆 =
𝛴𝑃
where,
𝑇𝑆 : Level of Difficulty
𝛴𝐵 : Number of students who answered the question correctly
𝛴𝑃 : Number of students taking the test

The item difficulty score of essay can be obtained using the following formula:
𝑇𝑜𝑡𝑎𝑙 𝑠𝑐𝑜𝑟𝑒
𝑚𝑒𝑎𝑛 =
𝑁𝑢𝑚𝑏𝑒𝑟 𝑜𝑓 𝑠𝑡𝑢𝑑𝑒𝑛𝑡𝑠 𝑡𝑎𝑘𝑖𝑛𝑔 𝑡ℎ𝑒 𝑡𝑒𝑠𝑡
and
𝑀𝑒𝑎𝑛
𝑇𝑆 =
𝑀𝑎𝑥 𝑠𝑐𝑜𝑟𝑒

As for the index by Sudjana, it is as follows.

Value Classification
0,00 < 𝑇𝑆 ≤ 0,30 Hard
0,30 < 𝑇𝑆 ≤ 0,70 Medium
0,70 < 𝑇𝑆 ≤ 100 Easy
Table 2.1 Classification of Level of Difficulty

2) Discrimination Index
According to Hartono (2017), discrimination index is an important indicator in
measuring the ability of the item to discriminate between students who have high and
low abilities. The index of the discrimination of each item is usually also expressed in
the form of a proportion. The higher the index of the discrimination of the question
means that the more able the question concerned distinguishes students who have
understood the material from students who have not understood the material.
To find out the discrimination index of multiple choice questions is to use the
following formula (Arikunto, 2010).
𝐵𝐴 𝐵𝐵
𝐷𝑃 = −
𝐽𝐴 𝐽𝐵
DP : Discrimination index
 𝐵𝐴 : The number of upper group test takers who answered correctly
𝐵𝐵 : The number of lower group test takers who answered correctly
𝐽𝐴 : Number of upper group test participants
𝐽𝐵 ∶ Number of lower group test participants

To find out the discrimination index of essay questions is to use the following
formula.
𝑀𝑒𝑎𝑛 𝑜𝑓 𝑢𝑝𝑝𝑒𝑟 𝑔𝑟𝑜𝑢𝑝 − 𝑀𝑒𝑎𝑛 𝑜𝑓 𝑙𝑜𝑤𝑒𝑟 𝑔𝑟𝑜𝑢𝑝
𝐷𝑃 =
𝑀𝑎𝑥 𝑆𝑐𝑜𝑟𝑒
As for the classification, it is as follows.
Value Classification
0,00 < 𝐷𝑃 ≤ 0,20 Poor
0,20 < 𝐷𝑃 ≤ 0,40 Satisfactory
0,40 < 𝐷𝑃 ≤ 0,70 Good
0,70 < 𝐷𝑃 ≤ 1,00 Excellent
Table 2.2 Classification of Discrimination Index

2.3 Calculating the Score of Discrimination Index and Level of Difficulty

The data used as an example for the calculation of the Score of Discrimination Index and Level
of Difficulty comes from the data of an article that analyzes mathematics olympiad questions
as many as 20 olympiad questions, the data used are the answer sheets of all OMI participants
at the junior high school level as many as 221 sheets in the multiple choice question section.

Table 1. Results of calculating the difficulty level of questions

 source: Dewi, et al (2019).

The item difficulty score can be obtained using the following formula:
𝛴𝐵
𝑇𝑆 =
𝛴𝑃
where,
𝑇𝑆 : Level of Difficulty
𝛴𝐵 : Number of students who answered the question correctly
𝛴𝑃 : Number of students taking the test

From the 221 answers to question number 1, there are only 71 correct answers, so the difficulty
index is 71221 = 0.32, including moderate criteria.
Based on the table above, the percentage of the results of the analysis of the level of difficulty
can be determined in the following table.

Table 2. Percentage of question difficulty level.

Categories Question number Number of questions Percentage

easy 4 1 5%
 Medium 1,3,8,9,11,13,15,19 8 40%

Difficult 2,5,6,7,10,12,14,16,17,8,20 11 55%

The table shows the results of 5% of junior high school OMI questions have an easy category,
40% of the questions have a medium category and 55% of the questions have a difficult
question category.
In addition to the level of difficulty, an analysis of the discrimination identified from the
answers of the OMI participants at the junior high school level was also conducted. There were
221 participants who participated in the OMI at the junior high school level. The differentiating
power is measured by identifying the correct answers of all junior high school OMI
participants, then how to find the discrimination index of the questions, first by forming the
upper group (KA) and the lower group (KB) by taking 27% from the top, and 27% from the
bottom. Since the number of participants is 221 participants, it is determined that 27% of 221
is 61. So the results of the highest participants number 1 to number 61 are used as the upper
group, while numbers 160 to 221 are the lower group. After that, the answers of the upper
group and lower group participants were analyzed using the D (discrimination) formula so that
the results were obtained as in the following table.
Table 3. Discrimination index calculation results on junior high school OMI questions

Source: Dewi, et al (2019).

The discrimination index abouve can calculate using formula:

𝐵𝐴 𝐵𝐵
𝐷𝑃 = −
𝐽𝐴 𝐽𝐵
DP: Discrimination index
 𝐵𝐴 : The number of upper group test takers who answered correctly
𝐵𝐵 : The number of lower group test takers who answered correctly
𝐽𝐴 : Number of upper group test participants
𝐽𝐵 ∶ Number of lower group test participants
𝐵𝐴 𝐵 38 8
For the question number one 𝐷𝑃 = − 𝐽 𝐵 = 21 − 21 = 0,491
𝐽𝐴 𝐵

Based on the table above, the percentage of discrimination index analysis results can be
determined in the following table
Table. Percentage of Discrimination index

Categories Question number Number of questions Percentage

Poor 2,5 2 10%

Satisfactory 4,9,10,12,16,17,18,20 8 40%

Good 1,6,7,8,11,13,14,15,19 9 45%

Excellent 3 1 5%

The table states that there are 5% of questions that have excellent differentiating power, 45%
of questions have good differentiating power, 40% of questions have fair differentiating
power which means they still need a little improvement and 10% of questions have poor
differentiating power which means they must be discarded or replaced by other questions.

2.4 Item Analysis Software

Item analysis by software computer means that the review of question items
quantitatively whose calculations use the help of a computer program. Data analysis using a
computer program is very appropriate. Because the level of accuracy of calculations using
computer program is higher when compared to being processed manually or using a
calculator/hand. Computer software that used to analyze data models vary depending on the
purpose and intent of the analysis required. Untuk membantu dalam proses analisis ini,
beberapa software telah dikembangkan guna mempermudah perhitungan tingkat kesukaran
dan daya pembeda. Berikut adalah beberapa software yang umum digunakan serta deskripsi
penggunaannya:

a. SPSS (Statistical Package for the Social Sciences)

SPSS is statistical software that can be used for various data analyses, including
item analysis. In calculating the difficulty level and differentiating power of questions,
SPSS makes it easy for users through the descriptive statistics and correlation
calculation features. Users can enter data on the answer scores for each question from
groups of respondents, then use correlation analysis to measure differentiating power
 by comparing the upper and lower groups. The level of difficulty can be calculated
manually using the frequency statistics output from SPSS.
b. Anates
Anates is a software specifically designed for item analysis. This software is
easy to use by educators or researchers who do not have an in-depth statistical
background. Anates provides a special feature to calculate the level of difficulty and
distinguishing power more quickly. The user simply enters the answer score of each
respondent, and the software will automatically provide the results of the level of
difficulty and distinguishing power of each item, as well as provide recommendations
for question improvement.
c. ITEMAN
ITEMAN is software that is often used in item analysis for large-scale tests.
This software can analyze items based on item response theory (IRT) and classical
models: ITEMAN allows users to analyze tests with many items at once. Users only
need to import participant response data in a specific format, and ITEMAN will provide
a complete report on the test's difficulty level, differentiating power, as well as
additional information such as test reliability.
d. Microsoft Excel
Microsoft Excel, although not a specialized software for question analysis, is
still often used because of its ease in performing manual calculations. Excel can be used
to calculate the level of difficulty and discriminating power using simple mathematical
formulas. Participant response data is entered into the spreadsheet, and the user can
create formulas to calculate the proportion of correct answers per group (top and
bottom) and calculate the discriminating power by comparing the results of the two
groups.
 CHAPTER III
CONCLUSION

Item analysis is a crucial method for evaluating the quality of test items, the purpose of
item analysis, among others, is diagnostic information to examine the lessons of the class and
its learning failure, and then to provide better learning guidelines. Item analysis can also
improve item quality, teacher competence and student learning outcomes. These measures help
distinguish well-designed questions that can accurately differentiate between high- and low-
performing students. Through this paper, we have explored the concepts of item analysis,
including the key aspects of discrimination and difficulty, and described how to calculate them
using specific formulas. Furthermore, we have provided insights into various software tools,
such as SPSS, Anates, ITEMAN, and Microsoft Excel, which greatly facilitate the analysis
process. Each of these tools offers different levels of functionality and ease of use, allowing
users to choose the most appropriate one based on their needs and level of expertise.
 REFERENCES

Arikunto, S. (2012). Prosedur Penelitian. Jakarta: Rineka Cipta

Aziza, R. N., & Dzhalila, D. (2018). Metode Kuantitatif Dengan Pendekatan Klasik Pada
Aplikasi Analisis Butir Soal Sebagai Media Evaluasi Penentuan Soal Yang Berkualitas.
Jurnal KILAT (Kajian Ilmu Dan Teknologi), 7(1), 15–23.
https://doi.org/10.33322/kilat.v7i1.104
Dewi, S. S., Hariastuti, R. M., & Utami, A. U. (2019). Analisis tingkat kesukaran dan daya
pembeda soal olimpiade matematika (OMI) tingkat SMP tahun 2018. Transformasi:
Jurnal Pendidikan Matematika Dan Matematika, 3(1), 15-26.
Eleje, L. I., Onah, F. E., & Abanobi, C. C. (2018). Comparative study of Classical Test Theory
and Item Response Theory using diagnostic quantitative economics skill test item
analysis results. European Journal of Educational & Social Sciences, 3(1).
Fatimah, L. U., & Alfath, K. (2019). Analisis kesukaran soal, daya pembeda dan fungsi
distraktor. AL-MANAR: Jurnal Komunikasi dan Pendidikan Islam, 8(2), 37-64.
Hartono. (2017). Analisis Daya Pembeda Butir Soal pada Ujian Sekolah Berstandar Nasional.
Jurnal Kependidikan, 46(1), 23–32.
Nana Sudjana. (2014). Penelitian Hasil Proses Belajar Mengajar. Bandung: PT Remaja
Rosdakarya.
Pradita, E., Megawanti, P., & Yulianingsih, Y. (2023). Analisis Tingkat Kesukaran, Daya
Pembeda, dan Fungsi Distraktor PTS Matematika SMPN Jakarta. Himpunan: Jurnal
Ilmiah Mahasiswa Pendidikan Matematika, 3(1), 109-118.
Purwanto Ngalim. (1984). Prinsip- prinsip dan Teknik Evaluasi Pengajaran.Penerbit
Bandung:Remaja Karya