2023 International Conference Advancement in Data Science, E-learning and Information Systems (ICADEIS)

ISBN: 979-8-3503-0341-4

Adaptive E-Assessment Framework

Oktariani Nurul Pratiwi Riska Yanu Fa’rifah Iqbal Rahmadhian Pamungkas
Fakultas Rekayasa Industri Fakultas Rekayasa Industri Fakultas Teknik
Universitas Telkom Universitas Telkom Universitas Pancasila
Bandung, Indonesia Bandung, Indonesia Jakarta Selatan, Indonesia
[email protected] [email protected] [email protected]

Rani Megasari Annisa Syafarani Callista

Program Studi Ilmu Komputer Fakultas Rekayasa Industri
Fakultas Pendidikan Matematika dan Ilmu Pengetahuan Alam Universitas Telkom
Bandung, Indonesia Bandung, Indonesia
[email protected] [email protected]

Abstract— The important part of the learning process is Technology has combined the processes of learning,
practice and evaluation. Evaluation is intended to determine teaching and assessment. Both systems are in the form of
students' abilities so that the learning process can be improved learning management systems, e-learning, e-assessments, and
according to student needs. Unfortunately, exercises and others. Quoted from research.com the number of LMS users
evaluations that are appropriate to the student's personality are is 73.8 million [5]. This proves that the existence of
difficult to do traditionally by teachers. In order to make the technology in education can be accepted to help the learning
evaluation process easier and more personal, a system that is process.
able to assist the process of training and evaluating students is
needed. The adaptive training system allows the system to E-assessment is an assessment system that can be in the
provide questions that can increase students' motivation and form of online exercises or exams [14]. There are many
abilities. In addition, the form of the questions can be in advantages to implementing e-assessment, including more
accordance with the circumstances of the students. A flexibility for students, getting feedback quickly and
methodology based on qualitative research using the Grounded automatically [14]. Likewise, it is easier for teachers to
Theory method was applied in this study. The proposed provide feedback so that the learning process and student
framework consists of several layers, including the User layer, learning outcomes are better [1]. In addition to teachers and
Input Output layer, Model and AI/Machine Learning layers. students, e-assessment is considered to provide many benefits
Input Output can consist of a variety hardware. The model layer
for the school. These advantages include being able to be
can consist of student model and domain model. The
accessed 24/7 and from anywhere [16].
AI/Machine Learning layer can consist of item classification
modules, student condition monitoring, questions Currently the development of e-assessment leads to e-
recommendations, student performance evaluations, learning assessments that can be tailored to the personal needs of
path recommendations, gamification engine, prevention action, students, in other words, adaptive e-assessment. Personalized
questions generator. It is intended that the framework created e-assessment can be based on cognitive level, goal or
would satisfy the proper evaluation requirements of each motivation [16]. The purpose of an adaptive e-assessment
student in order to encourage learning and enhance student system is that the learning process can meet the personal
abilities. This framework is intended for use in secondary
learning needs of students so as to increase student
schools.
effectiveness in learning [19].
Keywords—adaptive learning; e-assessment, framework
II. RELATED WORKS
I. INTRODUCTION In the research conducted [13] by building adaptive
Assessment in education has a very important role [6]. hypermedia learning systems (AHLS) which was tested on a
According to the Webster Dictionary, Assessment means group of students has made the ability of students who learn
appraisal. With the assessment, learning and teaching to use AHLS better than those who do not use AHLS. In
activities in the classroom are measurable. Assessment can be addition, students have a more positive perception of AHLS
used to direct learning activities that need to be carried out on which adjusts to the cognitive abilities of each student. The
students [20]. The types of assessment commonly used in study presented in this paper developed an adaptive
education are formative assessment and summative hypermedia learning system adapted to students' cognitive
assessment [3]. styles, with an emphasis on the Holist-Serialist Pask
dimension.
Although assessment has an important role, however, the
ideal assessment process has many challenges and is difficult The development of another adaptive e-assessment system
to do [11]. The more students, the more burden the teacher is [4] “a general system to adapt any component of the
will have on checking student answers. In addition, the type assessment process (model, activity, question…) based on
of assessment method carried out by the teacher must be in different evidence gathered from the learning process of the
accordance with the desired learning outcomes [15]. learner.” The system proposed by this researcher has 2 main
modules, namely: evidential module and adaptive module.

978-1-6654-6387-4/22/$31.00 ©2022 IEEE

This system was tested on students and compared the results
with students who studied without an adaptive e-assessment.
The development of an adaptive framework is carried out
by [2]. The framework has 6 core components, namely
interaction interfaces (learner interface and instructor
interface), learner model, authentication engine, domain
model, adaptive engine and exam generator. The development
of this framework has not yet been implemented in a system,
but its needs have been tested using interview and survey
methods. The researcher also revealed 8 technical factors that
support a safe and reliable system, namely: Authentication,
Authorization, Accessibility, Usability, Availability,
Controllability, Privacy, Monitoring.
Another research is Firecracker [9] an online adaptive e-
learning platform, and assessment software, which promotes
critical thinking, helps prepare students for courses and high-
stakes examinations, and evaluates progress relative to co-
learners. Firecracker was perceived very positively and
welcomed by the students. Students were satisfied with the
Firecracker as a formative assessment tool, and its use
correlated with improved performance in the course
examinations.
Figure 1 The Grounded Theory
In developing an adaptive e-assessment system, the most
important thing is how the system can be adaptive according b. Data collection and sampling stage. The data collection
to the needs of students. Where the questions are personalized process that can be done is observation and interviews.
for the requirements of the students. So that students can be Observations were carried out before and during the
encouraged to answer many questions, learn more and be able research which included a general description, the
to understand more. In order for the system to be adaptive, atmosphere of social life, physical conditions,
there are many models that must be built so that the system economic and social conditions that occurred. In-depth
can truly understand the students as a whole. The system can interviews were conducted directly with informants
identify a student's cognitive, emotional, and psychological separately in their respective environments.
characteristics while they are taking an exam. In addition, the
modeling of the test material must also be well mapped. Based Data collection in this study was carried out to collect
on the cognitive level of Bloom's taxonomy or based on the a lot of literature to support the design of this
subject mapping. In this paper, the author tries to fulfill the framework. Literature in the form of research journals
shortcomings of previous studies so that the needs of an taken from various sources. The data is collected to see
adaptive system can be fulfilled according to the student's how far the development of e-assessment has been. So
condition as a human being. that researchers can build an adaptive, safe and
appropriate framework to be implemented in
III. RESEARCH METHODS secondary schools. In addition to collecting literature
data, researchers have previously made observations
The method used in this study is a qualitative research regarding the need for e-assessment in schools. It is
approach with the Grounded Theory method. Based on [22] hoped that the results of this framework can meet the
"The grounded theory approach is a qualitative research needs of the school.
method that uses a set of systematic procedures to develop an
inductive grounded theory derived about a phenomenon." The c. Data analysis stage. The data analysis stage in the
stages of this research can be seen in Figure 1. grounded theory method is carried out in the form of
coding, which is the process of parsing data,
The following is an explanation based on [22]: conceptualizing and rearranging it in a new way.
a. The problem formulation stage is the problem Based on the collection of data regarding e-assessment,
formulation stage aimed at narrowing and focusing the especially adaptive e-assessment, there are many main
scope of the research. Theoretical overview of the points that are mentioned with different terms but with
usage stage. At this stage, there is a comparison of the
similar intentions. It was adapted to complement the
theory that emerged from the research results with the parts of the adaptive e-assessment framework
existing theory in the literature. In this case, an activity proposed in this study.
is carried out to compare conflicting frameworks and
harmonized frameworks. This comparison is intended d. Conclusion and report writing stage. The conclusion-
to refine the definition of the construct and increase its making stage in qualitative research using the
internal validity as well as to increase its external grounded theory method is not based on
validity. generalizations but rather on specifications. So the
theoretical formulation which is the final result found
in qualitative research using the grounded theory
method does not justify its applicability to all
 populations but is only used for that situation or device, such as PC or mobile devices, which include monitor,
condition. cameras, fingerprint scanners, speakers, and microphones.
User interface facilitates user communication for
IV. ADAPTIVE E-ASSESSMENT FRAMEWORK authorization, questions answers forms, and report.
Based on [2] the e-assessment framework can consist of Authorization is needed to maintain security in carrying
several components, including: interaction interfaces (learner out the assessment. Logging into the system is required for
interface and instructor interface), learner model, authorization. Login techniques consist of entering a
authentication engine, domain model, adaptive engine and username and password, using a pattern, or employing
exam generator. In this paper, the researcher proposes The biometric authentication such as fingerprints, iris, retina,
Adaptive e-Assessment Framework which has 4 main voice, face, palm prints, and palm veins. [24].
components, namely: User, Input/Output, Student and The system can combine many question items into one
Domain model, AI/Machine Learning Process. question package that is suitable for students. That is the

Fig. 1 Adaptive E-Assessment Framework

This framework consists of several components, namely: function of automatic question package recommendations
[27]. This helps students get good questions that suit their
A. User Layer
personality, which improves student motivation and ability.
This layer consists of 3 kinds of user, namely: student, Student reports are given not just scores on the results of
teacher, and parents. Each user is assigned a unique role. User question packages, but also learning recommendations and
engagement in the system plays a vital role as input data that predictions of student skills [28].
may be analyzed by machine learning to assist students,
teachers, and parents in making important decisions. C. Student Model & Domain model
Students can work on question groups and view The student model is built for the introductory needs of
individualized assessment results reports with comments to students. Student models can be made based on:
help them improve their skills. Teachers can access
• Learning style. Definition of Learning Style by Keefe
recommendations for relevant question packs as well as
[10] which defines learning style as "a combination of
decision guidance in appropriate learning activities for their cognitive characteristics, affective, and physiological
students. Parents have access to view the results of their factors that serve as relatively stable indicators of how
guardian's child's assessment. a learner perceives, interacts with, and responds to the
The three users access the system through input and learning environment".
output devices. These devices are described in the
Input/Output Layer section. • Cognitive style. Tennant's (1988) define cognitive style
is individual characteristics and a consistent approach
B. Input/Output Layer to organizing and processing information [21].
This framework can be developed into a system that runs
on a certain set of hardware. It used as data input and output
 • Problem solving time. Student modeling based on e-assessment is built. In this paper, we propose a framework
problem solving time shows a linear relationship that can be used for the development of an adaptive e-
between problem solving and the logarithm of time assessment. The components that build the e-assessment are
[18]. Input/Output components which can consist of hardware and
user-friendly interfaces, models consisting of student models
The domain model is a representation of several things and domain models, and AI/Machine learning consisting of
including: condition monitoring, questions type recommendations,
• Curriculum. The curriculum is a written plan that is performance evaluations & decision making, learning path,
prepared to facilitate the teaching and learning process gamification engine, prevention action and questions
[23]. generator
• Learning materials. In an e-learning system, learning
content can be complemented by the content of ACKNOWLEDGMENT
learning materials. In an e-assessment, it can be a This research supported by Telkom University. Their
collection of questions and cases given to students. support and consideration are highly appreciated.
• Bloom taxonomy. It is a level of thinking which is
divided into 6 levels of cognitive abilities [8]. REFERENCES
D. AI / Machine Learning Process
[1] Alruwais, N., Wills, G., & Wald, M. (2018). Advantages and
The e-assessment system requires an AI/Machine learning Challenges of Using e-Assessment. International Journal of
process that can support this system to be adaptive to the user. Information and Education Technology, 8(1), 34-37.
[2] Alshammari, M. T. (2020). An Adaptive Framework for Designing
The AI/Machine learning needed, including: Secure e-Exam Systems. IJCSNS International Journal of Computer
• Item Question Classification. This system groups the Science and Network Security, 20(5), 189-196.
question items based on the domain model, namely [3] ASCD. (2012). Types of Assessment. In Differentiated Instruction:
Using Ongoing Assessment to Inform Instruction. PDO.
curriculum, standards and learning materials.
[4] Baneres, D., Baró, X., Guerrero-Roldán, A.-E., & Rodríguez, M. E.
• Condition monitoring. It is a part that serves to (2016). Adaptive e-Assessment System: A General Approach. iJET,
determine the condition of students in the form of 11, 16-23.
emotions, cognitive [29] to physical conditions. [5] Bouchrika, I. (2020, September 2). Education Sep 2, 2020 51 LMS
• Questions type recommendation. Students' questions Statistics: 2021/2022 Data, Trends & Predictions. Research.com.
play an important role in meaningful learning and Retrieved March 7, 2022, from https://research.com/education/lms-
scientific inquiry [30]. Questions item statistics
recommendation is an automation in selecting the right [6] Brown, D. H. (1990). Language assessment: Principles and classroom
practices. London: Longman.
questions for students.
[7] Chin, C., & Osborne, J. (2008). Students’ questions: a potential
• Performance evaluation. Each assessment certainly resource for teaching and learning science. Studies in Science
produces results that can be reported. Based on the Education, 44(1), 1-39. 10.1080/03057260701828101
assessment, a performance evaluation can be carried [8] Forehand, M. (2005). Bloom's Taxonomy: Original and Revised. M.
out on students. Orey (Ed.), Emerging perspectives on learning, teaching, and
• Learning path recommendation [7]. It is a technology.
recommendation for appropriate learning steps for [9] Gupta, S., Ojeh, N., Sa, B., Majumder, M. A. A., Singh, K., & Adams,
students personally to become better. These O. P. (2020). Use of an Adaptive e-Learning Platform as a Formative
Assessment Tool in the Cardiovascular System Course Component of
recommendations can be determined by students and an MBBS Programme. Advances in Medical Education and Practice,
teachers. 11, 989-996. 10.2147/AMEP.S267834
• Gamification engine. Gamification engine aims to [10] Keefe, J. (1979). Learning style: An overview. Student learning styles:
increase student motivation and engagement on Diagnosing and prescribing programs, 1-17.
learning [17]. [11] Kwok, R. C.-W., Ma, J., Vogel, D., & Zhou, D. (1999). Assessment
• Prevention and Security action. Prevention and Challenges and Solutions.
security action is one of the important points in e- [12] Liu, B., Wei, H., Niu, D., Chen, H., & He, Y. (2020). Asking Questions
assessment. So that the online exam and practice the Human Way: Scalable Question-Answer Generation from Text
Corpus. arXiv:2002.00748v2.
process requires a good security system. This is https://doi.org/10.48550/arXiv.2002.00748
encouraged because cheating is easier to do if the exam [13] Mampadi, F., Chen, S. Y., Ghinea, G., & Chen, M.-P. (2011). Design
and practice process is done online [2]. of adaptive hypermedia learning systems: A cognitive style approach.
• Questions generator. Question generator allows the Computers and Education, 56(4), 1003-1011.
system to automatically generate questions from text 10.1016/j.compedu.2010.11.018
documents [12]. [14] Massing, T., Schwinning, N., Striewe, M., Hanck, C., & Goedicke, M.
(2018). E-Assessment Using Variable-Content Exercises in
Mathematical Statistics. Journal of Statistics Education, 26(3), 174-
189. 10.1080/10691898.2018.1518121
V. CONCLUSION
[15] Moges, B. (2018). The Implementations and Challenges of Assessment
Assessment is not an easy task, so technology needs to be Practices for Students' Learning in Public Selected Universities,
present to help facilitate the assessment process in learning. A Ethiopia. Universal Journal of Educational Research, 6(12), 2789-
reliable e-assessment framework is needed (secure, easy to 2806. 10.13189/ujer.2018.061213
use, motivating, etc.) before e-assessment development is [16] Mulwa, C., Lawless, S., Sharp, M., Arnedillo-Sanchez, I., & Wade, V.
(2010). Adaptive Educational Hypermedia Systems in Technology
carried out. In addition, so that the assessment process can be Enhanced Learning: A Literature Review. Proceedings of the 2010
in accordance with the personality of each student, so as to ACM conference on Information technology education (SIGITE '10),
increase the effectiveness of students in learning, an adaptive 73-84. DOI:https://doi.org/10.1145/1867651.1867672
[17] Nah, F. F.-H., Zeng, Q., Telaprolu, V. R., Ayyappa, A. P., & [24] J. Curran and K. Curran, “Biometric Authentication Techniques in
Eschenbrenner, B. (2014). Gamification of Education: A Review of Online Learning Environments,” Res. Anthol. Dev. Eff. Online Learn.
Literature. Springer International Publishing, 401-409. Courses, pp. 867–879, 2021.
[18] Pelanek, R., & Jarusek, ·. (2015). Student Modeling Based on Problem [25] D. Kumar, R. Jaipurkar, A. Shekhar, G. Sikri, and V. Srinivas, “Item
Solving Times. International Journal of Artificial Intelligence analysis of multiple choice questions: A quality assurance test for an
Education, 25(4), 493-519. 10.1007/s40593-015-0048-x assessment tool,” Med. J. Armed Forces India, vol. 77, pp. S85–S89,
2021.
[19] Popescu, E., Trigano, P., & Badica, C. (2007). Towards a Unified
[26] Y. Li, M. Rakovic, B. Xin Poh, D. Gasevic, and G. Chen, “Automatic
Learning Style Model in Adaptive Educational Systems. IEEE
Classification of Learning Objectives Based on Bloom’s Taxonomy,”
International Conference on Advanced Learning Technologies
Proc. 15th Int. Conf. Educ. Data Min., no. July, pp. 530–537, 2022.
(ICALT 2007).
[27] K. Mao, Q. Dong, Y. Wang, and D. Hong, “An Exploratory Approach
[20] Ridgway, J., Mccusker, S., & Pead, D. (2004). Literature review of e- to Intelligent Quiz Question Recommendation,” Procedia Comput.
assessment. Futurelab, Bristol. Sci., vol. 207, pp. 4065–4074, 2022.
[21] Riding, R. J. (1997). On the Nature of Cognitive Style. Educational [28] R. R. Kabra and R. S. Bichkar, “Performance Prediction of Engineering
Psychology, 17(1 and 2), 29-49. 10.1080/0144341970170102. Students using Decision Trees,” Int. J. Comput. Appl., vol. Vol. 36 No,
[22] Budiasih, I. G. A. N. (2014, Januari). Metode Grounded Theory Dalam pp. 8–12, 2011.
Riset Kualitatif. Jurnal Ilmiah Akuntansi dan Bisnis, 9(1). [29] P. Jiang and X. Wang, “Preference Cognitive Diagnosis for Student
Performance Prediction,” IEEE Access, pp. 219775–219787, 2020.
[23] Hernawan, A. H., & Andriyani, D. (2019). Pengertian, Fungsi [30] X. Diao, Q. Zeng, L. Li, H. Duan, H. Zhao, and Z. Song, “Personalized
Kurikulum, dan Komponen Kurikulum. In Pengembangan Kurikulum Learning Path Recommendation Based on Weak Concept Mining,”
dan Pembelajaran Ekonomi dan Koperasi. Universitas Terbuka Mob. Inf. Syst., vol. 2022, 2022.
Repository.