Papers by Benjamin Kralj
Šolske dosežke šolarjev merimo z različnimi pripomočki. Z razvojem računalniške tehnologije in me... more Šolske dosežke šolarjev merimo z različnimi pripomočki. Z razvojem računalniške tehnologije in medmrežja so se pojavile tudi možnosti uporabe različnih spletnih orodij. Učitelji zato vedno pogosteje posegajo po novih orodjih in z njimi poenostavljajo proces vrednotenja znanja šolarjev. Prehod s papirnega preverjanja na elektronsko je postopen in individualen proces, odvisen od posameznika, kolektiva in za vsako izobraževalno ustanovo edinstven, ne nazadnje pa pogojen tudi z vsakokrat dosegljivimi vsebinami na spletu. V tej doktorski disertaciji predstavljamo rezultate raziskave, s katero smo ugotavljali učinke uporabe spletnega sistema za preverjanje znanja na znanje učencev. Na podlagi literature in lastnih opažanj smo razvili spletni sistem za vrednotenje znanja. Učinke delovanja sistema smo vrednotili s pomočjo dveh skupin šolarjev (N = 686) iz Republike Slovenije in njihovih učiteljev. Šolarje smo na podlagi rezultatov predpreizkusa razporedili v dve po predznanju enakovredni skupini: eksperimentalno in kontrolno. Šolarji eksperimentalne skupine so imeli možnost uporabljati sistem, šolarji kontrolne skupine pa ne. Uporaba sistema je šolarjem omogočala reševanje nalog in pregled njihovih rezultatov. Med potekom raziskave so šolarji s svojimi učitelji v desetih učnih urah obravnavali in spoznavali vsebine poglavja kisikovih organskih spojin. Po koncu obravnave so šolarji pisali preizkus. Mesec dni po preizkusu pa še pozni preizkus. Primerjava rezultatov šolarjev v preizkusu in poznem preizkusu je pri šolarjih, ki so imeli dostop do spletnega sistema, pokazala pozitivne vplive na poznavanje obravnavanih vsebin. Natančneje smo učne navade in mnenja šolarjev preverili s pomočjo anketnega vprašalnika, stališča učiteljev pa povzeli v analizi polstrukturiranih intervjujev. Rezultati kažejo na primerno zasnovo spletnega sistema ter možnost širitve njegove uporabe. »Would the same work, performance and evidence of attainment be assesed similarly by these two teachers, or indeed any other teacher?«
Constructing an online system for knowledge assessment must result in a system that is useful for... more Constructing an online system for knowledge assessment must result in a system that is useful for all types of user, their learning methods and styles. It should be easy to work with it at school or at home, and the online system should use its web capabilities to broaden the data management from more than one school and offer users their data for coeval comparison. When we describe online systems used in natural sciences learning, the system should provide users with delivery of e-learning materials, record of learning experience and profile and management of e-learning materials and e-courses (Chu, Chang, & Tsai, 2009). Acknowledging these characteristics, we can build an online system for knowledge assessment that enables students to learn while solving tasks. Systems like this were used for monitoring students' progress in subject comprehension (Ibabe & Jauregizar, 2010) and for detecting high-risk students before they put their academic career at risk (Beck & Davidson, 2001). 'Knowing the score of the test and predicted trouble areas, individual students can exercise control over their own destiny in the course,' (Kennepohl, Guay, & Thomas, 2010, p. D), and researchers (Marzano, 2010) reported that achievement monitoring has a positive effect on students. Students who use this system can assess their knowledge on their own and acknowledge their weak spots in subject comprehension. With the help of their teacher, they can recoup a loss. In the progress of this research, an online system for chemistry knowledge assessment was developed. Students were able to monitor their results and learn from their solved tasks. The main part of the system was the database of chemistry tasks (since the focus of the research was on chemistry). This database was used in the system through three main functions: (1) entering the learning material, (2) solving tasks stored within the database, and (3) achievement observation available for students and teachers. With its construction, teachers should be able to 'yield more information in less testing time,' (Grenwelge, 2009, p. 345), and the system can provide students with instant feedback since they are more likely to 'pay attention to the electronic feedback that is returned quickly,' (Denton, 2001, p. 7). Research was performed in the school year 2010/2011. We have focused on the 9th grade last year of primary school students (age 13 to 15) and have chosen a chapter from organic chemistry: the group of oxygen organic molecules. Twenty primary schools in Slovenia (4.43 % out of 451) and their chemistry teachers joined the research, and with them 686 students (3.6 % out of 17.854 in the country), 357 girls (52 %) and 329 boys (48 %). The students' age was from 13 to 15 (M = 13.92; SD = 0.352). Since primary school students are underage, their parents were informed about the research and asked for their consent. During knowledge assessment we must consider why some students do not use the system (motivation problem, lack of PC availability, low computer skill …) (Ibabe & Jauregizar, 2010), and prefer print format over digital because of better navigation, availability and ownership (Precel, Eshet-Alkalai, & Alberton, 2009). Students in every school took three paper-and-pencil knowledge tests: pre-test, test and post-test. As regards of results in the pre-test, students were divided into experimental and control group so that both groups on average reached same achievements. Both groups took 10 school hours to elaborate on the prescribed chapter. While students and teachers in the research group used an online system for knowledge assessment at their work, students and teachers in the control group did not. No specific instructions were given to any of the teacher or student on how to work with the system. After 10 school hours both groups of students solved the second knowledge test. Results of this test were used to compare whether there were any differences between control and experimental group. A month after the second knowledge test the post-test took place with which we tried to determine system's influence on knowledge sustainability. The online database of chemistry tasks was found useful. The system has been established and data analysis shows positive effects on experimental group students' achievements. An independent-samples t-test was conducted to compare mean values of experimental and control group. The test results show that students in the experimental group achieved significantly better results (M = 12.27; SD = 3.51) than students in the control group (M = 11.47; SD = 3.13; t(483.373) = 2.945; p = 0.003). In our presentation we will show specific results of our research and share our findings. We will present some of the options on how to use our online system and its capabilities at school or at home. Meanwhile, we will encourage the audience to use the online system on their ...
Acta Chimica Slovenica, Jun 18, 2013
Variety of online tools were built to help assessing students’ performance in school. Many teache... more Variety of online tools were built to help assessing students’ performance in school. Many teachers changed their methods of assessment from paper-and-pencil (P&P) to online systems. In this study we analyse the influence that using an online system for knowledge assessment has on students’ knowledge. Based on both a literature study and our own research we designed and built an online system for knowledge assessment. The system is evaluated using two groups of primary school teachers and students (N = 686) in Slovenia: an experimental and a control group. Students solved P&P exams on several occasions. The experimental group was allowed to access the system either at school or at home for a limited period during the presentation of a selected school topic. Students in the experimental group were able to solve tasks and compare their own achievements with those of their coevals. A comparison of the P&P school exams results achieved by both groups revealed a positive effect on subject topic comprehension for those with access to the online self-assessment system.
Conference Presentations by Benjamin Kralj
The system for knowledge assessment TikTakTest is available on-line at the www.tiktaktest.si. Sys... more The system for knowledge assessment TikTakTest is available on-line at the www.tiktaktest.si. System is used to save tasks in the on-line database and order them into various groups regarding type of task, taxonomy (Bloom), author, keywords etc. Teachers can save their own tasks into the system and use it together with other tasks that teachers from other schools have saved. Teachers can group tasks into exams for meeting their school need. When students use the system, they can solve tasks from specific chapter, or decide to focus only on tasks from their chemistry teacher. System provides students with randomly chosen tasks to solve. They can also solve exams that prepared their teacher or teacher from any other school that uses the system. After solving tasks, students receive the information about their success and can compare their achievements with their previous attempts, or with their classmates or with all students that solved the tasks. Teachers responded that TikTakTest can be used in schools in various possibilities.
Constructing an online system for knowledge assessment must result in a system that is useful for... more Constructing an online system for knowledge assessment must result in a system that is useful for all types of user, their learning methods and styles. It should be easy to work with it at school or at home, and the online system should use its web capabilities to broaden the data management from more than one school and offer users their data for coeval comparison. When we describe online systems used in natural sciences learning, the system should provide users with delivery of e-learning materials, record of learning experience and profile and management of e-learning materials and e-courses (Chu, Chang, & Tsai, 2009). Acknowledging these characteristics, we can build an online system for knowledge assessment that enables students to learn while solving tasks. Systems like this were used for monitoring students' progress in subject comprehension (Ibabe & Jauregizar, 2010) and for detecting high-risk students before they put their academic career at risk (Beck & Davidson, 2001). 'Knowing the score of the test and predicted trouble areas, individual students can exercise control over their own destiny in the course,' (Kennepohl, Guay, & Thomas, 2010, p. D), and researchers (Marzano, 2010) reported that achievement monitoring has a positive effect on students. Students who use this system can assess their knowledge on their own and acknowledge their weak spots in subject comprehension. With the help of their teacher, they can recoup a loss. In the progress of this research, an online system for chemistry knowledge assessment was developed. Students were able to monitor their results and learn from their solved tasks. The main part of the system was the database of chemistry tasks (since the focus of the research was on chemistry). This database was used in the system through three main functions: (1) entering the learning material, (2) solving tasks stored within the database, and (3) achievement observation available for students and teachers. With its construction, teachers should be able to 'yield more information in less testing time,' (Grenwelge, 2009, p. 345), and the system can provide students with instant feedback since they are more likely to 'pay attention to the electronic feedback that is returned quickly,' (Denton, 2001, p. 7). Research was performed in the school year 2010/2011. We have focused on the 9th grade last year of primary school students (age 13 to 15) and have chosen a chapter from organic chemistry: the group of oxygen organic molecules. Twenty primary schools in Slovenia (4.43 % out of 451) and their chemistry teachers joined the research, and with them 686 students (3.6 % out of 17.854 in the country), 357 girls (52 %) and 329 boys (48 %). The students' age was from 13 to 15 (M = 13.92; SD = 0.352). Since primary school students are underage, their parents were informed about the research and asked for their consent. During knowledge assessment we must consider why some students do not use the system (motivation problem, lack of PC availability, low computer skill …) (Ibabe & Jauregizar, 2010), and prefer print format over digital because of better navigation, availability and ownership (Precel, Eshet-Alkalai, & Alberton, 2009). Students in every school took three paper-and-pencil knowledge tests: pre-test, test and post-test. As regards of results in the pre-test, students were divided into experimental and control group so that both groups on average reached same achievements. Both groups took 10 school hours to elaborate on the prescribed chapter. While students and teachers in the research group used an online system for knowledge assessment at their work, students and teachers in the control group did not. No specific instructions were given to any of the teacher or student on how to work with the system. After 10 school hours both groups of students solved the second knowledge test. Results of this test were used to compare whether there were any differences between control and experimental group. A month after the second knowledge test the post-test took place with which we tried to determine system's influence on knowledge sustainability. The online database of chemistry tasks was found useful. The system has been established and data analysis shows positive effects on experimental group students' achievements. An independent-samples t-test was conducted to compare mean values of experimental and control group. The test results show that students in the experimental group achieved significantly better results (M = 12.27; SD = 3.51) than students in the control group (M = 11.47; SD = 3.13; t(483.373) = 2.945; p = 0.003). In our presentation we will show specific results of our research and share our findings. We will present some of the options on how to use our online system and its capabilities at school or at home. Meanwhile, we will encourage the audience to use the online system on their devices, while answering some general knowledge questions. We will present our vision in developing the system for national and international online knowledge assessment. More data about students' achievements can equip their results with additional value. Users of our online system do not have to wait for yearly summative assessments, since online formative assessment can provide students and teachers with their own results and the results of other students' and receive fast national (or international) evaluation of their achievements. Some questions on how to establish such a system remain open and debatable.
Uploads
Papers by Benjamin Kralj
Conference Presentations by Benjamin Kralj