Papers by Fusun Sahin
Large-scale Assessments in Education, 2020
The item responses of examinees who rapid-guess, who do not spend enough time reading and engagin... more The item responses of examinees who rapid-guess, who do not spend enough time reading and engaging with an item, will not reflect their true ability on that item. Rapid-disengagement refers to rapidly selecting a response to multiple-choice items (i.e., rapid-guess), omitting items, or providing short-unrelated answers to open-ended items in an unreasonably short time. Such rapid responses need to be identified to enhance the validity of the arguments from the test data. Detection of rapid-guessing behaviors is typically based on identifying a threshold to represent the minimum response time required for the student to have thoughtfully considered a given item. This study investigates whether using response behaviors can improve the detection of rapid-disengagement by investigating two approaches: (a) using response behaviors to decide on the size of the threshold, and (b) using response behaviors as a condition for detecting disengaged examinees in addition to response times, referred as enhanced methods. Process data and item responses from the PISA 2012 computer-based mathematics assessment were used to examine both approaches under threshold values varying from very small (5 s) to very large (60 s). Results suggested that response behaviors can provide meaningful input on establishing the size of the threshold and that while enhanced methods performed better than using only response times in recognizing rapid-disengagement in some cases, no clear pattern was observed as to when such improvement occurs. This study makes a unique contribution by inspecting the response behaviors of disengaged examinees and providing guidelines on using response behaviors to decide on the size of the threshold. This study suggests response behavior categories that can be applicable to many item and response types, which make them suitable for use in digitally-based large-scale assessments.
NCME 2019 Annual Meeting , 2019
Xiaying 'James' Zhang for their contribution to the SAS code used for the multilevel analysis don... more Xiaying 'James' Zhang for their contribution to the SAS code used for the multilevel analysis done in this study.
This study reports on an evaluation tool for instructors of a non-traditional vocational program ... more This study reports on an evaluation tool for instructors of a non-traditional vocational program and the effectiveness of
providing frequent formative feedback to instructors based on data collected from students. A comprehensive survey on
instructional effectiveness was administered across eight urban sites across the USA. Fidelity measures were taken in
administrating the survey, which enabled uniform procedures in collecting data and for reporting the data. The vocational
training consisted of three 7-week modules that spanned 6 months. Around 3000 students participate in the vocational
training program.
Students completed the survey after each module. The instructors received the unpacked feedback from the survey
administrators in digestible format, as well as through discussions with their academic supervisors. The goal of the feedback
framework was to enhance accountability between students and instructors, providing instructional transparency, and
narrowing the lag time variance in acting on student feedback. Unlike broad summative assessments carried out at the end of
trainings, the content and frequency of the feedback in formative evaluations offered instructors quick insights into their
practices and the opportunity to adjust their teaching to students’ learning styles.
The survey was designed based on established educational theories and practices, and instructors were given feedback based
on the practices included in the survey. This study showcases the usefulness of formative assessments of instructors based on
quantitative and qualitative evaluations over a six-month cycle. Results indicated that the instructors employed suggested
teaching practices more frequently in later modules compared to their performance after the first module.
NERA Annual Meeting
There are many software packages that estimate item response theory parameters and examinee abili... more There are many software packages that estimate item response theory parameters and examinee abilities. This study evaluates the open-source ltm package used in R under six conditions on the number of items and examinees. Item and ability estimates were compared with true parameters by, bias, MAD, and RMSE.
In Yan, Z. (Ed). Encyclopedia of Mobile Phone Behavior,Hershey, PA: IGI Global., 2015
Mobile assessment (m-assessment) is a natural extension of incorporating technology into educatio... more Mobile assessment (m-assessment) is a natural extension of incorporating technology into educational assessment. M-assessment is an emerging area as first known studies were published in 2005. Although it is a new area, scholars from all around the world showed interest and examined delivery and effects of m-assessment. Current scientific knowledge encompasses the use of m-assessment in various contexts including mobile environment, classroom, work based settings, informal learning settings, and distance education. Current studies also reported effects of m-assessment on student achievement and attitude as well as advantages and concerns regarding administration of m-assessment. The chapter concludes with stating future directions in four areas including extending the purpose of m-assessment, extending the context in which m-assessment can be used, improving delivery of m-assessment, and advancing research to evaluate effects of m-assessment.
Mobile phones are increasingly popular tools not only for daily use but also for research purpose... more Mobile phones are increasingly popular tools not only for daily use but also for research purposes. The authors systematically searched related literature using mobile phones as a tool for data collection and found 171 publications consisting of review, empirical, methodological, and theoretical studies in various disciplines such as medicine, engineering, and education. After reviewing contributions of previous review studies, the authors presented a description of data collection process consisting of four steps and used these four steps as a framework to review the existing empirical literature. The authors then reviewed contributions of methodological and theoretical studies, and end with a summary of current practices of collecting mobile data. Current challenges and future directions were also mentioned.
Statistical literacy was defined as the ability to understand basic concepts, vocabulary and symb... more Statistical literacy was defined as the ability to understand basic concepts, vocabulary and symbol of statistics, and some probability; and critically evaluate statistical information in everyday life situations. The aim of this study was to develop a valid and reliable instrument measuring statistical literacy for university students. Statistics content covered in previous instruments on statistics learning (CAOS- Web ARTIST Project, 2005; Statistical Literacy Skills Survey, Schield, 2008; ARTIST Topic Scales, 2006) and 6-12 grades curricula implemented in Turkey were examined. A Statistical Content Rating Form
(SLCRF) was formed in the light of knowledge and skills involved in the related domain. Scholars who were offering statistics and research methods courses were asked which statistics topics undergraduate students are required to know for being statistically literate.
Content coverage was determined according to scholars’ answers, and then questions were selected among existing instruments in the literature. For topics that questions in the literature are insufficient then new questions were written by the researcher. Suggested questions were examined by experts and the 42 questions were chosen and tried out with a pilot study with 33 participants. Based on the results, the number of questions was reduced to 20 and it was tried again with a sample consisting of 90 participants. Then, the number of questions was reduced to 17 and Statistical Literacy Scale (SLS) was developed. SLS was administered to 476 undergraduate students. The construct validity of SLS was examined with experts’ item based opinions and results of factor analysis. Content validity
was assured with SLCRF results. From the data gathered from 476 participants the Cronbach alpha coefficient was calculated as .532. It is possible to say that SLS has the attributes of construct, content, and curricular validity.
The purpose of the study giving rise to this article was to highlight factors that play a role in... more The purpose of the study giving rise to this article was to highlight factors that play a role in the identification of gifted students through analysis of literature and current issues in Turkey. The article points out differences between what is typically found in literature and what ocurrs in Turkey. It is based on a review of definitions and data presented by national institutions and programs in addition to available results of research studies. Reviews showed that while many of the issues presented in the literature are similar, many contextual issues pertain to Turkey and other collectivist, highly populated countries which subscribe to an understanding of education which is uniform and reliant upon high stakes examinations.
Statistical literacy can be defined as understanding statistical concepts, understanding statisti... more Statistical literacy can be defined as understanding statistical concepts, understanding statistical results and critical evaluation of statistical information as it is encountered in everyday life. The aim of this study is to understand the most important basic statistics content that is necessary to know to be statistically literate pre-service teachers in the Turkish context. Statistics topics that were rated as necessary by most of the professors of faculty of education and topics rated as necessary by pre-service students were identified and compared. There were differences in answers; the reasons for this discrepancy were questioned.
Citizens who know more about research and statistics are expected to criticize wrong conclusions ... more Citizens who know more about research and statistics are expected to criticize wrong conclusions appeared in media. This study deals with questioning the existence of students' ways of questioning causal conclusions in media and hierarchical statistical literacy levels that Watson and Callingham (2004) proposed. This study is a preliminary study for understanding students' questioning while they are exposed to real life stimuli from a statistical literacy perspective. Seven media excerpts were selected and The Survey of Causality in Media was constructed. The study was carried out with 20 participants. Student answers were coded and compared with the statistical literacy levels.
Book Chapters by Fusun Sahin
The main focus of this chapter will be using mobile assessment (m-assessment) for teaching and le... more The main focus of this chapter will be using mobile assessment (m-assessment) for teaching and learning in formal and informal education. M-assessment has been handy for improving learning realized in traditional and contemporary classrooms such as digital classrooms, informal and formal learning settings, professional development settings, and anywhere that learning happens. M-assessment has increased accessibility anywhere, anytime, and by anyone. Moreover, m-assessment contributed to assessment practice by changing how information is collected and providing various mobile formative and summative assessment tools. Effectiveness of m-assessment for improving learning will be discussed by examining evidences regarding learning achievement, student engagement, and cognitive load. Various challenges of using m-assessment were highlighted. Suggestions for overcoming such challenges and using m-assessment effectively were provided under two headings: (a) smart use and (b) smart design of m-assessment. Future research directions were described.
Uploads
Papers by Fusun Sahin
providing frequent formative feedback to instructors based on data collected from students. A comprehensive survey on
instructional effectiveness was administered across eight urban sites across the USA. Fidelity measures were taken in
administrating the survey, which enabled uniform procedures in collecting data and for reporting the data. The vocational
training consisted of three 7-week modules that spanned 6 months. Around 3000 students participate in the vocational
training program.
Students completed the survey after each module. The instructors received the unpacked feedback from the survey
administrators in digestible format, as well as through discussions with their academic supervisors. The goal of the feedback
framework was to enhance accountability between students and instructors, providing instructional transparency, and
narrowing the lag time variance in acting on student feedback. Unlike broad summative assessments carried out at the end of
trainings, the content and frequency of the feedback in formative evaluations offered instructors quick insights into their
practices and the opportunity to adjust their teaching to students’ learning styles.
The survey was designed based on established educational theories and practices, and instructors were given feedback based
on the practices included in the survey. This study showcases the usefulness of formative assessments of instructors based on
quantitative and qualitative evaluations over a six-month cycle. Results indicated that the instructors employed suggested
teaching practices more frequently in later modules compared to their performance after the first module.
(SLCRF) was formed in the light of knowledge and skills involved in the related domain. Scholars who were offering statistics and research methods courses were asked which statistics topics undergraduate students are required to know for being statistically literate.
Content coverage was determined according to scholars’ answers, and then questions were selected among existing instruments in the literature. For topics that questions in the literature are insufficient then new questions were written by the researcher. Suggested questions were examined by experts and the 42 questions were chosen and tried out with a pilot study with 33 participants. Based on the results, the number of questions was reduced to 20 and it was tried again with a sample consisting of 90 participants. Then, the number of questions was reduced to 17 and Statistical Literacy Scale (SLS) was developed. SLS was administered to 476 undergraduate students. The construct validity of SLS was examined with experts’ item based opinions and results of factor analysis. Content validity
was assured with SLCRF results. From the data gathered from 476 participants the Cronbach alpha coefficient was calculated as .532. It is possible to say that SLS has the attributes of construct, content, and curricular validity.
Book Chapters by Fusun Sahin
providing frequent formative feedback to instructors based on data collected from students. A comprehensive survey on
instructional effectiveness was administered across eight urban sites across the USA. Fidelity measures were taken in
administrating the survey, which enabled uniform procedures in collecting data and for reporting the data. The vocational
training consisted of three 7-week modules that spanned 6 months. Around 3000 students participate in the vocational
training program.
Students completed the survey after each module. The instructors received the unpacked feedback from the survey
administrators in digestible format, as well as through discussions with their academic supervisors. The goal of the feedback
framework was to enhance accountability between students and instructors, providing instructional transparency, and
narrowing the lag time variance in acting on student feedback. Unlike broad summative assessments carried out at the end of
trainings, the content and frequency of the feedback in formative evaluations offered instructors quick insights into their
practices and the opportunity to adjust their teaching to students’ learning styles.
The survey was designed based on established educational theories and practices, and instructors were given feedback based
on the practices included in the survey. This study showcases the usefulness of formative assessments of instructors based on
quantitative and qualitative evaluations over a six-month cycle. Results indicated that the instructors employed suggested
teaching practices more frequently in later modules compared to their performance after the first module.
(SLCRF) was formed in the light of knowledge and skills involved in the related domain. Scholars who were offering statistics and research methods courses were asked which statistics topics undergraduate students are required to know for being statistically literate.
Content coverage was determined according to scholars’ answers, and then questions were selected among existing instruments in the literature. For topics that questions in the literature are insufficient then new questions were written by the researcher. Suggested questions were examined by experts and the 42 questions were chosen and tried out with a pilot study with 33 participants. Based on the results, the number of questions was reduced to 20 and it was tried again with a sample consisting of 90 participants. Then, the number of questions was reduced to 17 and Statistical Literacy Scale (SLS) was developed. SLS was administered to 476 undergraduate students. The construct validity of SLS was examined with experts’ item based opinions and results of factor analysis. Content validity
was assured with SLCRF results. From the data gathered from 476 participants the Cronbach alpha coefficient was calculated as .532. It is possible to say that SLS has the attributes of construct, content, and curricular validity.