IEEE Transactions on Learning Technologies, Oct 1, 2021
Adaptive learning technology solutions often use a learner model to trace learning and make pedag... more Adaptive learning technology solutions often use a learner model to trace learning and make pedagogical decisions. The present research introduces a formalized methodology for specifying learner models, Logistic Knowledge Tracing (LKT), that consolidates many extant learner modeling methods. The strength of LKT is the specification of a symbolic notation system for alternative logistic regression models that is powerful enough to specify many extant models in the literature and many new models. To demonstrate the generality of LKT, we fit 12 models, some variants of well-known models and some newly devised, to 6 learning technology datasets. The results indicated that no single learner model was best in all cases, further justifying a broad approach that considers multiple learner model features and the learning context. The models presented here avoid student-level fixed parameters to increase generalizability. We also introduce features to stand in for these intercepts. We argue that to be maximally applicable, a learner model needs to adapt to student differences, rather than needing to be pre-parameterized with the level of each student's ability. Index Terms-Educational technology, computer-aided instruction, learning management systems, models of learning, knowledge tracing, model comparison. I. CREATING AND USING LOGISTIC REGRESSION LEARNER MODELS TO INFORM LEARNING TECHNOLOGY PEDAGOGY Logistic regression is a statistical method that has been used by many investigators to characterize student performance for various learning tasks. In this paper, we explain a formalized approach to creating logistic regression models that subsumes other methods and provides flexibility that allows better determination of an accurate model than off-the-shelf approaches like the Additive Factors Model (AFM), Instructional Factors Analysis (IFA), Performance Factors Analysis (PFA), PFA-Decay or Recent-PFA (R-PFA) [1]-[7]. We focus this work on building models that generalize to new learners for which there is no data, which is a typical situation when attempting to optimize adaptive instruction in a learning system.
The ACT-R computational modeling system encapsulates an activation-based system of declarative me... more The ACT-R computational modeling system encapsulates an activation-based system of declarative memory. While this model has had a variety of successes in modeling memory phenomena, in its current conception it has no mechanism that would produce the spacing effect. A paired-associate memory experiment was conducted and the data from this experiment were fit with an ACT-R memory model using a new decay mechanism. Rather than a set decay rate, this mechanism bases decay for a trial on the current activation at the time of that trial. The spacing effect is a result of this mechanism. Experiment An experiment was conducted to look for evidence of the spacing effect. Our intent with this experiment was to provide a strong challenge to any proposed model of spacing by demonstrating the spacing effect at various levels of practice and spacing lag. Method Participants and Design In this experiment, participants were tested repeatedly over the course of two sessions on their knowledge of Japanese-English vocabulary pairs. There was either 1 or 7 days between the first and second testing sessions (S1 and S2). In both cases during S1, participants were tested on word pairs that were repeated 1, 2, 4, or 8 times spaced at intervals of 2, 14 or 98 trials. This indicates a 3x4 design; however, due to session length limitations the 8 x 98 condition was not included, resulting in 11 conditions. There were 8 word pairs in each condition, and 16 word pairs used for primacy and warm-up buffers as well as to enable the arrangement of the spacing conditions. An attempt was made to space the trials of conditions across the span of S1. Thus, learning trials could occur at any time since new items were introduced continuously. During S2, participants were tested 4 times with each pair at a spacing of 98 trials between tests. 40 participants were recruited for this study from the Pittsburgh, Pennsylvania community. All participants completed the experiment. 20 subjects were used in each condition. Sessions lasted between 60 and 90 minutes. Only subjects that professed no knowledge of Japanese were recruited. Materials The stimuli were a collection of 104 Japanese-English word pairs. English words were chosen according to certain criteria. Words had mean familiarity ratings of 548 and mean imagability ratings of 464 in the MRC Psycholinguistic Database (Coltheart, 1981). The data base mean of familiarity and imagability are 488 (s.d. 120) and 438 (s.d. 99) respectively. Japanese translations (from the possible Japanese synonyms) were chosen for dissimilarity to common English words. Only 4 letter English words were used, and 4 to 7 letter Japanese translations were used. Assignment of words to conditions was randomized for each participant. Procedure Participants arrived at the testing room, signed consent forms, and were told that this was a memory study.
Zenodo (CERN European Organization for Nuclear Research), Jul 5, 2023
In adaptive learning systems, various models are employed to obtain the optimal learning schedule... more In adaptive learning systems, various models are employed to obtain the optimal learning schedule and review for a specific learner. Models of learning are used to estimate the learner's current recall probability by incorporating features or predictors proposed by psychological theory or empirically relevant to learners' performance. Logistic regression for knowledge tracing has been used widely in modern learner performance modeling. Notably, the learning history included in such models is typically confined to learners' prior accuracy performance without paying attention to learners' response time (RT), such as the performance factors analysis (PFA) model. However, RT and accuracy may give us a more comprehensive picture of a learner's learning trajectory. For example, without considering RT, we cannot estimate whether the learner's performance has reached the automatic or fluent level since these criteria are not accuracy based. Therefore, in the current research, we propose and test new RT-related features to capture learners' correct RT fluctuations around their estimated ideal fluent RT. Our results indicate that the predictiveness of the standard PFA model can be increased by up to 10% for our test data after incorporating RTrelated features, but the complexity of the question format constrains the improvement during practice. If the question is of low complexity and the observed accuracy of the learner can be influenced by guessing, which results in the imprecision measured by accuracy, then the RT-related features provide additional predictive power. In other words, RT-related features are informative when accuracy alone does not completely reflect learners' learning processes.
EDM brings together researchers from computer science, education, psychology, psychometrics, and ... more EDM brings together researchers from computer science, education, psychology, psychometrics, and statistics to analyze large data sets to answer educational research questions. The increase in instrumented educational software and databases of student test scores, has created large repositories of data reflecting how students learn. The EDM conference focuses on computational approaches for analyzing those data to address important educational questions. The broad collection of research disciplines ensures cross fertilization of ideas, with the central questions of educational research serving as a unifying focus. We received a total of 54 full papers and 20 submitted posters from 21 countries. Paper submissions were reviewed by three or four reviewers and 23 of them were accepted as full papers (43% acceptance rate). All papers will appear both on the web, at www.educationaldatamining.org , as well as in the printed proceedings. The conference also included invited talks by Professor Cristina Conati , Computer Science Professor,
Zenodo (CERN European Organization for Nuclear Research), Jul 18, 2022
Many models of categorization focus on how people form and use knowledge of categories and make p... more Many models of categorization focus on how people form and use knowledge of categories and make predictions about human categorization behaviors [19]. However, few (if any) of them implement these theories into item selection algorithms for category training. The performance Factors Analysis (PFA) model is an alternative to the Bayesian Knowledge Tracing model that tracks students' learning of knowledge components and can be implemented into adaptive practice algorithms [17]. PFA-Difficulty model has been built to select items based on their difficulty level adaptively [4]. This paper describes how we are working to incorporate categorization theories into the PFA model so that it can be used for item selection. We used experiment data of Mandarin tone categorization training to test the model and suggest the implications of the results for item selection.
There is some research indicating that the presence of music adversely impacts academic task perf... more There is some research indicating that the presence of music adversely impacts academic task performance. While most of this research involves individuals reading text passages, few studies have explored how graphical representations contribute to the auditory distraction literature. The aim of our study was to investigate if concept maps, a graphical representation that depicts relations among concepts, and linear text differentially affect recall when they are studied in the presence of music. Participants studied a preconstructed concept map or text summary while listening to verbal or nonverbal music. Results indicated that participants who studied the concept map with verbal music recalled significantly more ideas than those who studied the text summary. This result was particularly robust for those with low to moderate prior knowledge in the domain being studied. These findings suggest that the novel structure of concept maps may induce greater concentration, which could provi...
This book constitutes the refereed proceedings of the 16th International Conference on Artificial... more This book constitutes the refereed proceedings of the 16th International Conference on Artificial Intelligence in Education, AIED 2013, held in Memphis, TN, USA in July 2013. The 55 revised full papers presented together with 73 poster presentations were carefully reviewed and selected from a total of 168 submissions. The papers are arranged in sessions on student modeling and personalization, open-learner modeling, affective computing and engagement, educational data mining, learning together (collaborative learning and social computing), natural language processing, pedagogical agents, metacognition and self-regulated learning, feedback and scaffolding, designed learning activities, educational games and narrative, and outreach and scaling up.
Proceedings of the Annual Meeting of the Cognitive Science Society, 2007
The FaCT (Fact and Concept Training) System provides a general platform for delivering practice i... more The FaCT (Fact and Concept Training) System provides a general platform for delivering practice in the form of discrete flashcard-like drills. The system optimizes practice schedules according to model-based predictions and can be used to deliver various types of assessment. The system's features satisfy the real world goals of educators using a theory-driven approach that gives researchers control over the model of practice delivery. For educators it provides web deployment, automatic reporting of student practice and assessment, and the ability to tailor content for specific curricular needs. For researchers it provides data export to MySQL, pluggable model architecture, and generalized model fitting algorithms.
This study investigated the effect of cloze item practice on reading comprehension, where cloze i... more This study investigated the effect of cloze item practice on reading comprehension, where cloze items were either created by humans, by machine using natural language processing techniques, or randomly. Participants from Amazon Mechanical Turk (N = 302) took a pre-test, read a text, and took part in one of five conditions, Do-Nothing, Re-Read, Human Cloze, Machine Cloze, or Random Cloze, followed by a 24-hour retention interval and post-test. Participants used the MoFaCTS system [27], which in cloze conditions presented items adaptively based on individual success with each item. Analysis revealed that only Machine Cloze was significantly higher than the Do-Nothing condition on posttest, d = .58, CI95[.21, .94]. Additionally, Machine Cloze was significantly higher than Human and Random Cloze conditions on post-test, d = .49, CI95[.12, .86] and d = .71, CI95[.34, 1.09] respectively. These results suggest that Machine Cloze items generated using natural language processing techniques are effective for enhancing reading comprehension when delivered by an adaptive practice scheduling system.
The effectiveness of Intelligent Tutoring Systems (ITS) research is enhanced by tools that allow ... more The effectiveness of Intelligent Tutoring Systems (ITS) research is enhanced by tools that allow researchers to quickly bridge the divide between theoretical and applied work. By providing a common infrastructure to test cognitive and learning science theories in authentic contexts with real students, the Mobile Fact and Concept Training System (MoFaCTS) can aid in accelerating ITS research and real world implementation. MoFaCTS is run from a web browser and allows the teacher or administrator to set up a sequence of units of content. Because the "optimal practice" module is interchangeable, the system allows for the comparison of alternative methods of adaptive practice. To foster faster research progress, data export supports the DataShop transaction format, which allows quick analysis of data using the DataShop tools. Integration with Amazon Turk allows quick and efficient data collection from this source.
The ACT-R computational modeling system encapsulates an activation-based system of declarative me... more The ACT-R computational modeling system encapsulates an activation-based system of declarative memory. While this model has had a variety of successes in modeling memory phenomena, in its current conception it has no mechanism that would produce the spacing effect. A paired-associate memory experiment was conducted and the data from this experiment were fit with an ACT-R memory model using a new decay mechanism. Rather than a set decay rate, this mechanism bases decay for a trial on the current activation at the time of that trial. The spacing effect is a result of this mechanism. Experiment An experiment was conducted to look for evidence of the spacing effect. Our intent with this experiment was to provide a strong challenge to any proposed model of spacing by demonstrating the spacing effect at various levels of practice and spacing lag. Method Participants and Design In this experiment, participants were tested repeatedly over the course of two sessions on their knowledge of Japanese-English vocabulary pairs. There was either 1 or 7 days between the first and second testing sessions (S1 and S2). In both cases during S1, participants were tested on word pairs that were repeated 1, 2, 4, or 8 times spaced at intervals of 2, 14 or 98 trials. This indicates a 3x4 design; however, due to session length limitations the 8 x 98 condition was not included, resulting in 11 conditions. There were 8 word pairs in each condition, and 16 word pairs used for primacy and warm-up buffers as well as to enable the arrangement of the spacing conditions. An attempt was made to space the trials of conditions across the span of S1. Thus, learning trials could occur at any time since new items were introduced continuously. During S2, participants were tested 4 times with each pair at a spacing of 98 trials between tests. 40 participants were recruited for this study from the Pittsburgh, Pennsylvania community. All participants completed the experiment. 20 subjects were used in each condition. Sessions lasted between 60 and 90 minutes. Only subjects that professed no knowledge of Japanese were recruited. Materials The stimuli were a collection of 104 Japanese-English word pairs. English words were chosen according to certain criteria. Words had mean familiarity ratings of 548 and mean imagability ratings of 464 in the MRC Psycholinguistic Database (Coltheart, 1981). The data base mean of familiarity and imagability are 488 (s.d. 120) and 438 (s.d. 99) respectively. Japanese translations (from the possible Japanese synonyms) were chosen for dissimilarity to common English words. Only 4 letter English words were used, and 4 to 7 letter Japanese translations were used. Assignment of words to conditions was randomized for each participant. Procedure Participants arrived at the testing room, signed consent forms, and were told that this was a memory study.
This research combines work in memory, retrieval practice, and depth of processing research. This... more This research combines work in memory, retrieval practice, and depth of processing research. This work aims to identify how the format and depth of a retrieval practice item can be manipulated to increase the effort required to successfully recall or formulate an answer, with the hypothesis that if the effort required to answer an item is increased there will be more benefit to learning. This hypothesis stems from work on desirable difficulties and the effortful retrieval hypothesis. Our data source was an experiment that used a 2 (question depth: factual, applied) x 2 (answer format: multiple choice, short answer) between-subjects design to investigate the effects of these conditions on retrieval practice performance. The experiment was delivered online though Mechanical Turk (n = 178). A logistic regression predicting performance during practice indicates that participants get more (in terms of an increase in future predicted success) from successful retrievals of items that fall ...
An intelligent textbook may be considered to be an interaction layer that lies between the text a... more An intelligent textbook may be considered to be an interaction layer that lies between the text and the student, helping the student to master the content in the text. The Mobile Fact and Concept Training System (MoFaCTS) is an adaptive instructional system for simple content that has been developed into an interaction layer to mediate textbook instruction and so is being transformed into the Mobile Fact and Concept Textbook System (MoFaCTS). In this project, MoFaCTS is being completely retooled to accept texts from a textbook and to automatically create cloze sentence practice content to help the student learn the material in the text. Additional features in the prototype stage include automatically generated refutational feedback for incorrect cloze responses and a dialog system, which will trigger a short conversation by a tutor to correct conceptual misunderstandings. MoFaCTS administers this content via a web browser, providing the teacher with score reports and class managemen...
Bayesian Knowledge Tracing [1], Performance Factors Analysis [6], MOOC activity analysis [3], and... more Bayesian Knowledge Tracing [1], Performance Factors Analysis [6], MOOC activity analysis [3], and others) or that have been uploaded to LearnSphere as a custom workflow, and (3) sharing their own analysis workflows with the community of researchers. Without any prior programming experience, researchers can use LearnSphere’s drag-and-drop interface to compare, across alternative analysis methods and across many different datasets, model fit metrics like AIC, BIC, and cross validation as well as parameter estimates themselves.
IEEE Transactions on Learning Technologies, Oct 1, 2021
Adaptive learning technology solutions often use a learner model to trace learning and make pedag... more Adaptive learning technology solutions often use a learner model to trace learning and make pedagogical decisions. The present research introduces a formalized methodology for specifying learner models, Logistic Knowledge Tracing (LKT), that consolidates many extant learner modeling methods. The strength of LKT is the specification of a symbolic notation system for alternative logistic regression models that is powerful enough to specify many extant models in the literature and many new models. To demonstrate the generality of LKT, we fit 12 models, some variants of well-known models and some newly devised, to 6 learning technology datasets. The results indicated that no single learner model was best in all cases, further justifying a broad approach that considers multiple learner model features and the learning context. The models presented here avoid student-level fixed parameters to increase generalizability. We also introduce features to stand in for these intercepts. We argue that to be maximally applicable, a learner model needs to adapt to student differences, rather than needing to be pre-parameterized with the level of each student's ability. Index Terms-Educational technology, computer-aided instruction, learning management systems, models of learning, knowledge tracing, model comparison. I. CREATING AND USING LOGISTIC REGRESSION LEARNER MODELS TO INFORM LEARNING TECHNOLOGY PEDAGOGY Logistic regression is a statistical method that has been used by many investigators to characterize student performance for various learning tasks. In this paper, we explain a formalized approach to creating logistic regression models that subsumes other methods and provides flexibility that allows better determination of an accurate model than off-the-shelf approaches like the Additive Factors Model (AFM), Instructional Factors Analysis (IFA), Performance Factors Analysis (PFA), PFA-Decay or Recent-PFA (R-PFA) [1]-[7]. We focus this work on building models that generalize to new learners for which there is no data, which is a typical situation when attempting to optimize adaptive instruction in a learning system.
The ACT-R computational modeling system encapsulates an activation-based system of declarative me... more The ACT-R computational modeling system encapsulates an activation-based system of declarative memory. While this model has had a variety of successes in modeling memory phenomena, in its current conception it has no mechanism that would produce the spacing effect. A paired-associate memory experiment was conducted and the data from this experiment were fit with an ACT-R memory model using a new decay mechanism. Rather than a set decay rate, this mechanism bases decay for a trial on the current activation at the time of that trial. The spacing effect is a result of this mechanism. Experiment An experiment was conducted to look for evidence of the spacing effect. Our intent with this experiment was to provide a strong challenge to any proposed model of spacing by demonstrating the spacing effect at various levels of practice and spacing lag. Method Participants and Design In this experiment, participants were tested repeatedly over the course of two sessions on their knowledge of Japanese-English vocabulary pairs. There was either 1 or 7 days between the first and second testing sessions (S1 and S2). In both cases during S1, participants were tested on word pairs that were repeated 1, 2, 4, or 8 times spaced at intervals of 2, 14 or 98 trials. This indicates a 3x4 design; however, due to session length limitations the 8 x 98 condition was not included, resulting in 11 conditions. There were 8 word pairs in each condition, and 16 word pairs used for primacy and warm-up buffers as well as to enable the arrangement of the spacing conditions. An attempt was made to space the trials of conditions across the span of S1. Thus, learning trials could occur at any time since new items were introduced continuously. During S2, participants were tested 4 times with each pair at a spacing of 98 trials between tests. 40 participants were recruited for this study from the Pittsburgh, Pennsylvania community. All participants completed the experiment. 20 subjects were used in each condition. Sessions lasted between 60 and 90 minutes. Only subjects that professed no knowledge of Japanese were recruited. Materials The stimuli were a collection of 104 Japanese-English word pairs. English words were chosen according to certain criteria. Words had mean familiarity ratings of 548 and mean imagability ratings of 464 in the MRC Psycholinguistic Database (Coltheart, 1981). The data base mean of familiarity and imagability are 488 (s.d. 120) and 438 (s.d. 99) respectively. Japanese translations (from the possible Japanese synonyms) were chosen for dissimilarity to common English words. Only 4 letter English words were used, and 4 to 7 letter Japanese translations were used. Assignment of words to conditions was randomized for each participant. Procedure Participants arrived at the testing room, signed consent forms, and were told that this was a memory study.
Zenodo (CERN European Organization for Nuclear Research), Jul 5, 2023
In adaptive learning systems, various models are employed to obtain the optimal learning schedule... more In adaptive learning systems, various models are employed to obtain the optimal learning schedule and review for a specific learner. Models of learning are used to estimate the learner's current recall probability by incorporating features or predictors proposed by psychological theory or empirically relevant to learners' performance. Logistic regression for knowledge tracing has been used widely in modern learner performance modeling. Notably, the learning history included in such models is typically confined to learners' prior accuracy performance without paying attention to learners' response time (RT), such as the performance factors analysis (PFA) model. However, RT and accuracy may give us a more comprehensive picture of a learner's learning trajectory. For example, without considering RT, we cannot estimate whether the learner's performance has reached the automatic or fluent level since these criteria are not accuracy based. Therefore, in the current research, we propose and test new RT-related features to capture learners' correct RT fluctuations around their estimated ideal fluent RT. Our results indicate that the predictiveness of the standard PFA model can be increased by up to 10% for our test data after incorporating RTrelated features, but the complexity of the question format constrains the improvement during practice. If the question is of low complexity and the observed accuracy of the learner can be influenced by guessing, which results in the imprecision measured by accuracy, then the RT-related features provide additional predictive power. In other words, RT-related features are informative when accuracy alone does not completely reflect learners' learning processes.
EDM brings together researchers from computer science, education, psychology, psychometrics, and ... more EDM brings together researchers from computer science, education, psychology, psychometrics, and statistics to analyze large data sets to answer educational research questions. The increase in instrumented educational software and databases of student test scores, has created large repositories of data reflecting how students learn. The EDM conference focuses on computational approaches for analyzing those data to address important educational questions. The broad collection of research disciplines ensures cross fertilization of ideas, with the central questions of educational research serving as a unifying focus. We received a total of 54 full papers and 20 submitted posters from 21 countries. Paper submissions were reviewed by three or four reviewers and 23 of them were accepted as full papers (43% acceptance rate). All papers will appear both on the web, at www.educationaldatamining.org , as well as in the printed proceedings. The conference also included invited talks by Professor Cristina Conati , Computer Science Professor,
Zenodo (CERN European Organization for Nuclear Research), Jul 18, 2022
Many models of categorization focus on how people form and use knowledge of categories and make p... more Many models of categorization focus on how people form and use knowledge of categories and make predictions about human categorization behaviors [19]. However, few (if any) of them implement these theories into item selection algorithms for category training. The performance Factors Analysis (PFA) model is an alternative to the Bayesian Knowledge Tracing model that tracks students' learning of knowledge components and can be implemented into adaptive practice algorithms [17]. PFA-Difficulty model has been built to select items based on their difficulty level adaptively [4]. This paper describes how we are working to incorporate categorization theories into the PFA model so that it can be used for item selection. We used experiment data of Mandarin tone categorization training to test the model and suggest the implications of the results for item selection.
There is some research indicating that the presence of music adversely impacts academic task perf... more There is some research indicating that the presence of music adversely impacts academic task performance. While most of this research involves individuals reading text passages, few studies have explored how graphical representations contribute to the auditory distraction literature. The aim of our study was to investigate if concept maps, a graphical representation that depicts relations among concepts, and linear text differentially affect recall when they are studied in the presence of music. Participants studied a preconstructed concept map or text summary while listening to verbal or nonverbal music. Results indicated that participants who studied the concept map with verbal music recalled significantly more ideas than those who studied the text summary. This result was particularly robust for those with low to moderate prior knowledge in the domain being studied. These findings suggest that the novel structure of concept maps may induce greater concentration, which could provi...
This book constitutes the refereed proceedings of the 16th International Conference on Artificial... more This book constitutes the refereed proceedings of the 16th International Conference on Artificial Intelligence in Education, AIED 2013, held in Memphis, TN, USA in July 2013. The 55 revised full papers presented together with 73 poster presentations were carefully reviewed and selected from a total of 168 submissions. The papers are arranged in sessions on student modeling and personalization, open-learner modeling, affective computing and engagement, educational data mining, learning together (collaborative learning and social computing), natural language processing, pedagogical agents, metacognition and self-regulated learning, feedback and scaffolding, designed learning activities, educational games and narrative, and outreach and scaling up.
Proceedings of the Annual Meeting of the Cognitive Science Society, 2007
The FaCT (Fact and Concept Training) System provides a general platform for delivering practice i... more The FaCT (Fact and Concept Training) System provides a general platform for delivering practice in the form of discrete flashcard-like drills. The system optimizes practice schedules according to model-based predictions and can be used to deliver various types of assessment. The system's features satisfy the real world goals of educators using a theory-driven approach that gives researchers control over the model of practice delivery. For educators it provides web deployment, automatic reporting of student practice and assessment, and the ability to tailor content for specific curricular needs. For researchers it provides data export to MySQL, pluggable model architecture, and generalized model fitting algorithms.
This study investigated the effect of cloze item practice on reading comprehension, where cloze i... more This study investigated the effect of cloze item practice on reading comprehension, where cloze items were either created by humans, by machine using natural language processing techniques, or randomly. Participants from Amazon Mechanical Turk (N = 302) took a pre-test, read a text, and took part in one of five conditions, Do-Nothing, Re-Read, Human Cloze, Machine Cloze, or Random Cloze, followed by a 24-hour retention interval and post-test. Participants used the MoFaCTS system [27], which in cloze conditions presented items adaptively based on individual success with each item. Analysis revealed that only Machine Cloze was significantly higher than the Do-Nothing condition on posttest, d = .58, CI95[.21, .94]. Additionally, Machine Cloze was significantly higher than Human and Random Cloze conditions on post-test, d = .49, CI95[.12, .86] and d = .71, CI95[.34, 1.09] respectively. These results suggest that Machine Cloze items generated using natural language processing techniques are effective for enhancing reading comprehension when delivered by an adaptive practice scheduling system.
The effectiveness of Intelligent Tutoring Systems (ITS) research is enhanced by tools that allow ... more The effectiveness of Intelligent Tutoring Systems (ITS) research is enhanced by tools that allow researchers to quickly bridge the divide between theoretical and applied work. By providing a common infrastructure to test cognitive and learning science theories in authentic contexts with real students, the Mobile Fact and Concept Training System (MoFaCTS) can aid in accelerating ITS research and real world implementation. MoFaCTS is run from a web browser and allows the teacher or administrator to set up a sequence of units of content. Because the "optimal practice" module is interchangeable, the system allows for the comparison of alternative methods of adaptive practice. To foster faster research progress, data export supports the DataShop transaction format, which allows quick analysis of data using the DataShop tools. Integration with Amazon Turk allows quick and efficient data collection from this source.
The ACT-R computational modeling system encapsulates an activation-based system of declarative me... more The ACT-R computational modeling system encapsulates an activation-based system of declarative memory. While this model has had a variety of successes in modeling memory phenomena, in its current conception it has no mechanism that would produce the spacing effect. A paired-associate memory experiment was conducted and the data from this experiment were fit with an ACT-R memory model using a new decay mechanism. Rather than a set decay rate, this mechanism bases decay for a trial on the current activation at the time of that trial. The spacing effect is a result of this mechanism. Experiment An experiment was conducted to look for evidence of the spacing effect. Our intent with this experiment was to provide a strong challenge to any proposed model of spacing by demonstrating the spacing effect at various levels of practice and spacing lag. Method Participants and Design In this experiment, participants were tested repeatedly over the course of two sessions on their knowledge of Japanese-English vocabulary pairs. There was either 1 or 7 days between the first and second testing sessions (S1 and S2). In both cases during S1, participants were tested on word pairs that were repeated 1, 2, 4, or 8 times spaced at intervals of 2, 14 or 98 trials. This indicates a 3x4 design; however, due to session length limitations the 8 x 98 condition was not included, resulting in 11 conditions. There were 8 word pairs in each condition, and 16 word pairs used for primacy and warm-up buffers as well as to enable the arrangement of the spacing conditions. An attempt was made to space the trials of conditions across the span of S1. Thus, learning trials could occur at any time since new items were introduced continuously. During S2, participants were tested 4 times with each pair at a spacing of 98 trials between tests. 40 participants were recruited for this study from the Pittsburgh, Pennsylvania community. All participants completed the experiment. 20 subjects were used in each condition. Sessions lasted between 60 and 90 minutes. Only subjects that professed no knowledge of Japanese were recruited. Materials The stimuli were a collection of 104 Japanese-English word pairs. English words were chosen according to certain criteria. Words had mean familiarity ratings of 548 and mean imagability ratings of 464 in the MRC Psycholinguistic Database (Coltheart, 1981). The data base mean of familiarity and imagability are 488 (s.d. 120) and 438 (s.d. 99) respectively. Japanese translations (from the possible Japanese synonyms) were chosen for dissimilarity to common English words. Only 4 letter English words were used, and 4 to 7 letter Japanese translations were used. Assignment of words to conditions was randomized for each participant. Procedure Participants arrived at the testing room, signed consent forms, and were told that this was a memory study.
This research combines work in memory, retrieval practice, and depth of processing research. This... more This research combines work in memory, retrieval practice, and depth of processing research. This work aims to identify how the format and depth of a retrieval practice item can be manipulated to increase the effort required to successfully recall or formulate an answer, with the hypothesis that if the effort required to answer an item is increased there will be more benefit to learning. This hypothesis stems from work on desirable difficulties and the effortful retrieval hypothesis. Our data source was an experiment that used a 2 (question depth: factual, applied) x 2 (answer format: multiple choice, short answer) between-subjects design to investigate the effects of these conditions on retrieval practice performance. The experiment was delivered online though Mechanical Turk (n = 178). A logistic regression predicting performance during practice indicates that participants get more (in terms of an increase in future predicted success) from successful retrievals of items that fall ...
An intelligent textbook may be considered to be an interaction layer that lies between the text a... more An intelligent textbook may be considered to be an interaction layer that lies between the text and the student, helping the student to master the content in the text. The Mobile Fact and Concept Training System (MoFaCTS) is an adaptive instructional system for simple content that has been developed into an interaction layer to mediate textbook instruction and so is being transformed into the Mobile Fact and Concept Textbook System (MoFaCTS). In this project, MoFaCTS is being completely retooled to accept texts from a textbook and to automatically create cloze sentence practice content to help the student learn the material in the text. Additional features in the prototype stage include automatically generated refutational feedback for incorrect cloze responses and a dialog system, which will trigger a short conversation by a tutor to correct conceptual misunderstandings. MoFaCTS administers this content via a web browser, providing the teacher with score reports and class managemen...
Bayesian Knowledge Tracing [1], Performance Factors Analysis [6], MOOC activity analysis [3], and... more Bayesian Knowledge Tracing [1], Performance Factors Analysis [6], MOOC activity analysis [3], and others) or that have been uploaded to LearnSphere as a custom workflow, and (3) sharing their own analysis workflows with the community of researchers. Without any prior programming experience, researchers can use LearnSphere’s drag-and-drop interface to compare, across alternative analysis methods and across many different datasets, model fit metrics like AIC, BIC, and cross validation as well as parameter estimates themselves.
Uploads
Papers by Philip Pavlik