Multilevel Multidimensional Item Response Theory Modeling

การทดสอบแบบปรับเหมาะด้วยคอมพิวเตอร์แบบพหุมิติ (Multidimensional Computerized Adaptive Testing : MCAT) เป็นการผสมผสานแนวคิดระหว่างทฤษฎีการตอบสนองข้อสอบแบบพหุมิติ(Multidimensional Item Response Theory : MIRT) กับการทดสอบแบบปรับเหมาะด้วยคอมพิวเตอร์ (Computerized Adaptive Testing : CAT) ซึ่งเป็นการทดสอบที่มีประสิทธิภาพสูง มีความแม่นยำ และลดจำนวนข้อสอบ

The CO 2 laser cutting of three polymeric materials namely polypropylene (PP), polycarbonate (PC) and polymethyl methacrylate (PMMA) is investigated with the aim of evaluating the effect of the main input laser cutting parameters (laser power, cutting speed and compressed air pressure) on laser cutting quality of the different polymers and developing model equations relating input process parameters with the output. The output quality characteristics examined were heat affected zone (HAZ), surface roughness and dimensional accuracy. Twelve sets of tests were carried out for each of the polymer based on the central composite design. Predictive models have been developed by response surface methodology (RSM). First-order response models for HAZ and surface roughness were presented and their adequacy was tested by analysis of variance (ANOVA). It was found that the response is well modeled by a linear function of the input parameters. Response surface contours of HAZ and surface roughness were generated. Mathematical model equations have been presented that estimate HAZ and surface roughness for various input laser cutting parameters. Dimensional accuracies of laser cutting on polymers were examined by dimensional deviation of the actual value from the nominal value. From the analysis, it has been observed that PMMA has less HAZ, followed by PC and PP. For surface roughness, PMMA has better cut edge surface quality than PP and PC. The response models developed can be used for practical purposes by the manufacturing industry. However, all three polymeric materials showed similar diameter errors tendency in spite of different material properties. © 2009 Elsevier Ltd. All rights reserved.

http://ac.els-cdn.com/S0030399209001972/1-s2.0-S0030399209001972-main.pdf?_tid=41191522-1d34-11e4-b1fe-00000aacb35d&acdnat=1407307279_b9d1a06c1be9e0b532e3df78fb47e9b4 
http://www.sciencedirect.com/science/article/pii/S0030399209001972

Computer-based simulations can give a more nuanced understanding of what students know and can do than traditional testing methods. These extended, integrated tasks, however, introduce particular problems, including producing an overwhelming amount of data, multidimensionality, and local dependence. In this paper, we describe an approach to understanding the data from complex performances based on evidence-centered design (Mislevy, Almond, & Lukas, in press), a methodology for devising assessments and for using the evidence observed in complex student performances to make inferences about proficiency. We use as an illustration the NAEP Problem-Solving in Technology-Rich Environments Study, which is being conducted to exemplify how nontraditional skills might be assessed in a sample-based national survey. The paper focuses on the inferential uses of ECD, especially how features are extracted from student performance, how these extractions are evaluated, and how the evaluations are accumulated to make evaluative judgments.

Computer-based simulations can give a more nuanced understanding of what students know and can do than traditional testing methods. These extended, integrated tasks, however, introduce particular problems, including producing an overwhelming amount of data, multidimensionality, and local dependence. In this paper, we describe an approach to understanding the data from complex performances based on evidence-centered design (Mislevy, Almond, & Lukas, in press), a methodology for devising assessments and for using the evidence observed in complex student performances to make inferences about proficiency. We use as an illustration the NAEP Problem-Solving in Technology-Rich Environments Study, which is being conducted to exemplify how nontraditional skills might be assessed in a sample-based national survey. The paper focuses on the inferential uses of ECD, especially how features are extracted from student performance, how these extractions are evaluated, and how the evaluations are accumulated to make evaluative judgments.

Educational policy-makers are placing increasing emphasis on testing. All this energy devoted to standardized educational assessment presents a great opportunity for improving instructional decision-making, if testing programs can provide instructionally meaningful results quickly. Traditional educational assessments usually treat the assessed domain as a single construct, and correspondingly use a unidimensional latent variable model to represent student knowledge on the target domain. However, a single global score for student achievement is likely useful only for the coarsest of educational decisions. To be most useful for teachers' instructional decisions, an educational test needs to provide detailed diagnostic information about students. While diagnostic assessment models do exist (such as Diagnostic Classification Models and, to a smaller extent, Multidimensional Item Response Theory), most tests that use these models ignore the structure of the sub-domains they diagnose, either treating them as independent or arbitrarily correlated. But, testing efficiencies may be available if the structural relationships between sub-domains are modeled explicitly. These structural relationships include pre-requisite relationships (learning `A' is required to learn `B') and hierarchical relationships (both `A' and `B' are concepts within the larger field of `X'). Graphical modeling (e.g., Bayesian networks and structural equation models) provides a convenient and intuitive way to represent and model these structural relationships explicitly. This dissertation investigates the use of graphical knowledge models for educational assessment in four parts: First, the existing literature on the use of Bayesian networks in educational assessment is thoroughly reviewed, identifying paths for future research. Second, a graphical knowledge model is developed for the domain of an operational curriculum-linked assessment (exams for a university physics course in classical mechanics), providing an authentic example of model development and use. Third, recovery of network, item, and person parameters for tests based on graphical knowledge models is investigated via simulation to guide certain practical questions in field-testing and calibrating such models for educational assessments. Finally, graphical knowledge models are placed in a Computerized Adaptive Testing context, and performance of a mutual information-based item selection criterion is investigated when the item bank provided different levels of information for different sub-domains. For the physics exam, all path models fit the item responses better than a unidimensional model and a multidimensional model that treated all sub-domains as independent. Both an expert-derived model for sub-domain relationships and a linear model that connected sub-domains in the order in which they were taught exhibited adequate model fit, though the linear model performed slightly better than the expert-derived model. The parameter recovery study found that the precision of person and path parameters depend on where the parameter sits in the network: Sub-domains with many descendants benefit from additional information provided by items for their descendants' sub-domains, particularly as the strength of the relationships increases. Sub-domains with many parents and few or no descendants may need greater numbers of items to achieve similar measurement precision. Moreover, the parameter recovery study demonstrated that network structures may be investigated with very modest test and sample sizes (3 items per sub-domain and 300 examinees), though larger samples are required for precise item and person parameters. Finally, the restricted mutual information-based item selection criteria investigated successfully equalized precision across sub-domains in item banks that provided more information for some sub-domains than others, and the three alternative forms of the restricted criteria performed similarly.