The present study examines the dimensions of reading interest that may predict undergraduate stud... more The present study examines the dimensions of reading interest that may predict undergraduate students' reading behaviors in this digital age. The a priori framework in conceptualizing the subcomponents of reading interest is based on two modes of reading (printed-text based and Internetbased), two purposes of reading (academic and recreational), and three types of psycho-behavioral motives/intentions of reading (affective, cognitive, and behavioral). Participants in this study were students (M = 20.14 years old) from an Indonesian university (n = 993). Exploratory and confirmatory factor analyses showed the salience of 15 factors across reading modes, purposes of reading, and psycho-behavioral domains of reading. It was shown that the primary source of the 15 factors was interpreted as psycho-behavior dimensions rather than reading modes or purposes of reading. Analyses of variance confirmed that student interest in reading differs mostly by university year level, study major, and self-expected education. The most acceptable SEM models that explore the relationships among the sub-components of reading interest had the student reading interest in the print mode related to interest in reading online, and interest in recreational reading is linked to the development of academic reading. Four distinct groups that characterized student pattern of reading interest were identified with latent profile analysis. Implications of these findings are discussed for theory development and practice.
Educational Assessment, Evaluation and Accountability
Although there has been intense criticism of NAPLAN in educational policy debates in Australia, l... more Although there has been intense criticism of NAPLAN in educational policy debates in Australia, little scholarly efforts have been made to understand the underlying cognitive mechanisms that contribute to the public narrative about the national testing program. We aim to provide tentative evidence about the way public perceptions about NAPLAN may be formed. Our results show empirical support for the incentive, interpretative, and institutional effects, which suggest ways that national testing program can be improved. That is, it needs to (a) provide a diverse range of incentives to promote people’s self-interest (incentive effect); (b) demonstrate good alignment with the core values, social norms, and attitudes of the given society (interpretative effect); and (c) build a consensus about the institutional use of the test results (institutional effect). We conclude with practical implications and recommendations about seeking public support for the seemingly unpopular national educat...
COVID-19 booster vaccinations have been recommended as a primary line of defence against serious ... more COVID-19 booster vaccinations have been recommended as a primary line of defence against serious illness and hospitalisation. This study identifies and characterises distinct profiles of attitudes towards vaccination, particularly the willingness to get a booster dose. A sample of 582 adults from Australia completed an online survey capturing COVID-related behaviours, beliefs and attitudes and a range of sociodemographic, psychological, political, social and cultural variables. Latent Profile Analysis (LPA) identified three subgroups: Acceptant (61%), Hesitant (30%) and Resistant (9%). Compared to the Acceptant group, the Hesitant and Resistant groups were less worried about catching COVID-19, used fewer official COVID-19 information sources, checked the news less, were lower on the agreeableness personality dimension and reported more conservatism, persecutory thinking, amoral attitudes and need for chaos. The Hesitant group also reported checking the legitimacy of information sour...
Diagnosis of misconceptions or ‘‘bugs’’ in procedural skills is difficult because of their unstab... more Diagnosis of misconceptions or ‘‘bugs’’ in procedural skills is difficult because of their unstable nature. This study addresses this problem by proposing and evaluating a probability-based approach to the diagnosis of bugs in children’s multicolumn subtraction performance using Bayesian networks. This approach assumes a causal network relating hypothesized subtraction bugs to the observed test items. Two research questions are tested within this framework. First, it is investigated whether more reliable assessment of latent subtraction bugs can be achieved by hypothesizing and using subskill nodes in the Bayesian network as causal factors affecting bugs. Second, network performance is evaluated using two types of testing situations, one using binary data (items scored as correct or incorrect) and the other simulating a multiple-choice test format with diagnostic use of specific wrong answers. The resulting four types of Bayesian networks are evaluated for their effectiveness in bug...
The present study examined the globally relevant predictors of adolescents’ subjective well-being... more The present study examined the globally relevant predictors of adolescents’ subjective well-being (SWB) using data from the Programme for International Student Assessment (PISA) 2015 Student Questionnaire. Guided by the social-cognitive-behavior model (Suldo et al., 2008) and self-determination theory (Ryan & Deci, 2000), nine variables were selected and subjected to a statistical procedure of decision tree modeling (DTM). Our results showed that parent and peer support were the most salient predictors of adolescents’ SWB across all eight world regions under investigation, followed by anxiety and gender. However, we also noted some variations in the relative importance of parent and peer support and multiple pathways that lead to SWB. The study concludes with practical implications.
EURASIA Journal of Mathematics, Science and Technology Education, 2019
This study presents empirical findings demonstrating predictive validities of noncognitive variab... more This study presents empirical findings demonstrating predictive validities of noncognitive variables for mathematics achievement among primary and secondary school students from crosscountry perspectives. Results based on TIMSS 2015 assessment showed that confidence was a moderately strong predictor of mathematics achievement in all TIMSS participant countries (100%). Interest, on the other hand, was a moderately strong predictor in most countries for eighth-graders (77%) but only in about a dozen countries for fourth-graders (20%), showing a stronger interestachievement link for the secondary school students, from the cross-national perspective. The sense of belonging measure demonstrated a lack of its relevance to primary school students' achievement. Further, bullying showed predictive relevance for student achievement only in three countries (South Africa, Jordan, and Egypt). The study concludes that while confidence was a universally relevant predictor of student achievement across all countries/regions, predictive utilities of interest, sense of belonging, and bullying appeared to be heavily dependent on a grade-level and country/regional context.
This paper examined the effects of training in creative problem-solving on intelligence. We revis... more This paper examined the effects of training in creative problem-solving on intelligence. We revisited Stankov’s report on the outcomes of an experiment carried out by R. Kvashchev in former Yugoslavia that reported an IQ increase of seven points, on average, across 28 tests of intelligence. We argue that previous analyses were based on a conservative analytic approach and failed to take into account the reductions in the IQ test variances at the end of the three-years’ training. When standard deviations of the initial test and 2nd retest were pooled in the calculation of the effect sizes, the experimental group’s performance was 10 IQ points higher on average than that of the control group. Further, with the properly defined measures of fluid and crystallized intelligence, the experimental group showed a 15 IQ points higher increase than the control group. We concluded that prolonged intensive training in creative problem-solving can lead to substantial and positive effects on intel...
This study aims to identify which socioeconomic status (SES) variables have the best predictive v... more This study aims to identify which socioeconomic status (SES) variables have the best predictive validity for academic achievement, based on the international data sets of the Programme for International Student Assessment (PISA) in 2012, 2009, 2006, and 2003. From among 10 SES measures, two composite variables-Index of economic, social and cultural status (ESCS) and Home possessions (HOMEPOS)-showed superior predictive power for student achievement. Their pan-cultural correlations with the PISA 2012 mathematics achievement were r = .40 and r = .36, respectively. Parental occupation status (r = .33) outperformed all other single measures of SES, including parental education (r = .29). Only two SES variables (i.e., family wealth and home possessions) showed non-linear relationships with academic achievement. We conclude with practical implications and recommendations for using SES measures as predictors of student achievement in educational research and point to the importance of a theoretical alignment between SES measures and particular issues to be addressed. Varied views exist about core definition, measurement tools, and optimal indicators of socioeconomic status (SES) measures, in spite of 150 years of relevant research (see Oakes & Rossi, 2003). Based on SES interpretations evolved and elaborated from 1970s to 1990s, a simpler definition was offered: "differential access (realized and potential) to desired resources" (Oakes & Rossi, 2003, p. 775). An earlier account of SES emphasizing power (control over others), prestige (honor and social position) and economic dimensions (Mueller & Parcel, 1981) seems to be losing its popularity while the three main domains expressed in Coleman's (1988) social capital theorypossession or lack of material capital (e.g., parental income), human capital (e.g., parental education) and social capital (e.g., social networks)appear to be a dominant framework for SES in many recent studies (e.g., Oakes & Rossi, 2003). The present study is grounded in three lines of related work: (a) Coleman's (1988) social capital theory, (b) the concepts delineated in the "Big Three" variables ("family income, parental educational attainment, and parental occupational status", p. 4) in the U.S. Department of Education (2012) report, and (c) the predictability gradient hypothesis by Stankov (2013) that provides a framework for evaluating predictive validity of socio-psychological measures of student achievement. Close, logical links exist between Coleman's (1988) three social capital domains and the "Big Three" variables since income can be interpreted as material capital, education attainment as human capital, and parental occupation status as an important vehicle for social networks and social capital. The "Big Three", traditionally known as the tripartite of SES background indicators (Duncan, Featherman, & Duncan, 1972), suggests that relatively simple measures of education, occupation, and income can be employed as useful proxies of one's SES position within a particular social structure (Conger, Conger, & Martin, 2010; Mueller & Parcel, 1981; Sirin, 2005). In educational research, a host of other SES measures have been employed beyond the tripartite of family SES, including home educational resources, household possessions, student eligibility for governmentsponsored programs, and neighborhood or school SES (Sirin, 2005).
This study investigated the relationship between students' actual performance (accuracy) and thei... more This study investigated the relationship between students' actual performance (accuracy) and their subjective judgments of accuracy (confidence) on selected English language proficiency tests. The unidimensional and multidimensional IRT Rasch approaches were used to model the discrepancy between confidence and accuracy at the item and test level and to assess disattenuated strength of association between accuracy and confidence. The analysis results indicate a pattern of overconfidence bias (i.e., overestimation of success rate), which was related to item difficulty. In addition, the strength of association between accuracy and confidence dimension was relatively high: The confidence dimension explained 45% and 52% of the variability in the accuracy dimension for the two tests employed in this study.
The use of IRT models has not been rigorously applied in studies of the relationship between test... more The use of IRT models has not been rigorously applied in studies of the relationship between test-takers' confidence and accuracy. This study applied the Rasch measurement models to investigate the relationship between test-takers' confidence and accuracy on English proficiency tests, proposing potentially useful measures of under or overconfidence. The Rasch approach provided the scaffolding to formulate indices that can assess the discrepancy between confidence and accuracy at the item or total test level, as well as at particular ability levels locally. In addition, a "disattenuated" measure of association between…
The present study investigates higher-order factor structure among fifteen primary variables sele... more The present study investigates higher-order factor structure among fifteen primary variables selected from four broad noncognitive domains of academic self-beliefs, motivation, learning strategy, and attitudes toward school. The PISA 2003 international dataset was analyzed. Several EFA, CFA, and SEM models were tested, hypothesizing the structure among the primary first-order variables and their relationships to the mathematics scores. The analyses indicated no single, general factor at the second-order level, encompassing all fifteen first-order variables. Instead, the fifteen primary variables were best represented by a three-level factor structure with the four salient domain factors at the second-order level and one general noncognitive factor at the third-order. The most plausible SEM model had each of the three self-belief primary variables individually linked to the mathematics achievement scores, independent of the third-order factor. Self-efficacy was the strongest predictor of mathematics achievement and its predictive power was comparable to that of the common part of all 15 primary variables captured by the general noncognitive factor.
ogenic' environment. Studying the factors hypothesised to underpin behaviours associated with res... more ogenic' environment. Studying the factors hypothesised to underpin behaviours associated with resistance to overconsumption may inform weight management strategies in an adverse environment. Trait (BIS-11) and behavioural (response inhibition, GoStop) self control were assessed in the laboratory. Snack food consumption was measured covertly via a sham taste test. Lack of motor control was positively correlated (r = .32, p <. 05) and successful response inhibition was negatively correlated (r = À.35, p <. 05) with snack food intake. Low motor control was also associated with further food intake when satiated (r = .39, p < .01). These relationships were independent of selfreported palatability and perceived reward value of the food. Motor control may be an important factor implicated in 'mindless' eating in an environment abundant in palatable, energy-dense snack foods.
In this paper we report the results from a study that assessed confidence together with scales me... more In this paper we report the results from a study that assessed confidence together with scales measuring self-beliefi.e., self-efficacy, different kinds of self-concepts, and anxietyamong the 15-year old students from Singapore. A distinct confidence factor was identified in the domains of mathematics (N= 1940) and English (N=1786). Our results show that confidence is: a) a robust individual differences dimension; b) that can be combined with accuracy information to obtain bias scores that may be useful for group comparisons and for identification of misconceptions about particular topics. Confidence as studied in our work to date has been c) the best predictor of achievement in both mathematics and English; d) is related to both cognitive and self-belief measures; and e) it captures much of the predictive variance of other self-beliefs that are, in turn, among the best known predictors of achievement.
ABSTRACT This is a review of five studies that reported new empirical data relevant for the predi... more ABSTRACT This is a review of five studies that reported new empirical data relevant for the predictability gradient hypothesis. This hypothesis is focused on within-person psychological variables typically collected in background questionnaires that examine the role of non-cognitive influences on students’ academic achievement. Broad measures of maladjustment and motivation/goal orientation have the lowest correlations with achievement. Measures of confidence, on the other hand, have the highest predictive validity. The other self-beliefs measures are in the middle, although they can also be ordered from lower (self-concept) through medium (academic anxiety) to high (self-efficacy) levels of predictability.
European Journal of Psychological Assessment, 2009
This paper addresses measurement and conceptual issues related to the realism of people’s confide... more This paper addresses measurement and conceptual issues related to the realism of people’s confidence judgments about their own cognitive abilities. We employed three cognitive tests: listening and reading subtests from the Test of English as a Foreign Language (TOEFL iBT) and a synonyms vocabulary test. The sample consisted of community college students. Our results show that the participants tend to be overconfident about their cognitive abilities on most tasks, representing poor realism. Significant group differences were noted with respect to gender and race/ethnicity: female and European American participants showed smaller levels of overconfidence than males and African Americans or Hispanics. We point out that there appear to be significant individual differences in the understanding of subjective probabilities, and these differences can influence the realism of confidence judgments.
We investigated the factorial structure of four major domains in social psychology (personality t... more We investigated the factorial structure of four major domains in social psychology (personality traits, social attitudes, values, and social norms) with an emphasis on cross-cultural differences. Three distinctive approachespancultural, multigroup, and multilevel-were applied to the data based on 22 measures that were collected from 2029 participants from 73 countries. First, in a pancultural approach, exploratory and confirmatory factor analyses were conducted on the entire sample of participants, disregarding country of origin. Second, in the multigroup (etic/emic) approach, nine societal clusters were fitted using a four-factor model. Several tests of invariance were applied to assess equivalence across the cultures. Finally, a multilevel approach was used to assess the structure at the individual-level and at the between-country (ecological) level. Our results show that the fourfactor solution emerges from the cultural domains, and this is supported by all three approaches. The factors are Personality/Social Attitudes, Values, Social Norms, and Conservatism. In the multilevel analysis, only two factors emerge at the ecological (between) level as opposed to four factors at the individual (within) level, but due to methodological difficulties, their status needs to be studied further. We discuss our findings in terms of the insideout view of social interactions.
In this article, nine world regions (based on samples from 33 nations) are compared in their perf... more In this article, nine world regions (based on samples from 33 nations) are compared in their performance on a cognitive ability test and confidence ratings obtained from the items of the same test. Our results indicate that differences between the world regions are greater on cognitive ability than they are on confidence ratings. Consequently, overconfidence—that is, the degree to which people overestimate their performance on cognitive tasks—is pronounced within the world regions that have lower scores on measures of cognitive ability. A less pronounced overconfidence is also present among the high-achieving world regions. Our findings support a cognitive hypothesis according to which individuals suffer from illusory superiority if the task is difficult. Thus, a commonly observed overconfidence can be seen as a self-deceiving, probably unconscious, mechanism that cushions a person (and countries) from experiencing negative feelings due to cognitive failures.
The present study examines the dimensions of reading interest that may predict undergraduate stud... more The present study examines the dimensions of reading interest that may predict undergraduate students' reading behaviors in this digital age. The a priori framework in conceptualizing the subcomponents of reading interest is based on two modes of reading (printed-text based and Internetbased), two purposes of reading (academic and recreational), and three types of psycho-behavioral motives/intentions of reading (affective, cognitive, and behavioral). Participants in this study were students (M = 20.14 years old) from an Indonesian university (n = 993). Exploratory and confirmatory factor analyses showed the salience of 15 factors across reading modes, purposes of reading, and psycho-behavioral domains of reading. It was shown that the primary source of the 15 factors was interpreted as psycho-behavior dimensions rather than reading modes or purposes of reading. Analyses of variance confirmed that student interest in reading differs mostly by university year level, study major, and self-expected education. The most acceptable SEM models that explore the relationships among the sub-components of reading interest had the student reading interest in the print mode related to interest in reading online, and interest in recreational reading is linked to the development of academic reading. Four distinct groups that characterized student pattern of reading interest were identified with latent profile analysis. Implications of these findings are discussed for theory development and practice.
Educational Assessment, Evaluation and Accountability
Although there has been intense criticism of NAPLAN in educational policy debates in Australia, l... more Although there has been intense criticism of NAPLAN in educational policy debates in Australia, little scholarly efforts have been made to understand the underlying cognitive mechanisms that contribute to the public narrative about the national testing program. We aim to provide tentative evidence about the way public perceptions about NAPLAN may be formed. Our results show empirical support for the incentive, interpretative, and institutional effects, which suggest ways that national testing program can be improved. That is, it needs to (a) provide a diverse range of incentives to promote people’s self-interest (incentive effect); (b) demonstrate good alignment with the core values, social norms, and attitudes of the given society (interpretative effect); and (c) build a consensus about the institutional use of the test results (institutional effect). We conclude with practical implications and recommendations about seeking public support for the seemingly unpopular national educat...
COVID-19 booster vaccinations have been recommended as a primary line of defence against serious ... more COVID-19 booster vaccinations have been recommended as a primary line of defence against serious illness and hospitalisation. This study identifies and characterises distinct profiles of attitudes towards vaccination, particularly the willingness to get a booster dose. A sample of 582 adults from Australia completed an online survey capturing COVID-related behaviours, beliefs and attitudes and a range of sociodemographic, psychological, political, social and cultural variables. Latent Profile Analysis (LPA) identified three subgroups: Acceptant (61%), Hesitant (30%) and Resistant (9%). Compared to the Acceptant group, the Hesitant and Resistant groups were less worried about catching COVID-19, used fewer official COVID-19 information sources, checked the news less, were lower on the agreeableness personality dimension and reported more conservatism, persecutory thinking, amoral attitudes and need for chaos. The Hesitant group also reported checking the legitimacy of information sour...
Diagnosis of misconceptions or ‘‘bugs’’ in procedural skills is difficult because of their unstab... more Diagnosis of misconceptions or ‘‘bugs’’ in procedural skills is difficult because of their unstable nature. This study addresses this problem by proposing and evaluating a probability-based approach to the diagnosis of bugs in children’s multicolumn subtraction performance using Bayesian networks. This approach assumes a causal network relating hypothesized subtraction bugs to the observed test items. Two research questions are tested within this framework. First, it is investigated whether more reliable assessment of latent subtraction bugs can be achieved by hypothesizing and using subskill nodes in the Bayesian network as causal factors affecting bugs. Second, network performance is evaluated using two types of testing situations, one using binary data (items scored as correct or incorrect) and the other simulating a multiple-choice test format with diagnostic use of specific wrong answers. The resulting four types of Bayesian networks are evaluated for their effectiveness in bug...
The present study examined the globally relevant predictors of adolescents’ subjective well-being... more The present study examined the globally relevant predictors of adolescents’ subjective well-being (SWB) using data from the Programme for International Student Assessment (PISA) 2015 Student Questionnaire. Guided by the social-cognitive-behavior model (Suldo et al., 2008) and self-determination theory (Ryan & Deci, 2000), nine variables were selected and subjected to a statistical procedure of decision tree modeling (DTM). Our results showed that parent and peer support were the most salient predictors of adolescents’ SWB across all eight world regions under investigation, followed by anxiety and gender. However, we also noted some variations in the relative importance of parent and peer support and multiple pathways that lead to SWB. The study concludes with practical implications.
EURASIA Journal of Mathematics, Science and Technology Education, 2019
This study presents empirical findings demonstrating predictive validities of noncognitive variab... more This study presents empirical findings demonstrating predictive validities of noncognitive variables for mathematics achievement among primary and secondary school students from crosscountry perspectives. Results based on TIMSS 2015 assessment showed that confidence was a moderately strong predictor of mathematics achievement in all TIMSS participant countries (100%). Interest, on the other hand, was a moderately strong predictor in most countries for eighth-graders (77%) but only in about a dozen countries for fourth-graders (20%), showing a stronger interestachievement link for the secondary school students, from the cross-national perspective. The sense of belonging measure demonstrated a lack of its relevance to primary school students' achievement. Further, bullying showed predictive relevance for student achievement only in three countries (South Africa, Jordan, and Egypt). The study concludes that while confidence was a universally relevant predictor of student achievement across all countries/regions, predictive utilities of interest, sense of belonging, and bullying appeared to be heavily dependent on a grade-level and country/regional context.
This paper examined the effects of training in creative problem-solving on intelligence. We revis... more This paper examined the effects of training in creative problem-solving on intelligence. We revisited Stankov’s report on the outcomes of an experiment carried out by R. Kvashchev in former Yugoslavia that reported an IQ increase of seven points, on average, across 28 tests of intelligence. We argue that previous analyses were based on a conservative analytic approach and failed to take into account the reductions in the IQ test variances at the end of the three-years’ training. When standard deviations of the initial test and 2nd retest were pooled in the calculation of the effect sizes, the experimental group’s performance was 10 IQ points higher on average than that of the control group. Further, with the properly defined measures of fluid and crystallized intelligence, the experimental group showed a 15 IQ points higher increase than the control group. We concluded that prolonged intensive training in creative problem-solving can lead to substantial and positive effects on intel...
This study aims to identify which socioeconomic status (SES) variables have the best predictive v... more This study aims to identify which socioeconomic status (SES) variables have the best predictive validity for academic achievement, based on the international data sets of the Programme for International Student Assessment (PISA) in 2012, 2009, 2006, and 2003. From among 10 SES measures, two composite variables-Index of economic, social and cultural status (ESCS) and Home possessions (HOMEPOS)-showed superior predictive power for student achievement. Their pan-cultural correlations with the PISA 2012 mathematics achievement were r = .40 and r = .36, respectively. Parental occupation status (r = .33) outperformed all other single measures of SES, including parental education (r = .29). Only two SES variables (i.e., family wealth and home possessions) showed non-linear relationships with academic achievement. We conclude with practical implications and recommendations for using SES measures as predictors of student achievement in educational research and point to the importance of a theoretical alignment between SES measures and particular issues to be addressed. Varied views exist about core definition, measurement tools, and optimal indicators of socioeconomic status (SES) measures, in spite of 150 years of relevant research (see Oakes & Rossi, 2003). Based on SES interpretations evolved and elaborated from 1970s to 1990s, a simpler definition was offered: "differential access (realized and potential) to desired resources" (Oakes & Rossi, 2003, p. 775). An earlier account of SES emphasizing power (control over others), prestige (honor and social position) and economic dimensions (Mueller & Parcel, 1981) seems to be losing its popularity while the three main domains expressed in Coleman's (1988) social capital theorypossession or lack of material capital (e.g., parental income), human capital (e.g., parental education) and social capital (e.g., social networks)appear to be a dominant framework for SES in many recent studies (e.g., Oakes & Rossi, 2003). The present study is grounded in three lines of related work: (a) Coleman's (1988) social capital theory, (b) the concepts delineated in the "Big Three" variables ("family income, parental educational attainment, and parental occupational status", p. 4) in the U.S. Department of Education (2012) report, and (c) the predictability gradient hypothesis by Stankov (2013) that provides a framework for evaluating predictive validity of socio-psychological measures of student achievement. Close, logical links exist between Coleman's (1988) three social capital domains and the "Big Three" variables since income can be interpreted as material capital, education attainment as human capital, and parental occupation status as an important vehicle for social networks and social capital. The "Big Three", traditionally known as the tripartite of SES background indicators (Duncan, Featherman, & Duncan, 1972), suggests that relatively simple measures of education, occupation, and income can be employed as useful proxies of one's SES position within a particular social structure (Conger, Conger, & Martin, 2010; Mueller & Parcel, 1981; Sirin, 2005). In educational research, a host of other SES measures have been employed beyond the tripartite of family SES, including home educational resources, household possessions, student eligibility for governmentsponsored programs, and neighborhood or school SES (Sirin, 2005).
This study investigated the relationship between students' actual performance (accuracy) and thei... more This study investigated the relationship between students' actual performance (accuracy) and their subjective judgments of accuracy (confidence) on selected English language proficiency tests. The unidimensional and multidimensional IRT Rasch approaches were used to model the discrepancy between confidence and accuracy at the item and test level and to assess disattenuated strength of association between accuracy and confidence. The analysis results indicate a pattern of overconfidence bias (i.e., overestimation of success rate), which was related to item difficulty. In addition, the strength of association between accuracy and confidence dimension was relatively high: The confidence dimension explained 45% and 52% of the variability in the accuracy dimension for the two tests employed in this study.
The use of IRT models has not been rigorously applied in studies of the relationship between test... more The use of IRT models has not been rigorously applied in studies of the relationship between test-takers' confidence and accuracy. This study applied the Rasch measurement models to investigate the relationship between test-takers' confidence and accuracy on English proficiency tests, proposing potentially useful measures of under or overconfidence. The Rasch approach provided the scaffolding to formulate indices that can assess the discrepancy between confidence and accuracy at the item or total test level, as well as at particular ability levels locally. In addition, a "disattenuated" measure of association between…
The present study investigates higher-order factor structure among fifteen primary variables sele... more The present study investigates higher-order factor structure among fifteen primary variables selected from four broad noncognitive domains of academic self-beliefs, motivation, learning strategy, and attitudes toward school. The PISA 2003 international dataset was analyzed. Several EFA, CFA, and SEM models were tested, hypothesizing the structure among the primary first-order variables and their relationships to the mathematics scores. The analyses indicated no single, general factor at the second-order level, encompassing all fifteen first-order variables. Instead, the fifteen primary variables were best represented by a three-level factor structure with the four salient domain factors at the second-order level and one general noncognitive factor at the third-order. The most plausible SEM model had each of the three self-belief primary variables individually linked to the mathematics achievement scores, independent of the third-order factor. Self-efficacy was the strongest predictor of mathematics achievement and its predictive power was comparable to that of the common part of all 15 primary variables captured by the general noncognitive factor.
ogenic' environment. Studying the factors hypothesised to underpin behaviours associated with res... more ogenic' environment. Studying the factors hypothesised to underpin behaviours associated with resistance to overconsumption may inform weight management strategies in an adverse environment. Trait (BIS-11) and behavioural (response inhibition, GoStop) self control were assessed in the laboratory. Snack food consumption was measured covertly via a sham taste test. Lack of motor control was positively correlated (r = .32, p <. 05) and successful response inhibition was negatively correlated (r = À.35, p <. 05) with snack food intake. Low motor control was also associated with further food intake when satiated (r = .39, p < .01). These relationships were independent of selfreported palatability and perceived reward value of the food. Motor control may be an important factor implicated in 'mindless' eating in an environment abundant in palatable, energy-dense snack foods.
In this paper we report the results from a study that assessed confidence together with scales me... more In this paper we report the results from a study that assessed confidence together with scales measuring self-beliefi.e., self-efficacy, different kinds of self-concepts, and anxietyamong the 15-year old students from Singapore. A distinct confidence factor was identified in the domains of mathematics (N= 1940) and English (N=1786). Our results show that confidence is: a) a robust individual differences dimension; b) that can be combined with accuracy information to obtain bias scores that may be useful for group comparisons and for identification of misconceptions about particular topics. Confidence as studied in our work to date has been c) the best predictor of achievement in both mathematics and English; d) is related to both cognitive and self-belief measures; and e) it captures much of the predictive variance of other self-beliefs that are, in turn, among the best known predictors of achievement.
ABSTRACT This is a review of five studies that reported new empirical data relevant for the predi... more ABSTRACT This is a review of five studies that reported new empirical data relevant for the predictability gradient hypothesis. This hypothesis is focused on within-person psychological variables typically collected in background questionnaires that examine the role of non-cognitive influences on students’ academic achievement. Broad measures of maladjustment and motivation/goal orientation have the lowest correlations with achievement. Measures of confidence, on the other hand, have the highest predictive validity. The other self-beliefs measures are in the middle, although they can also be ordered from lower (self-concept) through medium (academic anxiety) to high (self-efficacy) levels of predictability.
European Journal of Psychological Assessment, 2009
This paper addresses measurement and conceptual issues related to the realism of people’s confide... more This paper addresses measurement and conceptual issues related to the realism of people’s confidence judgments about their own cognitive abilities. We employed three cognitive tests: listening and reading subtests from the Test of English as a Foreign Language (TOEFL iBT) and a synonyms vocabulary test. The sample consisted of community college students. Our results show that the participants tend to be overconfident about their cognitive abilities on most tasks, representing poor realism. Significant group differences were noted with respect to gender and race/ethnicity: female and European American participants showed smaller levels of overconfidence than males and African Americans or Hispanics. We point out that there appear to be significant individual differences in the understanding of subjective probabilities, and these differences can influence the realism of confidence judgments.
We investigated the factorial structure of four major domains in social psychology (personality t... more We investigated the factorial structure of four major domains in social psychology (personality traits, social attitudes, values, and social norms) with an emphasis on cross-cultural differences. Three distinctive approachespancultural, multigroup, and multilevel-were applied to the data based on 22 measures that were collected from 2029 participants from 73 countries. First, in a pancultural approach, exploratory and confirmatory factor analyses were conducted on the entire sample of participants, disregarding country of origin. Second, in the multigroup (etic/emic) approach, nine societal clusters were fitted using a four-factor model. Several tests of invariance were applied to assess equivalence across the cultures. Finally, a multilevel approach was used to assess the structure at the individual-level and at the between-country (ecological) level. Our results show that the fourfactor solution emerges from the cultural domains, and this is supported by all three approaches. The factors are Personality/Social Attitudes, Values, Social Norms, and Conservatism. In the multilevel analysis, only two factors emerge at the ecological (between) level as opposed to four factors at the individual (within) level, but due to methodological difficulties, their status needs to be studied further. We discuss our findings in terms of the insideout view of social interactions.
In this article, nine world regions (based on samples from 33 nations) are compared in their perf... more In this article, nine world regions (based on samples from 33 nations) are compared in their performance on a cognitive ability test and confidence ratings obtained from the items of the same test. Our results indicate that differences between the world regions are greater on cognitive ability than they are on confidence ratings. Consequently, overconfidence—that is, the degree to which people overestimate their performance on cognitive tasks—is pronounced within the world regions that have lower scores on measures of cognitive ability. A less pronounced overconfidence is also present among the high-achieving world regions. Our findings support a cognitive hypothesis according to which individuals suffer from illusory superiority if the task is difficult. Thus, a commonly observed overconfidence can be seen as a self-deceiving, probably unconscious, mechanism that cushions a person (and countries) from experiencing negative feelings due to cognitive failures.
Uploads
Papers by Jihyun Lee