INFLUENCES ON STUDENT LEARNING

INFLUENCES ON STUDENT LEARNING John Hattie Inaugural Lecture: Professor of Education University of Auckland August 2, 1999 I welcome the opportunity to speak to my colleagues and friends on the research I wish to undertake during my next years here at the University of Auckland. This is my third Inaugural lecture since I became a Professor 15 years ago, hence I have learnt to take this opportunity seriously and cast a vision, as there are so few opportunities to talk to colleagues one’s own esoterica and passions. Maybe it is that I just need to do the same thing three times to at least say something, and I know my family has therefore renamed these my ‘Ignorable Lectures’. I do know that the fact most remembered from my first was that it was the first time that the chair Michael Scriven and the presenter presented an Inaugural and both did not wear ties. The fact from the second in North Carolina was that it was scheduled at the same time as the Simpson jury read its verdict. We know that students in lectures learn most in the first 8 minutes, only recall three things at most after one hour, and that if the content does not shake their prior beliefs they file away the fascinating facts in the deepest recesses of their brain, if at all. Hence, my aim tonight is to wax lyrical about my research projects, to build a theme around my measurement issues that have impact on students, to explain that about which I have much passion, and to give you three fascinating facts. Too much is known There is so much known about what makes a difference in the classroom. A glance at the journals on the shelves of most libraries, my colleagues’ shelves, and on web pages would indicate that the state of knowledge in the discipline of education is healthy. The worldwide picture certainly is one of plenty. We now can have a library solely consisting of Handbooks, most of which can not be held in the hand. Given this bounty, it would seem remarkable that there is much left to say. That never stopped an academic. There are at least three things to say: • We constantly bemoan the lack of integrative reviews and typically blame methodology; • The methodological war hinders us seeing the stories in each others work; • We have few integrative, bold and verifiable models of student learning, preferring to study one variable at a time, occasionally investigating interactive effects, too often look at classrooms through unfocused eyes, and rarely combine the texture of the learning Inaugural August 2, 1999 Page 1 experience with the rigour of asking whether there is an impact on student learning. This lack has not stopped us studying our pet beliefs. I must note that this love of pet belief is matched in classrooms, and herein are the three truths of research in classrooms: Teachers/Researchers have models of learning that are rarely externally elaborated or asked for 2. Teachers/Researchers seek evidence to buttress their models of learning and thus rarely seek to refute them or introduce major changes 3. There are no bad Teachers/Researchers. 1. We all seek positive evidence in that which we love. Teachers/Researchers, like lovers, are often blind. Hence, we have a school community peopled with teachers with self-fulfilling prophecies, all believing they are doing a good job, and with models of learning rarely based on any other evidence than "it works for me". As well we have an educational research community peopled with academics chasing their pet theory, promoting their own methodology while passing each other in corridors, rarely asking for negative evidence, and pushing with passion that "if only the teachers would do this, or know that". Both educational communities work behind closed doors, coming out to discuss kids, curricula, accountability, and each other, but rarely discussing the fundamental tenets about their teaching that leads to positive impacts on student learning. I believe the first step to resolve this dilemma is basic, simple, and powerful, and has taken me 10 years to build, and am still going. My three principles are: • • • We need to make relative statements about what impacts on student work We need estimates of magnitude as well as statistical significance – it is not good enough to say that this works because lots of people use it etc., but that this works because of the magnitude of impact We need to be building a model based on these relative magnitudes of effects. A model of effects on school and teaching A model of teaching and learning is proposed that is based on three postulates. In this session I wish to outline the model and the kinds of data that are being used to support it. • • That achievement is enhanced to the degree that students and teachers set and communicate appropriate, specific and challenging goals That achievement is enhanced as a function of feedback. Inaugural August 2, 1999 Page 2 • That increases in student learning involves more than surface and deep learning but also follows a reconceptualisation of information. A model of effects on school and teaching • • • Achievement is enhanced to the degree that students and teachers set and communicate appropriate, specific and challenging goals Achievement is enhanced as a function of feedback Increases in student learning involves not only surface and deep learning but also a reconceptualisation of information. Let me defend these simple propositions. MEASURING THE EFFECTS OF SCHOOLING A major purpose of this Inaugural is to show how the data from the past 30 years of educational research can be used to assess the effects of innovations and schooling, to provide insights for future innovation, and provide reasons why different advocates make apparently contrary claims about the effectiveness of schooling (even using the same data) by demonstrating how different points of comparison are used by each group. The beginning of the answer to the question as to the effects of schooling is to ask, "What are the 'typical' effects of schooling?" and then to use this typical effect as a benchmark. The problem is how to ascertain "typical effects" given the myriad of effects on schools, different teachers, subjects, school administration systems, ages of students, and other moderators such as gender, prior ability, quality of instruction and teaching styles. The first requirement is a continuum on which the effects of schooling, including the typical effect, can be summarised where 0 means that there is no effect from introducing some teaching package, innovation, or effect on schooling. A negative effect indicates that the innovation has a decreased effect on achievement, and a positive effect indicates that the innovation has as increased effect on achievement. For the present, the model is constrained to achievement outcomes, but the continuum can be generalised to other outcomes of schooling, and I have also undertaken a similar continuum for special education students. The next requirement is to formulate an appropriate scale and it is recommended that the scale is expressed in effect-sizes. An effect-size provides a common expression of the magnitude of study outcomes for all types of outcome variables, such as school achievement. An effect-size of 1.0 indicates an increase of one standard deviation, typically associated with advancing children's achievement by one year, improving the rate of learning by 50%, or a correlation between some variable (e.g., amount of homework) and achievement of approximately .50. When implementing a new program, an effect-size of 1.0 would mean that approximately 95% of outcomes positively enhance achievement, or average students receiving that treatment would exceed 84% of students not receiving that treatment. Cohen (1977) argued that an effect-size of 1.0 would be regarded as large, blatantly Inaugural August 2, 1999 Page 3 obvious, grossly perceptible, and he provided examples such as the difference between mean IQ of PhD graduates and high school students (we hope). The use of effect-sizes highlights the importance of the magnitude of differences, which is contrary to the usual emphasis on statistical significance. Cohen (1990) has commented that "under the sway of the Fisherian scheme (or dependence on statistical significance), there has been little consciousness of how big things are. ... science is inevitably about magnitudes" and the use of effect-sizes makes a welcome force towards the cumulation of knowledge" (p. 1310). For example, it was possible to locate 31 meta-analyses, 17952 studies, and 352 effect-sizes studies that investigated the effects of introducing computers on students' achievement (see Hattie, 1986). Using meta-analysis, these effects can be statistically synthesised to ascertain an overall effect as well as assessing the influence of differing groups of students (e.g., males versus females), different uses of computers, subject areas, and so on. The average effect-size across these 557 studies was .31. Thus, compared to classes without computers, the use of computers was associated with advancing children's achievement by approximately three months, improving the rate of learning by 15%, about 65% of the effects were positive (that is, improved achievement), thus 35% of the effects were zero or negative, and the average student achievement level after using computers exceeded 62% of the achievement levels of the students not using computers. An effect-size of .31 would not, according to Cohen (1977), be perceptible to the naked observational eye, and would be approximately equivalent to the difference between the height of a 5'11" and a 6'0" person. Of course, this is only an overall effect-size from introducing computers, although contrary to many beliefs the variability around these effects is quite small. There are many important moderators. For example, the effects decrease with age: primary students gain most (effect-size = .48), secondary students have medium gains (effect-size = .32), and college and university students gain least (effect-size = .25); there are differences in effect-sizes on achievement between males and females in secondary but not elementary classes (see Fitzgerald, Hattie, & Hughes, 1985; Hattie & Fitzgerald, 1987). Compared to not having computers in schools (effect=0) computing can help. Herein lie three major themes: • • • we can synthesis seemingly disparate studies to make an overall conclusion; diverse studies can be systematically integrated to address the question of the magnitude of effect; we can then determine those variables that may moderate this overall effect. There is a critical step missing here to build a defensible model of student learning. It will turn out that there are very few influences in education that are overall negative; hence it seems there is evidence for every teachers’ pet theme – and this is too close to reality. So to the missing step. Inaugural August 2, 1999 Page 4 The missing step is that the comparison should NOT be to zero point; we must not compare having computers to not having computers, we must not compare ourselves as teachers to not having us, but we must compare innovations to other innovations. I suggest to you that the synthesis of many studies can provide the normative effect to which other can be compared. Over the past 10 years I have been accumulating studies, and now have 337 meta-analyses, 200,000 effect-sizes from 180,000 studies, representing approximately 50+ million students, and covering almost all methods of innovation. The key question is "What is the typical effect of schooling"? The answer is derived from averaging the effects across the 357 meta-analyses and is .40 (with a se= .05). Most innovations that are introduced in schools improve achievement by about .4 of a standard deviation. This is the benchmark figure and provides a "standard" from which to judge effects. A comparison based on typical, realworld effects rather than based on the strongest cause possible, or with the weakest cause imaginable. At minimum, this continuum provides a method for measuring the effects of schooling. The typical effect does not mean that merely placing a teacher in front of a class would lead to an improvement of .4 standard deviations. Some deliberate attempt to change, improve, plan, modify, or innovate is involved. The best available estimate as to the effects of schooling not based on innovations is from the National Assessment of Educational Progress (NAEP) data bank (Johnson & Zwick, 1990). NAEP surveyed what students in American schools knew and could do in the subject areas of reading, writing, civics, US history, mathematics and science. The students were sampled at ages 9, 13 and 17, and the testing has been repeated every two years. The average effect-size across the six subject areas was .24 per year, which Inaugural August 2, 1999 Page 5 indicates that the effects of innovations is (.40-.24 =) .16 standard deviations above and beyond the teacher effects. A further contention of many researchers is that maturation alone can account for much of the enhancement of learning. The effect of maturation is probably about one-third of the achievement effect (.10; see Cahen & Davis, 1987). Schooling does enhance learning above the influences of maturation. Hence my three normative comparison points: Normative comparison points • • • student maturation a teacher in front of a classroom innovations in schooling .10 .24 .40 The obvious – schooling makes a difference – contrary to the Coleman study, and possible contrary to the Harris argument about the power of peers over parents and teachers – although I keenly await the results of my colleagues Ian W, Mike T, Judy P, Charlotte and Irene and others who are working on the peer influences literature review as part of a grant, at present. COMPUTERS. Let me return to computers and now compare their effects with other influences — then they are just in there. So, out with the computer salespeople who push computers as the magic answer — yes, compared to not having them they have an influence, but compared to other influences a different conclusion can be made — there are not so influential. Let me show you an example of the innovations that are below the typical effect. OVERALL EFFECTS No. of Effects 165,258 Peers Advance organizers Simulation & games Computer-assisted instruction Instructional media Testing Aims & policy of the school Affective attributes of students Calculators Physical attributes of students Learning hierarchies Ability grouping Programmed instruction Audio-visual aids Individualisation Inaugural 122 387 111 566 4421 1817 542 355 231 905 24 3385 220 6060 630 August 2, 1999 Effect-Size .40 .38 .37 .34 .31 .30 .30 .24 .24 .24 .21 .19 .18 .18 .16 .14 Page 6 Finances/money Behavioural objectives Team teaching Physical attributes of the school Mass media Retention No. of Effects 658 111 41 1850 274 861 Effect-Size .12 .12 .06 -.05 -.12 -.15 RETENTION The effect is among the very lowest of many possible innovations and it can be vividly noted that retention is overwhelmingly disastrous across many educational interventions at enhancing academic achievement. This topic is fascinating to me. Last year I was an expert witness for the NAACP in a Federal Court Case arguing that retention had a particularly invidious impact on African American children, I have been awe-stuck at the number of teachers who fundamentally believe retention is justified ("if you could only give me more time"), and reiterate to you that these defenders compare the potential impact of holding the student back a year because of a belief that the difference between the start of the grade and end of the grade could increase not appreciating that the comparison is the start of the grade with the performance of similar students who are now tested at a higher grade. The groups of non-promoted/retained students scored .15 to .26 standard deviation units lower than the promoted comparison groups on the various outcome measures: • • • • After one year the retained groups were scoring .45 standard deviation unit lower than the comparison groups who had gone on to the next grade and in many cases were being tested on more advanced material. Each subsequent year this difference became larger with the difference reaching .83 standard deviation unit for measures taken four or more years after the time of retention. Being retained one year almost doubled a student’s likelihood of dropping out, while failing twice almost guaranteed it. The negative effects are pervasive over all academic and personal educational outcomes, and at all ages (including kindergarten) There is a consistently negative picture of the association between retention and race, gender, SES, and school outcomes. Those who continue to retain pupils at grade level do so despite cumulative research evidence showing that the potential for negative effects consistently outweighs positive outcomes. It would be difficult to find another educational practice on which the evidence is so unequivocally negative (House, 1989). Inaugural August 2, 1999 Page 7 CLASS SIZE The research on the effects of class size has been among the more voluminous in educational research with very systematic findings: • • • Achievement, attitude, teacher morale, student satisfaction gains are appreciable in smaller classes, so long as we recognise that small classes mean 10-15, as there are negligible gains between 40 to 20 students per class. This effect was the same for primary and secondary schools, across all subjects, and across various ability levels. There is little evidence that instruction methods change when class size is reduced, although a large part of improvement can be explained by improvements in student task engagement. Reducing class sizes from the 30’s to the 20’s is in the right direction, but there is little support for the claim that there are increases in student achievement or satisfaction, or teacher attitude or morale. Only when the class size reduces to 15 or below are there appreciable positive benefits. TELEVISION This is one of the few non-linear effects. Less than 10 hours per week is associated with increases in achievement, more than 10 hours with decreases – but show me a New Zealand child who watches less than 10 hours per week. And this table provides examples of influences above the average. OVERALL EFFECTS No. of Effects 165,258 Reinforcement Students prior cognitive ability Instructional quality Instructional quantity Direct instruction Acceleration Home factors Remediation/feedback Students disposition to learn Class environment Challenge of Goals Bilingual programs Peer tutoring Mastery learning Teacher in-service education Parent involvement Homework Questioning Inaugural 139 896 22 80 253 162 728 146 93 921 2703 285 125 104 3912 339 110 134 August 2, 1999 Effect-Size .40 1.13 1.04 1.00 .84 .82 .72 .67 .65 .61 .56 .52 .51 .50 .50 .49 .46 .43 .41 Page 8 Five overall findings from these positive effects: • • • • • critical innovations feedback setting of appropriate, specific and challenging goals It is what some teachers do that makes the difference; and The introduction of most teaching and school influences merely impacts on the probability of the presence of feedback and challenging goals. First, innovation is the theme underlying most of these effects. That is, a constant and deliberate attempt to improve the quality of learning on behalf of the system, principal and teacher typically relates to improved achievement. The implementation of innovations probably captures the enthusiasm of the teacher implementing the innovation and the excitement of the students attempting something innovative. Often this has been explained as an experimental artifact in terms of a Hawthorne effect. No matter the reason, it appears that innovation per se can have positive effects on students’ achievement. Teachers who constantly question "How are I going", who wish to verify that their methods are having impacts on student learning are the prerequisites for excellence. I highlight this innovation and willingness to try new methods particularly as too many classes involve listening, mimicking, copying, and learning how to survive the information overload. Second, the most powerful single moderator that enhances achievement is feedback. The simplest prescription for improving education must be "dollops of feedback" -- providing information how and why the child understands and misunderstands, and what directions the student must take to improve. The most fundamental component of teaching is imparting information to students, assessing and evaluating the students understanding of this information, and then matching the next teaching act to the present understandings of the student. This is not feedback in the behavioural sense of input/output models, but the understanding of the constructions that student have made from the information. Feedback is the information component whereas reinforcement is the evaluative component relating to information and motivation. For example: • • • • • Inaugural reinforcement corrective feedback remediation and feedback diagnosis feedback mastery learning (which is based on feedback) August 2, 1999 1.13 .94 .65 .52 .50 Page 9 Although not all forms of feedback are as effective, for example: • • • extrinsic rewards immediate vs. delayed punishment .37 .28 .20 There is a contrast here in the old fashioned notions of intrinsic and extrinsic rewards, although the better jargon now is task vs. ego involvement. a. There are many ways in which teachers can deliver feedback to students, and for students to receive feedback from teachers, peers and other sources. The implication is NOT that we should automatically use many tests and provide over-prescriptive directions. Rather, it means providing information how and why the child understands and misunderstands, and what directions the student must take to improve. Tests can perform this function but too often they are devoid of much feedback to the students and thus can be an inefficient method of providing feedback. b. The incidence of feedback in the typical classroom is very low, usually in seconds at best per day. Ask yourself how much feedback there is in the typical lecture – yes, imparting information is there, but how do you know it is appropriately challenging, that the student shares or even understands the specific goals, and how do you provide appropriate information about what directions the student must take to improve their understanding. Perhaps it is no surprise to note that students who survive university undergraduate degrees are those with impeccable surface and not deep strategies, those who learn to be flexible to the instructors assessment demands which so often value information and not understanding, and that the students with the more deep, critical, and passionate learning strategies have the highest probability of not completing our degrees. c. It is likely that the information function of feedback for an individual may be too diluted in a regular classroom to afford much control over behaviour for special education students. d. Reducing class size, prescribing more homework, introducing more computers, etc. merely offers increased opportunities for more feedback and appropriately challenging goals to occur - it does not guarantee it occurs. e. It is predicted that programs that do not capitalise on effective classroom management practices to optimise feedback will not be successful. One example is individualised instruction. Individualisation programs produce an average effect-size of .14, and programmed instruction yields .18. Too often, individualisation means placing the child alone to work on a particular task, usually relating to his or her particular needs, progress, pace, and behaviour. These attempts usually have little feedback, little attention by a busy teacher catering to the other 30 or so students, and the student typically has little knowledge of success or failure at the specifics of the task. Teachers need excellent management systems with classes greater than 20, and superb management systems with classes greater than 30. To introduce individualisation with 30 students means that, at best, a Inaugural August 2, 1999 Page 10 single student can only receive seconds of a teachers’ time per day, minutes of a teachers’ time in preparation, and merely passing glimpses of time in feedback - no matter how noble the intentions of the teacher. It is neither efficient nor effective to individualise instruction for 30 students, expect high competence in the management of such programs, and provide feedback to all students. Individualisation in regular classes must fail and does fail. The negligible effects of individualisation are particularly important when it is recognised that students spend about 66 percent of their time working alone (Rosenshine, 1979). Feedback effects on performance are primarily via cues that direct attention to the task-motivation processes, particularly when coupled with information regarding erroneous hypothesis. The fewer the cognitive resources needed for a task and the more the feedback is directed at the task demands and not the person then the more powerful the feedback. If we, as teachers, are to have an impact on learning, then we must come to know what our students are thinking so that we can provide more feedback, task information, encourage trial and error, and develop deep understanding and transformations. Third, achievement is enhanced to the degree that students and teachers set challenging rather than "do your best" goals relative to the students' present competencies. Goals need to be specific, challenging, and it helps if the students are committed to these goals -- although it is the challenge that is most critical (Locke & Latham,1992). Never allow a student to "do their best" as this is the goal with the least challenge; everything the student does can be claimed as the best! Appropriate, challenging, and specific goals inform individuals "as to what type or level of performance is to be attained so that they can direct and evaluate their actions and efforts accordingly. Feedback allows them to set reasonable goals and to track their performance in relation to their goals so that adjustments in effort, direction, and even strategy can be made as needed" (Locke & Latham). • • • • • • They direct attention to relevant tasks or outcomes They energise task performance They motivate individuals to persist in their activities through time They convey normative information by suggesting or specifying what level of performance the student could be expected to attain They have dramatic effects on the development of self-efficacy, which in turn affects the choice of difficulty of goals Feedback without goal setting is less effective, and goal setting without feedback is ineffective. A combination of goal setting plus feedback is most effective = goals and challenging goals are mutually supportive. The greater the Inaugural August 2, 1999 Page 11 challenge the higher the probability of the student seeking, receiving, and assimilating feedback information. The scenario is that effective teachers set challenging goals and then structure situations so that students can reach these goals. If teachers can encourage students to share commitment to these challenging goals, and if they provide much feedback, then goals are more likely to be attained. A good example is the effectiveness of Outward Bound programs to enhance self-esteem. Of the many self-esteem programs, one of the most effective are the Outward Bound programs (Hattie, 1992). In these programs, a common feature is the setting of seemingly very difficult goals (e.g., abseiling), and then structuring the environment so that students can attain these goals, while providing much informative feedback. Further, the instructors remove many of the possibly irrelevant tasks so that students focus on these challenging goals. Fourth, it is teachers that make the difference. Let me go back to the big picture, and re-order the effects into those that are school policy and structural effects, social, home, what the student brings, teacher methods, and learning process related. The bold are those effects greater than the average. It is clear that structural and social influences are minor, what the student brings in terms of achievement and disposition to learn are powerful, teaching process are paramount, and the teacher methods are there – this must lead to the conclusion that, yes teachers make the difference, but only teachers who teach in certain ways. Too many teachers compare what they are doing with them not being there; they compare their methods with not using that method; whereas teachers need to be more informed evaluators/consumers of teaching methods. Like their students they must set challenging goals, seek feedback on the effectiveness of their teaching on students, and constantly be attention to improvement and innovating the methods which optimise feedback and meeting challenging goals. Influence Teacher process influence Reinforcement Instructional quality Remediation/feedback Challenge of Goals Teacher methods Direct instruction Class environment Peer tutoring Mastery learning Homework Teacher Style Questioning Inaugural August 2, 1999 Effect-Size 1.13 1.00 .65 .52 .82 .56 .50 .50 .43 .42 .41 Page 12 Influence Effect-Size Advance organisers Simulation & games Computer-assisted instruction Instructional media Testing Programmed instruction Audio-visual aids Individualisation Behavioural objectives Team teaching Student influences Students prior cognitive ability Students disposition to learn Affective attributes of students Physical attributes of students Home influences Home factors Parent involvement Social influences Peer Television School Policy Influences Aims & policy of the school Ability grouping Finances/money Physical attributes of the school Retention .37 .34 .31 .30 .30 .18 .16 .14 .12 .06 1.04 .61 .24 .21 .67 .46 .38 -.12 .24 .18 .12 -.05 -.15 Fifth, the major argument is that most of the structural innovations in schooling are probabilistic – their introduction merely alter the probabilities of the core effects occurring: these core effects include feedback, appropriate and challenging goals, and reconceptualisations as well as surface and deep knowledge. Hence the five themes: 1. Innovation is the theme underlying most of these effects; 2. The most powerful single moderator that enhances achievement is feedback; 3. The setting of appropriate, specific and challenging goals is critical; 4. It is what some teachers do that makes the difference; and 5. The introduction of most teaching and school influences merely impacts on the probability of the presence of feedback and challenging goals. Inaugural August 2, 1999 Page 13 Teachers must be more informed evaluators as they need to ask how their methods increase the probability of that which makes the difference – information, reconceptualisation, feedback, and appropriate goals. I must note that • • • • • These findings along the continuum have remarkable generality, across subjects, and ages. Generality is the norm. Aptitude treatment interactions are noted by their absence. God must love main effects. Attempt to individualise instruction are not noted by success. Individualisation programs produce an average effect-size of .14, and programmed instruction yields .18. The negligible effects of individualisation are particularly important when it is recognised that students spend about 2/3rds of their class time working alone. These studies have been undertaken primarily on children in western countries. Academic achievement in low-income countries is affected more by pupils' social status and less by school and teacher quality, whereas the converse is the case in high-income countries like New Zealand. I have expanded this meta-analysis from achievement to attitudes and a separate table for special education teachers – with similar conclusions. Thus, I hope I have • • • Demonstrated the power of meta-analysis, and research synthesis to address the more critical questions in education – I have spent much research time synthesising these studies, finding major gaps and then filling it with my own meta-analyses – for example on study skills, outof-school experiences, self-concept, and my current one is on the effects of ADHD/ADD medication on learning Demonstrated the power of assessing magnitude rather than statistical significance – which is one of my fascinations. And identified that which makes the difference on student learning. So far the prescriptions for influencing student learning are clear: dollops of feedback, specific and challenging goals, and a constant attention to asking, "How am I going". There is at least one major complicating factor. The inherent nature of learning is that there is a gap between the feedback and the attainment of goals. There are four possible ways for students to reduce this gap: • • Students can increase their effort, although the evidence we find of a "just-world" that more time in the class or more time studying increases learning is not convincing. Students can abandon the standards, and they are doing this in droves. We are successfully alienating so many students from our Inaugural August 2, 1999 Page 14 • schools, particularly high schools as we promulgate the merits of our subject cramming and chauvinism, while the students look to their older peers who are successful in their eyes without that cramming. We say students must have this knowledge about English when most of the adult population does not need it; we say students must have this advanced math when their fellow teachers do not have it and are successful. Students become not engaged, and do not wish to have their reputations based on attaining our social conformists goals of high school-based achievement; Students can change the standard by setting lower goals in achievement at school, accepting far below their capabilities as satisfactory. More often these drop out effects occur because students reject or reinterpret the feedback information. It is this latter reaction that has so driven my recent research program. Students too often have conceptions of learning inbred by years of shallow, fact-pushing, routine-ised teachers who pride themselves on presenting the very best content, teaching from the front, sitting tests, prodding students by external cues such as exams, and valuing themselves in terms of successful imparting knowledge usually via teaching models akin to drips into empty vessels. Students, however, are not inert recipients and build strategies to deal with this daily grind of the knowledge dump. Achievement is enhanced to the degree that students develop selfstrategies: to seek and receive feedback to verify rather than enhance their sense of achievement efficacy. Although I have spent many years exploring self-concept, this is NOT what I am talking about. We have demonstrated there is a very low relationship between self-concept and achievement (less then 4% of the variance), hence asking causal questions is misplaced. We know much about the multidimensionality of self-concept, and the age and gender effects etc. but this has rarely helped us in our understanding of learning. More recently I have moved towards asking about the relation between learning processes and self-strategies and conceptions of learning. I do wish to emphasise that the self-strategies that students have directly leads to enhanced or decreased performance AND their conceptions of learning they have can lead them to (mis-) interpret the often excellent teaching they receive – although I do not intend to purse the conceptions of learning tonight. There are two major self-strategies that students used when learning: Self status quo strategies • the wish to be viewed as one believes one is; • to do whatever it takes to preserve this concept of self Inaugural August 2, 1999 Page 15 by adopting strategies that maximise positive or minimise negative selfevaluations Self testing strategies • the seeking of confirmation and/or disconfirmation about conceptions of self • by adopting hypothesis-confirming strategies that allow for the best opportunities for self-expression. It is not so simple that we can match the use of these strategies with high and low self-esteem, as all individuals prefer self-verification but low self-esteem students have a greater tendency to use self-enhancing strategies: • • • When students are using self-verification strategies, initial success signifies a talent or potential ability, whereas for self-enhancement initial success confirms a deficiency needed to be remedied. Students using self-verification strategies cope better with disconfirmation as they can relate to positive and negative verifications of self. As a consequence of disconfirmation, those who use selfverification make more optimistic predictions about their performance after initial failure than after initial success, they seek specifically unfavourable feedback so as to excel at the tasks. Students using self-enhancement show moderately high persistence at the task after failure, consistent with the view that they are interested in remedying their deficiencies in order to reach a passable level of performance, which would afford them protection against humiliating failure. Further, they tend to avoid tasks following initial success as such success signifies that they have already reached an adequate level of performance, and further tests merely run the risk of disconfirming the favourable outcome. This can lead to rejection of success, not because they prefer failure but because success is disconfirming, may require alterations to expectations, or maybe be threatening because "they may lack confidence that they can repeat that success". Yes, feedback is powerful, but the self-strategies that students develop can alter the interpretation and consequences of this feedback. Students selfenhance by biasing information and by selecting information that provides affirmation of their prior beliefs. Providing feedback to students is not enough - as the ways and manner in which individuals interpret your information is the key to developing positive and valuable concepts of self. As has been demonstrated by many researchers, including my own, this is where cultural factors have a major mediating effect in the teaching-learning process. We have undertaken these studies in Australia, Hong Kong, Japan, the USA, and with tertiary, secondary, primary students, and more recently with groups of males and groups of female juvenile delinquents and teenage prisoners. Understanding their conceptions of learning and the self-strategies they have, and how they use these to enhance their reputations is most powerful. I would conject that there would be powerful self-strategies by many Inaugural August 2, 1999 Page 16 Maori and Pacifica students that mediate the reception and assimilating of feedback offered by Pakeha teachers. We have been working with nine major strategies. Nine major strategies 1. 2. 3. 4. 5. 6. 7. 8. 9. Self-handicapping Discounting Social comparison Disconfirmation Setting less challenging goals Setting performance rather than task goals Self-monitoring Confirming negative cultural stereotypes Seeking negative information Self-handicapping providing a handicap that can be used as an explanation for maintaining beliefs, and accounting for success or failure that is inconsistent with prior beliefs. Example: a student could claim he or she scored 100% on an examination because the items were too easy rather than because of ability or effort in learning; or scored lower because of the excellent television program they watched instead of studying. Self-handicapping occurs when: • students have high uncertainty about their competencies • there is high salience of an evaluative task • in public rather than private performance situations • there is an abnormal investment in the question of self-worth • when the students believe that the handicap will be viewed by others as a legitimate reason for potential failure It is used by both high and low self-esteem individuals: high self-esteem people to enhance success, low self-esteem people to protect themselves against the threat of failure. Discounting whereby feedback is "dismissed" as being information that is not valuable, accurate, or worthwhile for the individual. Example: when a teacher tells a student that he or she is doing a great job, and the student’s reaction is to discount this by claiming "she always says that," "she’s only trying to make me feel good," or "it’s only because it is neat, not correct". Social comparison whereby low self-esteem individuals constantly monitor other peers’ behavior for cues and attributions to explain/enhance their conceptions of self. They compare themselves with others, and social comparison sets standards or frames of reference. Inaugural August 2, 1999 Page 17 Example: Students often compare themselves to those less fortunate than themselves, and often attempt to present themselves as more confident to impress others and maybe even themselves. Disconfirmation whereby students ignore or reject information that disconfirms their self beliefs. Less challenging goals whereby students set less challenging goals and thus ensure more success and confirmation about their learning. Setting performance rather than task goals. Example, students who set performance goals are more concerned with gaining favourable judgements of their competence, whereas task goals are more concerned with increasing their competence on the task. Self-monitoring whereby students actively plan, enact and guide their behavioural choices in social situations through the process of selfmonitoring. High self-monitors are more dictated to by the external environment and by social comparison. Confirming negative cultural stereotypes whereby students absorb negative societal stereotype about their group’s intellectual ability and competence. Steele (1992; Steele & Aronson, 1995), for example, has argued that whenever African American students (particularly males) perform an explicitly scholastic or intellectual task, they face the threat of confirming or being judged by a negative societal stereotype about their group’s intellectual ability and competence. Such a reputation influences the academic functioning of these students, particularly during standardised testing. He claimed that this reputation "may have the further effect of pressuring these students to protectively disidentify with achievement in school" (p. 797), such that school achievement is neither a basis of self-evaluation nor a personal identity. There are various effects of this cultural reputation (such as spending more time answering fewer test items) that can undermine motivation, effort, and self-efficacy. Seeking on negative information. Although all students prefer favourable feedback some have adverse stress reactions to negative feedback – hence try to avoid seeking negative information altogether and seek only positive information. The effects of the strategies These self learning strategies explain how individuals can bias, select and retain information that affect their self-concepts and they have major influences on how students learn, how they set challenging goals, how they accept feedback, and their subsequent learning outcomes. The point is that a teacher may be providing a remarkable amount of feedback but this does not mean that this student is receiving this feedback. Inaugural August 2, 1999 Page 18 Excellent teaching requires teachers setting challenging goals and providing feedback, and students setting challenging goals, and seeking, biasing and interpreting information and accepting feedback. Oh, not so easy, eh! If only it was one-way. Teacher Education The implications of the above have much to say to teacher education. Teachers need to be well prepared in their content to know WHAT information is to be taught, and WHAT information can be used as feedback – clearly if they have deeper understandings of their content they increase the probability they can set challenging goals and provide feedback. More important teachers need to understand that the variations in individual students manner of receiving, transforming and being disposed to this information is considered in the act of teaching. This, I believe, requires that teachers care about their students, know where they are coming from, and overly attend to the issue of whether their students are learning. For example, we have been researching the Paideia teaching method, which was forcibly introduced into every class in 91 schools. Yes, it was innovation, yes it was a teacher method and not a structural change, and yes, it involved setting challenging goals and dramatic amounts of feedback – all the hallmarks for success; but forcibly introduced. The Paideia program promotes three modes of teaching: a didactic mode, a seminar component, and a coaching aspect. The seminar component is the most distinctive and involves much training in active listening by the teacher, constructing peer group tutoring for the whole class that uses rich and challenging texts (across all the curricula), and structures ways for the students to question the text and each others understandings. I used Paideia in my classes in measurement and statistics and it opened up to me, as teacher, the ways in which the students were constructing knowledge, and the ways they had already constructed it – very sobering – despite my having taught the material using the drip filter model perfectly. In the forced implementation there was remarkable success in the standardised test scores across all ages, subjects, and most critically in this particular county for all racial backgrounds, primarily by reducing the negative effects of social comparison that African American students had been using. There was also powerful evidence that Paideia influenced students’ perceptions of the quality of schooling. The students claimed that teachers who implemented Paideia were better at explaining information, more able in ensuring that students had a good understanding, put more effort into teaching, taught in interesting ways, and showed by example that learning is fun. There was less friction in classes, less fooling around, students were considered more calm and not mean, and they felt safe. Students see more flexibility in the classroom when Paideia is implemented. Teacher Education programs must produce teachers with a conception of teaching based on a high and rigorous set of standards of what beginning Inaugural August 2, 1999 Page 19 teachers know and are able to do. I have worked with NCATE developing a new model for teacher accreditation based on this simple notion. Instead of asking institutions, like ours here at UoA, what courses do your students experience, and are the staff qualified. We oversaw: • • • a major consultative process to determine the high and rigorous competencies of graduating teachers in terms of what they know, and are able to do asked the Institutions to provide evidence that their graduating teachers know and able to do the competencies to the appropriate standards of excellence, and ask the professionals to adjudge this evidence. We are in trials in many Institutions in the US at the moment and this is beginning to have a profound and positive effect on what is undertaken in teacher education institutions. No longer is exposure to wisdom sufficient, no longer are scores on our student sufficient, no longer are our posturing that we are quality providers convincing. Instead, the course is evaluated by the quality of the graduates in terms of knowing, performing and their dispositions in terms of their impact on their students’ learning outcomes– a hard but correct ask. We spent much time in teacher education programs imparting the necessary knowledge so that students are sufficiently learn to teach, we spent much time on teaching methods, and we need to spend much time understanding how students learn, their conceptions of learning, and how the feedback cycle works. Too often we compared experienced with novice when the better comparison is experienced and expert. This is my mission. Identifying Accomplished Teachers We have a system that spends inordinate energies worrying about initial teacher training when more attention needs to be placed at the highest ends of excellence. We have a model of teaching which assumes that experience is sufficient, we pride well run quiet classrooms, we pride mimicking, listening, and regurgitating information. I have argued extensively elsewhere that New Zealand needs a National Board for Professional Teaching Standards. I propose the development of an Australasian Board for Professional Education Standards (ABPES) that parallels the USA National Board (NBPTS) to advance the excellence of the principal and teaching profession. The mission would be to: • • • Establish high and rigorous standards for what accomplished teachers and principals should know and be able to do, Operate a national voluntary system to assess and certify teachers and principals who meet these standards, and To advance related education reforms for the purpose of improving student learning in Australasian schools. Inaugural August 2, 1999 Page 20 This has been done in the USA and I am still involved in projects relating to the National Board. Currently, I have a $.75m project on how NBCT and nonNBCT teachers manifest their expertise in the classroom, and not surprisingly given tonight, are finding differences relating to the amount of feedback, the manner the teacher understands the learner and how he or she is learning, deals with the connections of learning and strategies of learning, and cares and respects what the child brings to their classroom. These facets are often missing in the too-typical over-routines classroom where teaching is undertaken as knowledge forced into the recalcitrant brains, assembled in straight rows, silent, listening and waiting to be tested. A major imperative why we must identify, reward, and promote these excellent teachers in a fair, credible and dependable manner is that they are most likely to provide the voting public the confidence in a national system of public schooling. National testing, braying by parents and politicians about low quality and more accountability, and the highlighting of the minimally competent reinforces those 80% of New Zealanders who typically argue that the quality of public schooling in New Zealand is awful; whereas the identification of excellence reinforces those 80% of New Zealanders who believe that the quality of their child’s teacher is great. We must promote excellence. Research Program To conclude I wish to outline a series of research study that I and my colleagues (and I hope colleagues here in Auckland) have underway: What influences student learning • A model of teaching and learning based on a synthesis of metaanalyses From the student perspective • Defending the power of feedback, and goals (in classrooms, out-ofclassrooms such as Outward Bound, Adventure Sailing ships, and with at-risk students) • Developing models of conceptions of learning of teachers and students and how these affect how teachers teach and students learn • Discovering self strategies for learning (particularly cross-culturally) • Building models of conceptions of self particularly based on reputation enhancement • Ascertaining the study skills of students that assist in the learning process From the teacher perspective • Determining the differences between experienced and expert teachers • Assessing the teaching methods that increase the probability of impact on student learning • Devising measurement procedures to address the question as to how expert and experienced teachers differ in classroom practice and how to identify highly accomplished teachers • Developing accreditation processes for teacher education programs Inaugural August 2, 1999 Page 21 • The development of a New Zealand National Board that sets high and rigorous standards and has excellent performance measures to identify accomplished teachers Addressing the desirable outcomes • Tests and measures of models of achievement, particularly surface and deep outcomes based on the SOLO model • Tests and measures of learning self-strategies • Training teachers in setting standards (e.g., using exemplars, scoring rubrics) • Developing models of wellness to add to the achievement outcomes Using the best statistical modelling • Conducting meta-analyses to achieve integration, perspective, and magnitude of effects • Incorporating measures of magnitude with statistical significance in research designs • Building structural modelling to assess explanations and prediction • Using item response models to assess dimensionality and create better measurement, particularly in the affective domain • Developing and conducting evaluation models of successful teaching and school innovations I now consider myself inaugurated. Selected Bibliography Carroll, A., Durkin, K., Hattie, J., & Houghton, S. (1997). Goal setting among adolescents: A comparison of delinquent, at-risk, and not at-risk youth. Journal of Educational Psychology, 89, 441-450. Carroll, A., Durkin, K., Houghton, S., & Hattie, J.A. (1996). An adaptation of Mak’s self-reported delinquency scale for Western Australian adolescents. Australian Journal of Psychology, 48, 1-7. Carroll, A., Durkin, K., Houghton, S., & Hattie, J.A. (1996). An adaptation of Mak’s self-reported delinquency scale for Western Australian adolescents. Australian Journal of Psychology, 48, 1-7. Carroll, A., Hattie, J.A., Douglas, G. (in press). Adolescent reputation enhancement: Differentiating delinquent and nondelinquent youths. European Journal of Social Psychology. Carroll, A.M., Hattie, J.A., Houghton, S., & Durkin, K. (in press). Reputation Enhancing Goals: Integrating reputation enhancement and goal setting theory as an explanation of delinquent involvement. Cheung, D., Hattie, J.A., Bucat, R., & Douglas, G. (1996). Students’ perceptions of curriculum implementation. Curriculum and Teaching, 11, 49-60. Cheung, D., Hattie, J.A., Bucat, R., & Douglas, G. (1996). Measuring the degree of implementation of school-based assessment schemes for practical science. Research in Science Education, 26, 375-389. Crane, D., Hattie, J.A., & Houghton, S. (1997). Goal setting and the Adventure Experience. Australian Journal of Psychology, 49, 6-13. Inaugural August 2, 1999 Page 22 Dimmock, C. & Hattie, J. (1992). Principals’ and teachers’ reactions to school restructuring: The case of Western Australia. Australian Journal of Education, 38, 38-57. Dimmock, C. & Hattie, J. (1994). Principals’ and teachers’ reactions to school restructuring. Australian Journal of Education, 38, 36-55. Dimmock, C., & Hattie, J. (1996). School principals’ self-efficacy and its measurement in a context of restructuring. School Effectiveness and School Improvement: An International Journal of Research, Policy, and Practice, 7, 62-75. Elliott, E., Hattie, J.A., et al. (1998). Program Standards for Elementary teacher preparation. National Council for Accreditation of Teacher Education, Washington, DC. Forlin, C., Douglas, G., Hattie, J. (1996). Inclusive practices: Is it stressful for teachers? International Journal of Disability, Development and Education, 21, 199-217. Fraser, B., Walberg, H., Welch, W., & Hattie, J.A. (1987). Syntheses of educational productivity research. Monograph in International Journal of Educational Research Series. Oxford: Pergamon. pp. 252. Hattie, J. (1992). The levels of school accountability. Education Research and Perspectives, 19, 78-80. Hattie, J. (1995). Rural schools and student outcomes. In Schooling in Rural WA: Report of the Ministerial Review of Schooling in Rural Western Australia. WA Government. Hattie, J.A. & Jaeger, R.M. (1998). Assessment and learning. Assessment in Higher Education, 5, 111-122. Hattie, J.A. & Marsh, H.W. (1995). Future research in self-concept. In B. Bracken (Ed.), Handbook on Self-concept (pp. 421-463). Erlbaum, NJ. Hattie, J.A. & Purdie, N. (1998). The SOLO method and item construction. In G. Boulton-Lewis & B. Dart (Eds.), Learning in Higher Education. Hawthorn, Australia: ACER. Hattie, J.A. (1992). Self-concept. Erlbaum, USA, pp. 304. Hattie, J.A. (1992). Towards a model of schooling: A synthesis of metaanalyses. Australian Journal of Education, 36, 5-13. Hattie, J.A. (1993). Measuring the effects of schooling. SET, 2, 1-4. Hattie, J.A. (1997). Setting standards for beginning teachers: A discussion paper. National Council for Accreditation of Teaching Standards, Washington DC. Hattie, J.A. Clinton, J.C., Nagle, B., Kelkor, V., Reid, W., Spence, K., Baker, W., & Jaeger, R. (1998). The first year evaluation of Paideia. Bryan Foundation and Guilford County Schools. Hattie, J.A., & Clinton, J.C. (in press). Identifying Accomplished Teachers: A Validation Study. In L. Invargasson, (Ed.), Assessing teachers for professional Certification: The first ten years of the National Board for Professional Teaching Standards. JAI Press. Hattie, J.A., & Haines, L. (1998). An Evaluation of the Reading Together, USA program. Greensboro, NC. Hattie, J.A., & Jaeger, R.M. (1998).The construct validity of NBPTS assessments and classifications of candidate teachers. NPEAT, Washington, DC. Hattie, J.A., & Purdie, N.M. (in press). Individual differences. International Encyclopedia of Educational Psychology. Inaugural August 2, 1999 Page 23 Hattie, J.A., Biggs, J., & Purdie, N. (1996). Effects of learning skills intervention on student learning: A meta-analysis. Review of Research in Education, 66, 99-136. Hattie, J.A., Fletcher, R., Karkee, T. (1996). An analysis of school suspensions in Guilford County Schools. Confidential report presented to the School Board, Guilford County, NC. Hattie, J.A., Jaeger, R.M., & Bond, L. (in press). Pervasive problems in educational measurement. Review of Research in Education. Hattie, J.A., Kefford, R., & Porter, P. (1988). Skills, Technology, and Management in Education: The Dynamics of Learning. (Eds.) Australian College of Education, Melbourne, 1988, Deakin, ACT. pp. 141. Hattie, J.A., Krakowski, K., Rogers, H.J., & Swaminathan, H. (1996). An assessment of Stout’s index of essential unidimensionality. Applied Psychological Measurement, 20, 1-14. Hattie, J.A., Marsh, H.W., Neill, J.T., & Richards, G.E. (1997). Adventure Education and outward Bound: Out-of-class experiences that make a lasting difference. Review of Research in Education, 66, 43-87. Hattie. J.A. (in press). Validating the specification of a complex content domain: Applications to the National Board for Professional teaching Standards’ Assessments. In L. Invargasson, (Ed.), Assessing teachers for professional Certification: The first ten years of the National Board for Professional Teaching Standards. JAI Press. Hoffman, R.M., Border, L.D., & Hattie, J.A. (in press). Gender self-confidence: An alternative approach to the measurement of masculinity and femininity. Sex Roles. Jaeger, R. M. & Hattie, J. A. (1996). Artifact and artifice in education policy analysis: It’s not all in the data. The School Administrator, 53, 24-29. Jaeger, R.M., & Hattie, J.A. (1995, November). Detracking America’s Schools: Should we really care? Phi Delta Kappan, 77(3), 218-219. Law, H.G., Snyder, C.W., Hattie, J.A. & McDonald, R.P. (1984). Multi-mode models for data analysis. Praeger, USA. pp. 687. Maguire, T.O.M. & Hattie, J.A., & Haig, B. (1994) Construct validity and achievement assessment. Alberta Journal of Educational Research, 40, 109-126. Marsh, H.W., & Hattie, J.A. (1995). Theoretical models in self-concept. In B. Bracken (Ed.), Handbook on Self-concept. (pp. 38-92). Erlbaum, NJ. Moriarty, B., Punch, K., & Hattie, J. (1995). The importance of self-efficacy as a mediating variable between learning environments and achievement. British Journal of Educational Psychology, 65, 73-84. Purdie, N., & Hattie, J. (1995). The effect of motivation training on approaches to learning and self-concept. British Journal of Educational Psychology, 65, 73-81. Purdie, N., & Hattie, J.A. (1996). Cultural differences in the use of strategies for self-regulated learning. American Educational Research Journal, 33, 845-871. Purdie, N., Hattie. J.A., & Douglas, G. (1996). Student conceptions of learning and their use of self-regulated learning strategies: A cross-cultural comparison. Journal of Educational Psychology, 88, 87-100. Watkins, D., Hattie, J.A., & Astilla, E. (1990). Affective and cognitive learning outcomes in privileged and depressed areas of the Philippines. In A.H. Otheman & W.R.A. Rahman (Eds.), Psychology and socioeconomic Inaugural August 2, 1999 Page 24 development (pp. 203-217). Penerb and Universiti Kebangsaan, Malaysia, Bangi. Watkins, D., Hattie, J.A., & Regmi, M. (1994). The structure of self-esteem in Nepalese children. Psychological Reports, 74, 832-839. Inaugural August 2, 1999 Page 25