Value Added Models of Teacher/Principal Evaluation

Value-added estimates of teacher or school quality are increasingly used for both high-and low-stakes accountability purposes, making understanding of their limitations critical. A review of the recent value-added literature suggests three concerns with the state of the research. First, the issues receiving the most research attention have not always been the concerns of greatest importance to theorists or critics. Second, there has been insufficient research on the interactions among various issues or assumption violations. Third, some of the big issues in value-added modeling have been challenging to address and may require educators to step back and answer some underlying philosophical questions about the nature of teacher and school quality.

3rd International Conference on Integrating Technology in Education (ITE 2022) This forum also aims to provide a platform for exchanging ideas in new emerging trends that needs more focus and exposure and will attempt to publish proposals that strengthen our goals.

Authors are solicited to contribute to the conference by submitting articles that illustrate research results, projects, surveying works and industrial experiences that describe significant advances in the areas of Integrating Technology in Education.

Kvalitet rada nastavnika postao je važno društveno-političko pitanje, budući da se nastavnici posmatraju kao ključni činioci uspeha učenika, te i razvoja sistema obrazovanja i društva u celini. Savremeni  momenat politike i prakse regulisanja ovog pitanja u mnogim zemljama, pa i kod nas, ogleda se u nastojanjima da se definišu uniformne mere kvaliteta (standardi i/ili indikatori) i da se rad nastavnika stavi pod spoljašnju kontrolu, u čemu evaluacija ima važnu ulogu. Kako su sami nastavnici često isključeni iz procesa odlučivanja o pitanjima koja se tiču kvaliteta rada, u istraživanju koje smo organizovali nastojali smo da dođemo do nastavničkih ideja za promene u praksi kada je u pitanju način na koji se pristupa kvalitetu njihovog rada. Organizovane su 3 fokus grupe sa predmetnim nastavnicima iz različitih vrsta škola u Beogradu i Nišu, a dobijeni podaci analizirani su tehnikama tematske i evaluativne analize sadržaja. Analize pokazuju da se kao najbolja strategija za negovanje kvaliteta rada nastavnika u praksi kod nastavnika pozicionira razvijanje podržavajuće i podsticajne klime u školskoj zajednici. Evaluaciju nastavnici ne vide kao značajan proces za razvijanje kvaliteta rada, već kao procedure koje služe spoljašnjoj kontroli rada nastavnika. Ovo ukazuje na značaj zapitanosti o svrsi evaluacije rada nastavnika pri koncipiranju strategija usmerenih na promene u načinu na koji se pristupa kvalitetu rada nastavnika u praksi, ukoliko nam je cilj da se evaluacija ne doživljava kao birokratski namet od strane nastavnika, već kao oruđe razumevanja i razvijanja prakse.

This article examines the re-professionalization of teaching across a transformative decade of market-based and standards-based reforms in the U.S.A. The first section works with the sociological concept of the ‘new professionalism’ to situate the No Child Left Behind act, Race to the Top, and CAEP accreditation within a broader movement to align the professions with the commercial and managerial values of the private sector. The next section compares teacher interviews conducted at the onset of NCLB with interviews conducted at the height of Race to the Top to highlight how the ongoing shift from ‘occupational’ to ‘organizational’ governance of teaching can change how teachers think about and fashion themselves as teachers. This policy analysis and qualitative analysis aims to help the education field recognize, situate and interrupt education reforms that govern teaching and teacher education through standardization, commercialization and outcomes-based performance management.

Note: Free eprints of this article are available at the following link (for the first 50 readers):
http://www.tandfonline.com/eprint/JRKjp53tnCN3cVKC53MF/full

ABSTRACT
This article draws from Stephen Ball’s work on markets, managerialism,
and performativity to frame a comparative study that examines
the reconstitution of the teacher–subject across a pivotal decade
in which neoliberal standards and accountability reforms effected
significant changes in US education. It juxtaposes two qualitative
studies conducted during the implementation of successive
standards and accountability movements. The first study of early
career English teachers coincided with the implementation of the
Bush administration’s No Child Left Behind Act of 2001 (NCLB), and the
second took place nearly a decade later as states began to implement
value-added teacher assessments in conjunction with the Obama
administration’s Race to the Top (RTTT). The juxtaposition of these two
studies points to a paradigmatic shift in the construction of teachers’
professional knowledge and subjectivity. While teachers of the first
accountability stage positioned NCLB’s (self-) disciplinary mechanisms
as external intrusions on their autonomy, professionalism, and practice,
the second group positioned RTTT’s accountability mechanisms as
the very modes by which they knew themselves and their quality.
Thus, these studies show a collapse between the governed (i.e.
teachers) and the government (i.e. accountability mechanisms) and
the normalization of the marketized teacher, the managed teacher,
and the performative teacher.

A teacher evaluation plan must be flexible enough to be useful and pragmatic enough to be used.  This is such a plan.  And since schools and teachers are not standardized products, a standardized evaluation tool would be highly ineffective.

This plan provides the structure necessary for teachers and administrators to work together to identify goals that are right for their schools and classrooms. It should be adapted to meet the unique needs of varying types of schools and classrooms.

Previous research has demonstrated conclusively that value-added inferences are sensitive to the choice of outcome measure within the domains of math and reading.  This implies that math and reading are multidimensional domains.  However, conventional factor analyses offer little support of this interpretation.  In this study some conceptual distinctions are explored with respect to what it is that tests of math and reading uniquely measure.  This is done by using longitudinal item response data from grades 5 to 9 to contrast undimensional item response theory model estimates of ability with the estimates from a bifactor item response theory model.  These different variable estimates are used as competing outcomes measures for a school-level value-added model.  It is shown that (1) the bifactor model produces domain-specific estimates that appear interpretable, and (2) that when a simple average is computed for value-added estimates in math and reading, this is equivalent to estimating value-added on a general factor common to math and reading tests.

The purpose of this study is to critically evaluate value-added accountability measures currently enacted in the United States at the federal and state levels to assess teacher preparation programme (TPP) e ectiveness. We draw on Newton and Shaw’s framework for the evaluation of testing policy to evaluate the technical quality and social acceptability of using K-12 student test scores to assess TPP e ectiveness. Through six guiding questions, we examine the assumptions and arguments that support value-added assessment, culminating in overall judgments about the acceptability of implementing (or continuing to implement) the testing policy. Findings suggest policy-makers may have more pragmatic concerns about the e ciency of value-added assessment, while TPPs may have more theoretical concerns about the validity of value-added assessment. The relevance of this evaluation approach to improving policy-related decision-making will also be discussed.

Value-added models (VAMs) are used to measure changes in student achievement on large-scaled standardized test scores from year to year. When aggregated, VAM estimates are used to measure teacher effectiveness and hold teachers accountable for the value they purportedly add to or detract from student learning and achievement. In this study, researchers examined the extent to which purposeful (nonrandom) and random assign- ment of students into classrooms occurs in Arizona elementary schools (Grades 3–6). Researchers found that overwhelmingly, students are not randomly assigned and administrators, teachers, and parents play a prodigious role in the process. Findings have current implications for value-added anal- yses and the extent to which nonrandom assignment practices might impact or bias teachers’ value-added scores.

For decades, policymakers have promulgated legislation that requires schools to hire effective teachers in all classrooms. Simultaneously, the education research community has attempted to define what effective teachers do in the classroom. A decade ago, No Child Left Behind provided a framework for defining effective teachers as " highly qualified, " which required schools to ensure all of their teachers fit the new standard. This standard, however, is no longer appropriate, as continued evidence indicates that the relationship between credentials and achievement is tenuous. Therefore, policymakers and researchers need to revise the term " highly qualified, " and, by utilizing the advances in educational accountability over the previous decade, replace it with a term grounded in practice and directly connected to achievement and effectiveness.

The purpose of this study is to critically evaluate value-added accountability measures currently enacted in the United States at the federal and state levels to assess teacher preparation programme (TPP) effectiveness. We draw on Newton and Shaw's framework for the evaluation of testing policy to evaluate the technical quality and social acceptability of using K-12 student test scores to assess TPP effectiveness. Through six guiding questions, we examine the assumptions and arguments that support value-added assessment, culminating in overall judgments about the acceptability of implementing (or continuing to implement) the testing policy. Findings suggest policy-makers may have more pragmatic concerns about the efficiency of value-added assessment, while TPPs may have more theoretical concerns about the validity of value-added assessment. The relevance of this evaluation approach to improving policy-related decision-making will also be discussed.

Se analiza el uso de los modelos de valor agregado como mecanismo para evaluar a los docentes siguiendo el estudio de  Chetty,  Friedman & Rockoff  (2011): Measuring the Impacts of Teachers I: Evaluating Bias in Teacher Value-Added Estimates. The American Economic Review.

The SAS Educational Value-Added Assessment System (SAS® EVAAS®) is the most widely used value-added system in the country. It is also self-proclaimed as “the most robust and reliable” system available, with its greatest benefit to help educators improve their teaching practices. This study critically examined the effects of SAS® EVAAS® as experienced by teachers, in one of the largest, high-needs urban school districts in the nation – the Houston Independent School District (HISD). Using a multiple methods approach, this study critically analyzed retrospective quantitative and qualitative data to better comprehend and understand the evidence collected from four teachers whose contracts were not renewed in the summer of 2011, in part given their low SAS® EVAAS® scores. This study also suggests some intended and unintended effects that seem to be occurring as a result of SAS® EVAAS® implementation in HISD. In addition to issues with reliability, bias, teacher attribution, and validity, high-stakes use of SAS® EVAAS® in this district seems to be exacerbating unintended effects.

Despite growing calls for more accountability of teacher education programs (TEPs), there is little consensus about how to evaluate them. This study investigates the potential for using observational ratings of program completers to evaluate TEPs. Drawing on statewide data on almost 9,500 pro- gram completers, representing 44 providers (183 programs) in Tennessee across 3 years, we inves- tigate multiple models to estimate TEP quality. Results suggest that using observational ratings to evaluate TEPs has promise. We were able to detect significant and meaningful differences between TEPs, which were fairly robust across modeling approaches. Moreover, TEP rankings based on observational ratings were positively and significantly related to rankings based on student achievement gains.

A teacher evaluation plan must be flexible enough to be useful and pragmatic enough to be used.  And since schools and teachers are not standardized products, a standardized evaluation tool would be highly ineffective.  Non-profit entities are welcome to use this plan.  [email protected]

Az intézményes nevelés számos hatással bír a tanulók személyiségének fejlődését tekintve. Ezen eredmények közül kiemelkedő a pedagógiai hatás a tanulók egész életen át tartó tanulásra való felkészülése, a kompetenciák fejlődése, a műveltség formálódása szempontjából. A hazánkban 2003 óta működő, és 2008 óta pedig már a tanulók szintjén is követhető, összehasonlítható adatokat produkáló Országos Kompetenciamérés (OKM) fontos indikátorokkal szolgál a pedagógiai munka eredményességét illetően, az érintett évfolyamokon (6., 8. és 10.), valamint matematikából és szövegértésből közvetlenül, de áttételesen a nevelőmunka minőségének egészéről is.

This paper was about analyses of the principals' effectiveness on school performance and how principals play a major role to make the school do well in terms of academic results. Furthermore, the paper discussed why it is important to have good leaders (principals) in schools who are able to contribute to the school's effectiveness in terms of school performance. The mixed-method was employed in this study as both descriptive and statistical data were used. In addition, the study included 30 participants who comprised both principals and teachers. The results revealed that the majority of teachers disagreed that their principals are not highly effective while some principals seem not to be well acquainted with the knowledge of the effectiveness of principals' roles.

Este estudio espera contribuir a la discusión sobre calidad docente en Chile a través de dos enfoques: el primero se basa en estimaciones de valor agregado del profesor o "efecto Profesor" en el aprendizaje. En él se explora el rol de las características del profesor y del contexto en que éste trabaja (escuela y municipio) en la estimación del efecto Profesor. El segundo enfoque intenta predecir el desempeño de los alumnos utilizando las variables de contexto y características del profesor. Con respecto al primer enfoque, los resultados muestran inestabilidad en las estimaciones de valor agregado, ante el ajuste de distintos modelos que incorporan covariables progresivamente. Por otra parte, el análisis de desempeño de los alumnos arroja como resultado que, de las variables de contexto, la escuela es más importante que el municipio para la predicción. A nivel del profesor, sólo resulta estadísticamente significativa la participación en perfeccionamientos profesionales, desestimándose el valor de la experiencia y de la formación inicial.

The main purpose of this paper is to examine the values defining today’s paradigms of the quality of higher education. In order to have a closer look at the main criteria of quality in the globally commodified and standardized mass higher education of our age, it is essential to clarify the underlying values. After a panoramic view of quality in today’s higher education, a historical retrospection is made to examine some historical shifts in the aspects and priority orders of values basically affecting
education. Then focus is shifted to process of linking the sociological, historical and cultural background to personal interest, in order to approach the various definitions given for value. The concluding thoughts of the study tackle the social value in higher education and its role in value socialization. The findings enable us to have a closer
view leading to the conceptualisation of the quality and added value in current higher education.

Scope International Journal of Education (IJE) is a Quarterly peer-reviewed and refereed open access journal that publishes articles which contribute new results in all areas of Education. The journal is devoted to the publication of high quality papers on theoretical and practical aspects of Educational research. The goal of this journal is to bring together researchers and practitioners from academia and industry to focus on Educational advancements, and establishing new collaborations in these areas. Original research papers, state-of-the-art reviews are invited for publication in all areas of Education.

The purpose of this article is to ethnographically document the market-based ideological assumptions of Rio de Janeiro’s educational policymakers, and the ways in which those assumptions have informed these policymakers’ decision to implement value-added modeling-based teacher evaluation policies. Drawing on the anthropological literature on meaning making (Anderson-Levitt, 2012), the focus of this study is on the common understandings and ideological assumptions regarding “good” teacher education practice that undergird the policymaking decisions of Rio de Janeiro’s public education policymakers. On the basis of ethnographic interviews, I argue that the then-current Secretariat of Education in Rio was run primarily by people whose backgrounds in business and administration heavily influenced their ideological assumptions about good educational management. I further explore the ways in which Rio’s implementation of value-added modeling and high-stakes accountability-based teacher evaluation mechanisms reflects these latent ideological trends.

The central theme of this article is that the usual Shannon’s entropy [1] is not sufficient to address the unknown Gaussian population average. A remedy is necessary. By peeling away entropy junkies, a refined version is introduced and it is named nucleus entropy in this article. Statistical properties and advantages of the Gaussian nucleus entropy are derived and utilized to interpret 2005 and 2007 waste disposals (in 1,000 tons) by fifty-one states (including the District of Columbia) in USA. Each state generates its own, imports from a state for a revenue, and exports to another state with a payment waste disposal [2]. Nucleus entropy is large when the population average is large and/or when the population variance is lesser. Nucleus entropy advocates the significance of the waste policies under four scenarios: (1) keep only generated, (2) keep generated with receiving in and shipping out, (3) without receiving in, and (4) without shipping out. In the end, a few recommendations are suggested for the waste management policy makers

International Journal of Education (IJE) is a Quarterly peer-reviewed and refereed open access journal that publishes articles which contribute new results in all areas of Education. The journal is devoted to the publication of high quality papers on theoretical and practical aspects of Educational research. The goal of this journal is to bring together researchers and practitioners from academia and industry to focus on Educational advancements, and establishing new collaborations in these areas. Original research papers, state-of-the-art reviews are invited for publication in all areas of Education.

Scope International Journal of Education (IJE) is a Quarterly peer-reviewed and refereed open access journal that publishes articles which contribute new results in all areas of Education. The journal is devoted to the publication of high quality papers on theoretical and practical aspects of Educational research. The goal of this journal is to bring together researchers and practitioners from academia and industry to focus on Educational advancements, and establishing new collaborations in these areas. Original research papers, state-of-the-art reviews are invited for publication in all areas of Education.

Scope International Journal of Education (IJE) is a Quarterly peer-reviewed and refereed open access journal that publishes articles which contribute new results in all areas of Education. The journal is devoted to the publication of high quality papers on theoretical and practical aspects of Educational research. The goal of this journal is to bring together researchers and practitioners from academia and industry to focus on Educational advancements, and establishing new collaborations in these areas. Original research papers, state-of-the-art reviews are invited for publication in all areas of Education.

Se analizaron las prácticas de 30 escuelas secundarias de Baja California, con el objetivo de identificar las actividades de gestión del tiempo y de organización del personal, que se realizan en escuelas secundarias que obtienen resultados académicos por encima de lo esperado para su contexto –valor añadido–. Participaron 16 escuelas de alto residuo –valor añadido– y 14 escuelas de bajo residuo, previamente seleccionadas mediante técnicas multinivel.

Bajo el enfoque cualitativo, se realizaron entrevistas semi-estructuradas y se indagó sobre la percepción que supervisores y equipos directivos tenían acerca de temas como la gestión del tiempo escolar, puntualidad y asistencia, y otras prácticas de organización que el personal y los alumnos realizaban. Los resultados mostraron diferencias entre las prácticas que realizan las escuelas de alto residuo en comparación a las de bajo residuo, lo cual permitió confirmar que las actividades de organización escolar y gestión influyen en el logro educativo que los estudiantes presentan.