Teachers’ Perceptions of High-Stakes Tests: A Washback Study

Teachers‟ Perceptions of High-Stakes Tests: A Washback Study Hadi Salehi, Melor Md Yunus, and Zeinab Salehi  Abstract—The study of high-stakes tests and various effects of such tests is an educational need and it is recommended by curriculum planners and educational researchers. This paper, therefore, aimed to explore the effects of the Iranian universities entrance exam on the high school English teachers. The Entrance Exam of the Universities (EEU) in Iran is a multiple-choice competitive high-stakes test that affects the teachers, students, parents and even other stakeholders. This negative or positive effect, generally known as washback, depends on the test itself, the curriculum, the stakeholders, and even the aim and scope of the test. To investigate the teachers’ perceptions of the EEU washback effect, a validated survey questionnaire was administered to stratified random sample of 132 high school English teachers who were teaching in the five main educational districts in the city of Isfahan, Iran. The descriptive data analysis revealed that little attention was given to three language skills of speaking, writing, and listening in the classroom as these skills are not tested in the EEU. Moreover, the EEU negatively and implicitly influences English teachers to teach to the test format. Index Terms—Entrance exam of the universities (EEU), high-stakes test, washback, stakeholder, english teacher I. INTRODUCTION It is generally believed that in the educational system testing should serve the needs of teaching. Bachman states that tests are virtually always intended to serve the needs of an educational system [1]. Language testing, as Davies puts it, provides goals for language teaching, and it monitors, for both teachers and learners, success in reaching those goals [2]. However, the functions of the tests in reality are far more beyond the role as an evaluation instrument, or the practical part as information resources for pedagogical refinement [3]. Tests, especially the high-stakes ones, affect individuals, policies or practices within the classroom, the school, the educational system, or society as a whole [4]. In recent years, many researchers as well as methodology authorities have been concerned about the effect of testing on teaching and learning. Investigation in this area as to the effect of tests that is technically called washback– a term popular in British applied linguistics – has borne fruitful results. The concept of testing can be viewed from two different perspectives: traditional testing and use-oriented testing [5]. Manuscript received December 2, 2011; Revised December 21, 2011 Hadi Salehi and Zeinab Salehi are with Faculty of Literature and Humanities, Najafabad Branch, Islamic Azad University, Najafabad, Isfahan, Iran (e-mail: [email protected], [email protected]) Melor Md Yunus is with Faculty of Education, Universiti Kebangsaan Malaysia (e-mail: [email protected]). The main focus of traditional testing is on designing and developing quality tests to accurately measure the knowledge of testers. Traditional view focuses on the objective type of items and ignores the test use. In fact, traditional testing takes tests as isolated events from test takers, educational system, and society. Use-oriented testing, on the other hand, takes testing as an embedded issue in educational, social and political contexts. This view addresses issues related to the rationale for giving tests and the effects that tests have on test takers, education and society [5]. A great number of issues such as test takers‟ activities for the test, the teachers‟ method of teaching, materials designed for the test, and the effects of the test results on the stakeholders are all taken into consideration in use-oriented view of testing. Therefore, from use-oriented view of testing, this study aimed to examine the teachers‟ perceptions of the washback effect of the EEU on the Iranian high school English teachers. II. RESEARCH BACKGROUND In many parts of the world, policymakers continue to use the effects of the tests to control their local educational systems, to supervise curricula and to promote new textbooks and recent teaching methods. In spite of this fact that there has been much controversy over the tests as long as they have existed, they are the darling of the policy-makers [6]. One of the reasons for the longevity of tests is that they are considered as the primary tools through which changes in the educational system can be introduced without having to change other educational components such as teacher training or curricula. In 1992, Shohamy stated that the phenomenon of washback is the result of the strong authority of external testing and its major impact on the lives of test takers [7]. Four years later, Shohamy, Donitsa-Schmidt and Ferman added that the results obtained from tests can have serious consequences for individuals as well as for programmes, since many crucial decisions are made on the basis of test results [8]. The power of the tests provides policy-makers an authority to use tests as effective tools for supervising educational systems in general and stakeholders‟ behaviors in particular. Moreover, the school-wide examinations can be used by principals and administrators to enforce learning, while in the classrooms tests can be used by teachers to motivate learners and impose discipline [9]. Although a great number of studies have been conducted on the effect of the tests in different contexts, little attention has been given to the high-stakes university entrance exams. Researchers from China, Hong Kong, Japan, Israel, and Turkey dealt with the English section of the university entrance exams in their own countries. The findings of these studies revealed that high-stakes tests should be examined based upon context in which they are administered [10], [11], [12], [13], [14]. Therefore, picking up on the complexity of this phenomenon, this study aimed at exploring the impact of the EEU in Iran on the high school English teachers. Since1960s, the EEU in Iran has been the sole criterion for the admission into higher education. This national test is a combination of some multiple-choice items and cloze tests regarding most of the courses that the students have been taught during a period of four years study in high school and pre-university center. It is administered in five main groups of students depending on their fields of study in high school [15]. Some of the topics including English, Persian Literature, Arabic, and Theology are tested for the students in all the majors. The English section of the test includes 25 multiple-choice items and a cloze test. The students are given 20 minutes, as required by this section of the test, to answer the questions. The first six or seven items are regarding grammatical points and the remaining items include vocabulary items, a short cloze passage and two short reading passages followed by some multiple-choice comprehension questions. III. METHOD A. Research Design A validated questionnaire was used as the research instrument in this survey study to explore high school English teachers‟ attitudes towards the impact of the EEU on English teaching in Iranian high schools. The questionnaire consisted of two main parts and was designed and prepared in English. The first part consisted of six categories of teacher personal characteristics related to demographic information and current teaching situations. All the items in this part were designed on a multiple-choice basis. The second part consisted of two categories and 16 items altogether. This part mainly dealt with the teachers‟ attitudes towards the EEU and all the items were designed on a five-point Likert scale of agreement, where one = strongly disagree, two = disagree, three = undecided, four = agree, and five = strongly agree. B. Sampling for the Survey The exact population of high school English teachers in the city of Isfahan was not known because some of the English teachers taught in several high schools. However, according to the statistical report by the Ministry of Education (ME) in 2010, there were 473 high schools and 3488 high school classes in the city of Isfahan, located in 5 main educational districts. Considering these statistics, an estimated method based upon the total number of classes and average classes a teacher taught was used to calculate the target population. Each English teacher taught an average of 8 classes; therefore, the estimated number of teachers in the target population was 436, which was derived by having 3488 divided by 8. Based upon the table for determining random sample size from a given population suggested by Payne and McMorris, 200 English teachers were sampled [16]. Stratified random sampling was used to ensure that adequate numbers of English teachers were represented from each educational district. Two hundred questionnaires were distributed through two different channels: (i) some questionnaires were directly emailed to the teachers and (ii) some other questionnaires were mailed to the school principals and English language teachers‟ coordinators in five main educational districts with stamped addressed envelopes and they subsequently arranged for the questionnaires to be carried out and returned back to the researcher. The total return rate was 66 % (132 questionnaires were returned). IV. RESULTS A. Teachers’ Demographic Information The analysis of the teachers‟ demographic information revealed that almost half of the teacher respondents were male (50.8 %) and the other half (49.2 %) were female (see Table 1). Almost three fourth of the teachers aged 20 to 40 and the remaining one fourth aged over 40 indicating that the majority of the English teachers were young or middle-aged. About half of the respondents (48.5 %) had BA and more than 40 percent of them had MA. One teacher had also BSc which shows he has not studied English as an academic major. Almost two third (65.1 %) of the sampled teachers were qualified professionally because they held teacher‟s certificate or advanced diploma in education. As Fig. 1 shows, more than half of the respondents (53 %) were teaching in pre-university centers as the major grade that they were teaching. Fig. 1. Major grades that teachers currently teach When the high school teachers were asked about the medium of instruction they use when they teach English in the classroom, about one-third of the respondents (31.8 %) stated that they use English supplemented with occasional Persian explanation (see Table I). Another one-third of the sampled teachers (31.8 %) also reported that they use half English and half Persian when teaching English in the classroom. In fact, there are more teachers who use „English supplemented with occasional Persian explanation‟ and „half English and half Persian‟ as their medium of instruction compared to those who use „English only‟ as the medium of instruction in teaching English in the classroom (see Fig. 2). Male Female 20-30 31-40 41-50 Above 50 BA 67 65 42 53 26 11 64 BSc Masters Others Teacher‟s Certificate Advanced Diploma in Edu. Others Grade One 1 55 12 51 0.8 41.7 9.1 38.6 Age Academic qualifications Professional qualifications Major grades of teaching Medium of instruction 35 26.5 46 12 34.8 9.1 Grade Two Grade Three Pre-university level English only 12 38 70 9.1 28.8 53.0 25 18.9 English supplemented with occasional Persian explanation Half English and half Persian Mainly Persian 42 31.8 42 31.8 23 17.4 B. Teachers’ Attitudes and Perceptions of the EEU The purpose of the second part of the questionnaire was to explore the teachers‟ attitudes towards the EEU. The obtained results from two categories of this part are reported according to the following themes: 1) Possible pressure under the EEU 2) Teaching methods teachers adopt due to the context of the EEU Possible pressure under the EEU In this category, the high school English teachers were asked what kind of extra work or pressure the EEU puts on their teaching. As it can be seen in Table II and Fig. 3, five items got the highest ratings. The majority of the respondents believed that the EEU makes them organize more exam TABLE II: TEACHERS‟ PERCEPTIONS OF PRESSURE UNDER THE EEU ACCORDING TO THE MEAN SCORES Statements Mean Organizing more exam practices Employing new teaching methods Revising some of the existing materials Following the teaching syllabus Preparing more materials for students Meeting new challenges in teaching Setting up new teaching objectives Doing more lesson preparation 4.07 3.59 3.58 3.56 3.31 3.30 3.10 2.55 Fig. 3. Teachers‟ perceptions of pressure under the EEU TABLE III: TEACHERS‟ PERCEPTIONS OF PRESSURE UNDER THE EEU ACCORDING TO THE FREQUENCY AND PERCENTAGE Statements Following the teaching syllabus Strongly disagree and disagree Undecided Agree and strongly agree 30 22.7 21 15.9 81 Percent Gender Percent (%) 50.8 49.2 31.8 40.2 19.7 8.3 48.5 Frequency Frequency Percent Variables Frequency Items Percent TABLE I: DEMOGRAPHIC CHARACTERISTICS OF THE TEACHERS Frequency Fig. 2. Teachers‟ medium of instruction in the classroom practices (mean = 4.07). This belief is a clear indication of the teachers‟ attention to the EEU. They try to deal with practicing more mock exam papers of the EEU and prepare the students to sit for this exam. This does necessarily mean that the teachers are affected by this high-stakes exam and it makes them direct their teaching method towards the test. More than two third of the respondents indicated that they have to employ new teaching methods to fulfill the students‟ expectations (see Table III). It is obvious that the students‟ expectations affect teachers to teach in a way to satisfy students‟ needs for passing this exam. Moreover, about two third of the respondents believed that the EEU forces them to revise some parts of the existing materials, follow the teaching syllabus, and prepare more materials for the students. Such beliefs indicated that the teachers modify the available materials and prepare more materials for the test-takers to meet their needs for passing this exam. On the other hand, more than half of the sampled teachers believed that the EEU does not force them to attend the classes prepared (mean = 2.55). The teachers are not required to prepare different segments for each lesson section. However, this belief does not necessarily mean that the teachers are willing towards the possible changes in the format of the EEU. 61. 4 45 34.1 7 5.3 80 30 22.7 15 11.4 87 29 22.0 14 10.6 89 44 33.3 34 25.8 54 32 24.2 37 28.0 63 13 9.8 8 6.1 111 31. 8 60. 6 65. 9 67. 5 41. 0 47. 7 84. 1 Teaching methods teachers adopt due to the context of the EEU When the teachers were asked about changes they make in their teaching due to the context of the EEU, the majority of them (89.4 %; mean = 4.40) indicated that they put more stress on reading comprehension activities (see Table IV, Table V and Fig. 4). Hence, the teachers may be affected in a way to ignore the productive skills of writing and speaking and the receptive skill of listening. About three fourth of the teachers (74.2 %) also indicated that they teach according to the EEU format due to the pressure from the context of this test. This finding clearly shows that the teachers are negatively influenced by the effect of the EEU. According to the teachers‟ beliefs, they direct their teaching methods towards the test format and adopt new teaching methods to help their students perform well on the EEU. Even about two third of the teachers (60.6 %) believed that the EEU motivates them to encourage their students to participate in class. However, this does not necessarily mean that the teachers involve their students in all four language skills. On the other hand, the majority of the surveyed teachers (82 %) believed that they do not employ communicative approach in their teaching due to the context of the EEU indicating how much the EEU has negatively affected the teachers‟ method of teaching. Although most of the teachers consider the communicative approach and task-based approach as the best methods of teaching, they do not employ these methods when they are teaching in high school classes. Moreover, more than three fourth of the respondents (78 %) confirmed that they do not employ real life language tasks, and more than two third of the sampled teachers indicated that they even ignore communicative skills and integration of skills. TABLE IV: TEACHERS‟ PERCEPTIONS OF ADOPTING TEACHING METHODS DUE TO THE CONTEXT OF EEOU ACCORDING TO THE MEAN SCORES Statements Mean To put more stress on reading comprehension activities To teach according to the test format To adopt new teaching methods To encourage more students‟ participation in class To put more emphasis on the integration of skills To put more emphasis on communicative skills To use a more communicative approach in teaching To employ more real life language tasks 4.40 3.81 3.54 3.37 2.65 2.65 2.59 2.55 TABLE V: TEACHERS‟ PERCEPTIONS OF ADOPTING TEACHING METHODS DUE TO THE CONTEXT OF EEU Statements Strongly disagree and disagree To teach according to the test format To adopt new teaching methods To use a more communicative approach in teaching To put more stress on reading comprehension activities To put more emphasis on communicative skills To put more emphasis on the integration of skills To employ more real life language tasks To encourage more students‟ participation in class Undecided Agree and strongly agree Percent 42 Frequency 12.1 Percent 16 Frequency 56.1 Percent 74 Frequency Doing more lesson preparation Preparing more materials for students Revising some of the existing materials Employing new teaching methods Setting up new teaching objectives Meeting new challenges in teaching Organizing more exam practices 30 22.7 4 3.0 98 74.2 36 27.3 7 5.3 89 67.5 82 62.1 4 3.0 46 34.8 9 6.8 5 3.8 118 89.4 73 55.3 11 8.3 48 36.3 70 53.0 12 9.1 50 37.9 78 59.1 15 11.4 39 29.5 38 28.8 14 10.6 80 60.6 Fig. 4. Teachers‟ perceptions of adopting teaching methods due to the context of EEOU V. DISCUSSION AND CONCLUSION It is surprising that a few number of studies have been carried out assessing the effects of EEU on the stakeholders of this exam. This is maybe due to the complexity of washback phenomenon. Anyway, it seems necessary to conduct different researches on the various aspects of this competitive exam to determine its major shortcomings. The negative impact of the EEU can be seen in the teachers‟ tendency towards using Persian as the medium of instruction in English classes. The teachers try to get the meaning of the language across to students so the students can better perform on the EEU. The findings of the study clearly indicated that the majority of the surveyed teachers believed that the EEU makes them organize and practice more mock exam papers of the EEU and prepare their students to sit for this exam. This does necessarily mean that the teachers are affected by this high-stakes exam and it makes them direct their teaching method towards the test. Since the EEU is regarded as a passport letting the students experience a newer and brighter world, the teachers do not look at the high school years as the years for learning all four language skills. Instead, they motivate their students to spend a great amount of time and energy on practicing mock exam papers and test taking strategies which will be useless from the day after the EEU date. Although the overall results of this study indicated that the EEU has negative effects on the teachers‟ method of teaching in the classroom, the elimination of this exam would not be practical at the moment; however, there is optimism that this exam will become obsolete in the near future. Hence, the basic revision of the EEU format is recommended which can in turn affect teachers‟ teaching activities and students‟ learning activities in the high schools. Moreover, the necessity of conducting various research studies on the EEU stakeholders to know their perceptions and attitudes towards this exam seems apparent. [8] REFERENCES [15] [1] [2] [3] [4] [5] [6] [7] L. F. Bachman, Fundamental considerations in language testing. Oxford: Oxford University Press, 1990. A. Davies, Principles of language testing. Oxford: Blackwell, 1990. L. Chu, and P. Gao, “An empirical study of the washback of CET-4 Writing,” Sino US English Teaching, 3(5), 36-38, 2006. D. Wall, “The impact of high-stakes testing on teaching and learning: Can this be predicted or controlled?” System 28, 499–509, 2000. E. Shohamy, “The power of tests: A critical perspective on the uses of language tests,” Harlow: Pearson Education, 2001. G. F. Madaus, “Public policy and the testing profession you have never had it so good,” Educ. Measurement Issues Practice, 4: 5-11, 1985. E. Shohamy, “New models of assessment: The connection between testing and learning,” In E. Shohamy and R. Walton (Eds.), Language assessment for feedback:Testing and other strategies, 1992, Dubuque, IA: Kendall Hunt Publishing Company. [9] [10] [11] [12] [13] [14] [16] E. Shohamy, S. Donitsa-Schmidt, and I. Ferman, “Test impact revisited: Washback effect over time,” Language Testing, 13(3), 298-317, 1996. L. Cheng and A. Curtis, “Washback or backwash: A review of the impact of testing on teaching and learning,” In L. Cheng, Y. Watanabe, and A. Curtis (Eds.), Washback in language testing: Research contexts and methods, 2004, (pp. 3-17). Mahwah, NJ: Lawrence Erlbaum Associates. L. Qi, “Has a high-stakes test produced the intended changes?” In L. Cheng, Y. Watanabe, and A. Curtis (Eds.), Washback in language testing: Research contexts and methods, 2004, (pp. 171-190). Mahwah, NJ: Lawrence Erlbaum Associates. L. Cheng, “Impact of public English examination change on students‟ perceptions and attitudes toward their English learning,” Studies in Educational Evaluation. 1998, 24(3), 279-301. Y. Watanabe, “Teacher factors mediating washback,” In L. Cheng, Y. Watanabe, and A. Curtis (Eds.), Washback in language testing: Research contexts and methods, 2004, (pp. 129-146). Mahwah, NJ: Lawrence Erlbaum Associates. I. Ferman, “The washback of an EFL national oral matriculation,” In L. Cheng, Y. Watanabe, and A. Curtis (Eds.), Washback in language testing: Research contexts and methods. 2004, (pp. 191-210). Mahwah, NJ: Lawrence Erlbaum Associates. O. Yildirim, “Washback effects of a high-stakes university entrance exam: Effects of the English section of the university entrance exam on future English language teachers in Turkey,” The Asian EFL Journal Quarterly, 12 (2), 92-116, 2010. Melor Md Yunus, H. Salehi, and S. Kashefian-Naeeini, “The impact of high-stakes tests on the teachers: A case of the Entrance Exam of the Universities (EEU) in Iran,” in Proc. International Conference on Humanities, Society and Culture (ICHSC 2011), Malaysia, 2011, pp.221-225. D. A. Payne and R. F. McMorris, Educational and psychological measurement. Waltham, Mass: Blasisdell Pub. Co. 1967, p. 419. Hadi Salehi is a PhD candidate in TESL at National University of Malaysia (UKM). He received his BA in English Literature from Isfahan University, Iran, and his MA in TEFL from Tehran Payame Noor University. He is also a lecturer in the Faculty of Literature and Humanities, Najafabad Branch, Islamic Azad University, Iran. His main research interests include Language Learning Strategies, Material Development, and Language Assessment.