EVALUATING PRACTICE IN CONTEXT

e-ISSN: 2582-5208 International Research Journal of Modernization in Engineering Technology and Science ( Peer-Reviewed, Open Access, Fully Refereed International Journal ) Volume:03/Issue:12/December-2021 Impact Factor- 6.752 www.irjmets.com EVALUATING PRACTICE IN CONTEXT Dr. Saleena Shad Gil*1 *1English Language Institute, Princess Nourah Bint Abdulrahman University, Riyadh Saudi Arabia. ABSTRACT In this research paper, I am going to evaluate the appropriateness of the level 1 coursebook for teaching and learning vocabulary. To carry out this study the researcher developed two questionnaires: one for teachers and the other for students. The criteria addressed in the questionnaires were to some extent similar to the criteria identified by the previous researcher with some adaptations according to the needs of the research. Research showed that although, textbooks gave a reasonable contribution to the teaching-learning process both to the teachers and to learners and it offered a framework of guidance and orientation. However, apart from numerous advantages a single textbook frequently does not meet the diverse needs of the learners. This generates a need for textbook adaptation at the activity, unit, and syllabus levels. Adaptation provides teachers with an opportunity to make greater use of their professional skills and for learners to be involved in the learning process. Most learners, too, acknowledge the importance of vocabulary acquisition the participants rely heavily on the textbook. Keywords: Evaluate, Questionnaire, Acquisition, Material, Rationale. I. INTRODUCTION Since the late 1980s, experts have paid more attention to the issues of efficiency and quality of language materials as a result of higher demand for quality education, English as a Foreign Language (EFL) and English as a Second Language (ESL). According to Tomlinson (1988, p.3) textbooks are the "main experience of using materials" in each language learning course, where “language material” is mostly associated with the term “textbook”. The current study analyses the set of presented printed learning materials used for teaching English diplomas of Princess Nourah Bint AbdulRahman University. The purpose of this study is to assess the overall effectiveness of the specific set of teaching material for learning English as a second language from the perspective of teachers and students and to give a recommendation for any improvements to both the university and the authors of the book. English language teaching (ELT) materials establish the backbone for English language education (Tomlinson & Masuhara, 2017). For the evaluation of a set of learning material, the Level 1 coursebook has been chosen specifically for vocabulary. Sheldon (1988, p.237) suggests that textbooks not only “represent the visible heart of any ELT program” but also offer considerable advantages for both the student and the teacher. Level 1 coursebook was released by Cambridge University Press in 2011, by Louis Alexander. Louis (2011) claimed that the topics in the coursebook are motivating enough to raise students’ interest and the systematic method of presenting new words would probably assist learners to develop their own strategy of tackling vocabulary. The university offered 8 weeks Level 1 course to Arab students with basic English competencies. The vast majority of these students aim to get fluent in English speaking for studies, jobs, or travel purposes. The course was taught by two experienced teachers who have been teaching this course for a minimum of three years. The first rationale behind choosing to evaluate the Level 1 coursebook is that I have experience using this learning material in the classroom for 5 years now and Tomlinson (2013) suggests that post-use evaluation can reveal the most about materials' impact on language learning. Cunningsworth (1995, p.14) believes that “through identifying strengths and weaknesses in textbooks, optimum use can be made of strong points, and weaker points can be adapted or substituted from other books”. The second rationale behind evaluating the material in context is to focus particularly on teaching vocabulary. Schmitt (2008) describes that undoubtedly, learning vocabulary is an essential part of language mastery, and developing rich vocabulary is a necessity for both Language 1 (L1) and Language 2 (L2) learners, but due to the incremental nature of word learning, it is an on-going challenge. Being a second language learner, I still face this challenge which motivated me to focus particularly on vocabulary learning. The third rationale behind evaluating the material in context particularly for vocabulary is the recent research which indicates that teaching vocabulary may be problematic because many teachers are not confident about www.irjmets.com @International Research Journal of Modernization in Engineering, Technology and Science [1510] e-ISSN: 2582-5208 International Research Journal of Modernization in Engineering Technology and Science ( Peer-Reviewed, Open Access, Fully Refereed International Journal ) Volume:03/Issue:12/December-2021 Impact Factor- 6.752 www.irjmets.com the best practice in vocabulary teaching and at times do not know where to begin to form an instructional emphasis on word learning (Berne & Blachowicz, 2008). I, at times, do not feel confident enough to teach vocabulary as well, because Teaching words is a crucial aspect of learning a language as languages are based on words (Thornbury, 2002) which were one of the reasons to focus on vocabulary as well. The third rationale behind choosing the Level 1 coursebook for evaluation is the fact that it can be very beneficial in teachers’ development and professional growth. Cunningsworth (1995) states that textbook evaluation helps teachers to acquire useful, accurate, systematic, and contextual insights into the overall nature of textbook material. Therefore, coursebook evaluation can potentially be a means of conducting research as well as a form of professional empowerment and improvement (Sheldon, 1988). Another aim of this study is to pilot a checklist that is specifically developed for evaluating the effectiveness of the Level 1 coursebook for teaching and learning vocabulary. Based on the results of this study, the pros and cons of the textbook will be discussed, and some suggestions will be offered to improve this coursebook. II. METHODOLOGY To carry out this study, it is crucial to understand what is ‘material’? The term "materials" could be defined as anything which can be used to facilitate the teaching and learning of a language (Richards and Schmidt, 2002). Wala (2003, p.60) argues that the coursebook is essential because it, “fulfills a need, a purpose, it performs a function, conveys meaning… language and coursebooks do not exist in a vacuum they exist for and are shaped by a purpose within a particular context of use, culture, and ideology”. Tomlinson (2011, p.2) defines the material as ‘anything which is deliberately used to increase the learners’ knowledge and/or experience of the language.” McGrath (2013) and Hyland (2006) describe materials, as being texts, which are used to assist in the learning of language. According to Garton and Graves (2014), numerous institutions favor the coursebook as material and even as a syllabus. A textbook has always been the most preferred instructional material in ELT. They are best seen as a resource in achieving aims and objectives that have already been set concerning learner needs (Cunningsworth, 1995). Consequently, this literature has helped me to choose a level 1 coursebook as a material for evaluation to judge the worth of the coursebook, particularly for learning and teaching vocabulary. According to Neuman & Dwyer (2009, p.385) vocabulary can be defined as '' words we must know to communicate effectively.” Hence, “Lexical knowledge is central to communicative competence and the acquisition of a second language” (Schmitt, 2000, p.55). Diamond & Gutlohn (2006) believe “vocabulary is the knowledge of words and word meanings.” The importance of vocabulary in language learning is also known to students. As Schmitt (2010, p.4) stated, "learners carry around dictionaries and not grammar books". Wilkin (1972, p.111) argued that "while without grammar very little can be conveyed, without vocabulary nothing can be conveyed." Both teachers and students agree that acquisition of the vocabulary is a central factor in teaching and learning a language (Walters, 2004). Lewis (1993, p.89) also maintained that "lexis is the core or heart of language". This theoretical background leads me to focus specifically on vocabulary for language learning. According to Tomlinson (2013), material evaluation attempts to measure the value of materials that must maximize the likelihood of learners actually learning what they want and need to learn (Tomlinson, 2011). Therefore, material evaluation should be the top priority of any curriculum (Zohrabi, 2011). It is essential to undertake material evaluation as textbook evaluation enables teachers, supervisors and material developers, to make judgments about the effect the materials have on the users (Tomlinson and Masuhara, 2001). Although no textbook can be perfect, therefore textbook evaluation is very important to clarify the suitability of the sources and find the best one (Gholami, Nikou, & Soultanpour, 2012). McDonough and Shaw (2003) maintain that textbook evaluation is very useful as it allows the teachers to choose the best possible book for their language learners. Tomlinson (2013, also emphasized that the overall purpose of the evaluation is to decide whether or not material should be used in a language teaching context which is the aim of carrying out this study. Tomlinson (2013) suggests that material evaluation can be divided into two parts i.e. macro evaluation and micro evaluation. The distinction between both types of evaluation is that macro evaluation is concerned with large scale projects, typically carried out for accountability and or developmental purpose; whereas, micro evaluations are used at the classroom level, especially to recognize the achievement of learners, however, this achievement is not a solitary cognitive aspect but comprise all potential learners have. According to this definition, the material evaluation, selection, and adaptation can be categorized as micro evaluation. Therefore, www.irjmets.com @International Research Journal of Modernization in Engineering, Technology and Science [1511] e-ISSN: 2582-5208 International Research Journal of Modernization in Engineering Technology and Science ( Peer-Reviewed, Open Access, Fully Refereed International Journal ) Volume:03/Issue:12/December-2021 Impact Factor- 6.752 www.irjmets.com the material evaluation would involve evaluating how effective and beneficial the materials are found to be in actual use by a specific group of teachers and students or how effective they may have been in facilitating learning and selection of material which is concerned with the potential that a set of the material may have in effectively and efficiently supporting learning, as a frame for learning and teaching opportunities (Tomlinson, 2013). Evaluation, like the selection, is concerned with judging the appropriateness of something for a specific purpose and analyzing learner needs, and interests and how these are addressed. Therefore, the selection of materials inevitably incorporates a process of evaluation where evaluation can be undertaken for a variety of purposes and carried out in a variety of ways. According to Tomlinson (2013, p.7) “there are many different types of materials evaluation. It is possible to apply the basic principles of materials evaluation to all types of evaluation, but it is not possible to make generalizations about procedures which apply to all types”. However, as mentioned earlier in context, Tomlinson (2013) suggests that post-use evaluation can reveal the most about materials' impact on language learning. Post-use evaluation can measure the short-term effect concerning motivation, impact, achievability, instant learning, etc., and it can also measure the long-term effect regarding durable learning and application. Post-use material evaluation can play a key role in making effective educational decisions. According to McGrath (2002), this kind of evaluation aims to scrutinize the effectiveness of a textbook in order to establish if it needs to be substituted, supplemented, and adapted, or retained without any specific change. In order to achieve a successful textbook evaluation process, post-use evaluation deserves due consideration. Hence to carry out this study, a post-use evaluation has been employed to analyze the effectiveness of the Level 1 coursebook for teaching and learning vocabulary. Post-use effect of materials can be evaluated in numerous ways, for example, questionnaires, interviews, examinations, etc. (Tomlinson, 2013). To carry out the materials evaluation, it is important to develop a principled evaluation framework based on a thorough reading of core literature about English language teaching and learning, materials evaluation, and research methods. The use of already available materials implies pros and cons, and these fluctuate according to each target course. Unfortunately, there is no global recipe to perform an effective material evaluation compatible at all levels and for all areas; however, while the literature on the subject of textbook evaluation is not particularly extensive, numerous researchers have presented evaluation 'checklists' based on evidently generalizable criteria that can be used by both teachers and students in various situations, which may help to recognize gaps, avoid drawbacks, identify achievements, and antagonize strengths and weaknesses to make decisions for materials use. (Ghorbani, 2011; Mukundan & Ahour, 2010; Mukundan & Nimehchisalem, 2012; Tomlinson, 2012; Tsagari & Sifakis, 2014). According to Sheldon (1988), there is no general checklist applicable to all teaching and learning contexts without substantial modification, however, most of these standardized evaluation checklists encompass similar components that can facilitate starting points for evaluation in an extensive variety of situations. By developing or adopting an evaluation checklist can facilitate teachers to examine, evaluate and select new textbooks. An evaluation checklist can assist to ensure that we scrutinize textbooks from numerous angles (Wong, 2013). Dornyei emphasized that “One of the most common methods of data collection in the second language research is to use questionnaires of various kinds. The popularity of questionnaires is since they are easy to construct, extremely versatile, and uniquely capable of gathering a large amount of information quickly in a form that is readily processable” (Dornyei, 2003, p.1). “Questionnaires can yield three types of data about the respondent: factual, behavioral, and attitudinal” (Dornyei, 2003, p.8). Ur (1996), Jordan (1997), Dudley-Evans and St John (1998), Ellis (1998), Littlejohn (1998), Hall (2000), Tomlinson et al. (2001), Islam and Mares (2003), Rubdy (2003) or Tomlinson (2003c), have offered general guidelines on how to evaluate published language-learning materials for classroom use. Hence, two questionnaires were designed by selecting some features which were common to most of the checklists proposed by different authors, to commence the textbook evaluation for this study (See Appendix F & G). III. DATA COLLECTION According to Tomlinson (2003, p.23) “making an evaluation criterion-referenced can reduce subjectivity and can certainly help to make an evaluation more principled, rigorous, systematic and reliable.” Dornyei (2003) believes that asking questions is one of the most natural ways of gathering information and scientific research www.irjmets.com @International Research Journal of Modernization in Engineering, Technology and Science [1512] e-ISSN: 2582-5208 International Research Journal of Modernization in Engineering Technology and Science ( Peer-Reviewed, Open Access, Fully Refereed International Journal ) Volume:03/Issue:12/December-2021 Impact Factor- 6.752 www.irjmets.com facilitates finding answers to the questions systematically. Hence, questionnaires are certainly the most often employed data collection devices in statistical work. Questionnaires with sufficient and well-documented psychometric reliability and validity are not that easy to design (Dornyei, 2003). Hence, two questionnaires were carefully designed for the evaluation of the Level 1 coursebook (See Appendix D & E). To develop the questionnaire, the literature related to materials evaluation was reviewed. The review of the literature revealed that there were commonly used criteria in the textbook and material evaluation process. To elicit teachers and students’ perceptions of the effectiveness of the coursebook concerning the criteria determined based on the related literature, two questionnaires were designed by following the theorists such as Williams (1983), Sheldon (1988), Brown (1995), Cunningsworth (1995) and Harmer (1996) suggested evaluation checklists. The questionnaires were designed to collect both quantitative and qualitative data. According to Dornyei (2003), the items either asking about very specific pieces of information (e.g., name, gender) or giving various response options for the respondent to choose from (e.g., ticking a box) can facilitate quantitative or statistical data to test the research hypotheses. Whereas open-ended items would provide data that are qualitative and exploratory in nature. Open-ended questions include items where the actual question is not followed by response options for the respondent to choose from but rather by some blank space (e.g., dotted lines) for the respondent to fill. Qualitative data can lead us to identify issues not previously anticipated. Furthermore, sometimes we need open-ended items for the simple reason that we do not know the range of possible answers and therefore cannot provide pre-prepared response categories. According to Oppenheim (1992, p.180), “the writing of successful attitude statements demands careful pilot work, experience, intuition and a certain amount of flair.” Therefore, after piloting the questionnaires with some colleagues, it was revealed that questionnaires were too lengthy and time-consuming to fill in, as suggested by Brown (2001, p. 45) “[...] short questions are good questions”. It was also revealed that very like a market-research questionnaire and does not focus on vocabulary teaching. Consequently, these checklists (See Appendix F & G) were adapted to the needs of this study to collect meaningful data. As Gillham (2000, p.1) maintains, “that good research cannot be built on poorly collected data [...]” The student questionnaire was given to 10 students who volunteered to take part in this study. The questionnaire comprised three sections. The first section requested background information such as name (optional), gender, first language, etc. The second section encompassed 10 statements, focusing on the effectiveness and the number of vocabulary activities. Participants were required to endorse on a five-point Likert-type scale from “strongly disagree” to strongly agree”. The third section of the questionnaire included 4 open-end items related to the criteria determined through literature review (i.e. Ur (1996), Jordan (1997), Dudley-Evans and St John (1998), Ellis (1998), Littlejohn (1998), Hall (2000), Tomlinson et al. (2001), Islam and Mares (2003), Rubdy (2003) or Tomlinson (2003c), have offered general guidelines on how to evaluate published language-learning materials for classroom use) to conduct a micro-level material evaluation where students were requested to give their opinion about the effectiveness of the coursebook in terms of learning vocabulary. In this section, students were also given the opportunity to state if they have any other comments about the coursebook or learning vocabulary (See Appendix F). The 2nd questionnaire was developed for the two Arabic-speaking EFL teachers, having a minimum of three years’ experience of teaching the same course. Both teachers volunteered to take part in this study. Based on their teaching experience, it was believed that these teachers had a good command of the content, design, and objectives of the coursebook. The teachers’ questionnaire comprised two sections. The first section encompassed 5 tick-only items requesting background information such as name (optional), age, first language and how long they have been teaching English, etc. to achieve quantitative data for analysis. The second section of the questionnaire incorporated 10 open-end items to elicit teachers’ opinions about the effectiveness of the coursebook for teaching vocabulary to achieve qualitative data. The focus of the questionnaire was the suitability of the coursebook for teaching new vocabulary and if the teachers would recommend the Level 1 coursebook to non-native or inexperienced teachers. The teachers were asked if they have to supplement the book to teach vocabulary as they like? Like students, teachers were also given the opportunity to write comments they would like to add about the coursebook or teaching vocabulary (See Appendix G). The consent slips were given and signed before filling in the questionnaires and both students and teachers were fully www.irjmets.com @International Research Journal of Modernization in Engineering, Technology and Science [1513] e-ISSN: 2582-5208 International Research Journal of Modernization in Engineering Technology and Science ( Peer-Reviewed, Open Access, Fully Refereed International Journal ) Volume:03/Issue:12/December-2021 Impact Factor- 6.752 www.irjmets.com aware that their participation was voluntary, and they would have the right to withdraw up until one week after the questionnaire had been completed. The participants were fully aware of the aim of the research, and they were assured of their anonymity. The hard copies of the questionnaires were distributed to both students and teachers. The soft copies had not been sent to the participants through email because most of the participants did not have email accounts or they did not want to share them because of cultural reasons. Survey-monkey was not used because most of the participants refused to fill the questionnaires online. It took them around 20 minutes to fill the questionnaires. Due to the small number of participants, Microsoft Excel was used to save the data collected and it was analyzed and shown in charts using Microsoft word (See 4. Findings). IV. FINDINGS In this study, both quantitative and qualitative data were obtained through students’ and teachers’ questionnaires for the micro-level evaluation of the textbook. Most of the students were 20-29 years old and had been learning English for 1-5 years. Their first language is Arabic, and all strongly agreed (i.e., 100%) with the importance of learning vocabulary and how difficult it is to learn new vocabulary. 70% of the students seem happy with the coursebook in terms of vocabulary presentation, exercises, stories, images, Arabic translation, etc. 90% of the students strongly agreed that the coursebook does not help them to retain new vocabulary, while only 10% disagreed. 60% of the students strongly agreed that they found the coursebook interesting yet challenging, 30% agreed to this while only 10% disagreed. However, they want to learn the vocabulary they can use in daily life. e.g., in the hospital, in restaurants or cinemas, etc. 100% of the students agreed that the Arabic translation of new vocabulary words in the coursebook has made learning new vocabulary less challenging. 60% of the students strongly agreed that the new vocabulary words are repeated in subsequent lessons in the coursebook to reinforce their meaning and use while only 20% disagreed. There was an average response to the statements claiming that the coursebook includes written and interactive exercises to practice new vocabulary. Almost an average response to the statement that the coursebook contains interesting pictures to explain the meaning of new vocabulary. Student Questionnaire Coursebook contains interesting pictures to explain meaning of new vocabulary Coursebook include written and interactive exercises to practice new vocabulary 40 50 50 New Vocabulary are repeated in subsequent lessons 10 50 20 0 60 10 Arabic translation of new Vocabulary made it less 0 challenging 100 0 Need to learn relevant daily life vocabulary 0 100 0 Coursebook interesting yet challenging 10 Coursebook does not help to retain vocabulary 10 Coursebook for vocabulary presentation 60 90 30 0 0 70 Learning vocabulary is important yet challenging 0 100 0 Disagree 30 20 Strongly Agree 40 0 60 80 100 120 Other Both teachers were between 30-39 years old and were Arabic speakers. Both had a minimum of 3 years of teaching experience and had been teaching Level 1 coursebook since. Teachers' questionnaires showed a slight hesitation (i.e., 10%) in teaching vocabulary unless they use their own teaching material. Overall, they www.irjmets.com @International Research Journal of Modernization in Engineering, Technology and Science [1514] e-ISSN: 2582-5208 International Research Journal of Modernization in Engineering Technology and Science ( Peer-Reviewed, Open Access, Fully Refereed International Journal ) Volume:03/Issue:12/December-2021 Impact Factor- 6.752 www.irjmets.com recommend this coursebook for level 1, there are, however, some responses worth considering i.e., they recommended some adaptation in the book. For example, they recommend more vocabulary exercises. Hence, both teachers fully agree (i.e., 100%) to recommend some adaptations in the coursebook. Teachers agreed 100% that they have to supplement the book to teach vocabulary to achieve lesson objectives. Teachers agreed 70% recommend the book to novice teachers as it is simple and straightforward to achieve lesson objectives to a certain extent, however, to maximize the chances of achieving lesson objectives and to prepare students to pass the exams they recommend some adaptations. Like students, teachers also agreed to the need for vocabulary that students can use in real life. They recommend this book with extra vocabulary exercises. Teachers' response to the organization of vocabulary tasks from the simple to complex was average. Teachers completely disagreed with the fact that vocabulary is sufficient for students' level and that the coursebook has a top-down technique for learning new vocabulary. Teachers' response to the statement that the coursebook develops students' vocabulary in a structured way was also average. Teachers’ Questionnaire Coursebook develops Ss' vocabulary in a structured way 30 50 20 Coursebook have topdown technique for learning new vocabulary 100 0 Vocabulary is sufficient for students' level 100 0 Task and activities were organized as moving from simple to complex 50 50 Recommend relevant daily life vocabulary 0 Recommended the book to novice teachers 0 100 10 0 70 20 Have to supplement the book to teach vocabulary 0 100 0 Recommended some adaptation in the book 0 100 0 Feel confident teaching vocabulary 10 70 0 Disagree 20 Agree 40 20 60 80 100 120 Other In sum, the results of the present study offered that both teachers and students had a negative attitude towards the content of their textbooks. Teachers claimed that the content of the coursebook includes an overwhelming and complex text with too many unfamiliar vocabulary words and is not relevant to the learners’ levels and their needs. Students also concurred with the teachers’ view and just mentioned that it is better for them to use texts that are more original in this course because this textbook cannot be enough for their future use. V. DISCUSSION The findings of the analysis revealed that the Level 1 coursebook was not very effective in terms of the organization of the task and activities. Both teachers and students gave an average response about the tasks and activities moving from simple to complex and strongly agreed that the tasks and activities were not logically organized. In addition, teachers also overwhelmingly agreed that the coursebook does not develop students’ vocabulary in a structured way. According to Breen and Candlin (1987), the selection and organization of content in the materials are quite important. They also emphasized the organization of the materials in a way that learners can work on them and internalize the newly learned vocabulary. Hence, the coursebook should be organized from the simple and familiar ‘thing’ to more complex and ‘less familiar’. www.irjmets.com @International Research Journal of Modernization in Engineering, Technology and Science [1515] e-ISSN: 2582-5208 International Research Journal of Modernization in Engineering Technology and Science ( Peer-Reviewed, Open Access, Fully Refereed International Journal ) Volume:03/Issue:12/December-2021 Impact Factor- 6.752 www.irjmets.com The data revealed that teachers’ perceptions about the level of the coursebook in terms of vocabulary were considerably more negative than the students’ perceptions. The findings indicated that both teachers believed the coursebook was above the level of the students and were not appropriate for the language background and level of level 1 students. Consequently, both teachers and students highly agreed that the coursebook does not help them to retain newly learned vocabulary. However, the appropriateness of the level of materials to the language level of students is one of the most discussed and emphasized criteria in a textbook evaluation by researchers (Breen and Candlin 1987; Cunningsworth, 1995; Grant, 1987; Sheldon, 1987; Skierso, 1991). The findings indicated that both teachers and students had a negative perception of the coursebook in terms of ‘content’ due to several reasons. For example, most of the students and teachers believed that the topics of the units in the Level 1 coursebook are monotonous. Both teachers and students perceived that the topics were not up-to-date and did not relate to their daily lives, hence, the textbook did not appeal to their interests. According to Timmis, Mukundan, and Alkhaldi (2009), materials should involve relevant content to the target learners’ needs. Another reason mentioned in questionnaires was that the reading texts encompassed overwhelming unknown vocabulary words and complex sentence structures, which consequently made this coursebook rather boring. Cunningsworth (1995) highlighted that the textbooks should include relevant content to the target learners' interests i.e., topics of the area of interest as part of real-life (i.e., wants and needs) which motivates learners to learn the target language effectively. He suggested that textbooks that fail to cover relevant and interesting topics are in danger of losing the interest and attention of their users. Breen and Candlin (1987) also maintained that if book writers can anticipate the relevant daily life, interesting and motivating topics, learners will automatically be interested in a certain textbook. Ellis (1997) also suggests that materials should reflect the learners’ present and future uses. The results also revealed that the coursebook did not include sufficient written or interactive exercises to practice newly learnt vocabulary for internalization. According to Edge and Wharton (1998), materials should assist learners to use the target language for communicative purposes. Hence, the coursebook did not facilitate the improvement of speaking skills to practice new vocabulary in the classroom. Both teachers and students believed that the topics of the coursebook do not encourage them to communicate in the second language (L2) with each other, first because the topics are not relevant, interesting and second because of unknown vocabulary and complex sentence structure. According to Cunningsworth (1995), a really communicative textbook facilitates meaningful communication among its users through relevant and interesting topics. Tomlinson (1998) also maintained that learners should be provided with opportunities to practice L2 to achieve communicative purposes. Skierso (1991) emphasized that textbooks should provide meaningful communication through communicative tasks and activities. He also highlighted the need for the sufficiency, variety, and adequacy of the activities. In the literature, numerous authors perceive ‘practice of vocabulary’ as a vital part of the materials evaluation and incorporate it in their guidelines and checklists. Skierso (1991), emphasizes that the practice of vocabulary is compatible with students’ needs and interests, readability, level, load appropriateness, context, balanced distribution of new vocabulary words among the lessons, and recycling draw learners’ positive attitude towards coursebook. One of the potential advantages of the coursebook is providing the users with the necessary skills and activities for language practice and interaction (Richards, 2001). According to Crawford (2002) and McGrath (2002), the coursebook should offer useful input for learners that help them to learn the English language effectively. The findings of the analyses indicated a good presentation of vocabulary in the textbook i.e. storyline. However, the response to pictures explaining vocabulary meaning in the coursebook was average. One of the most striking findings of this study was that the coursebook encompassed an overwhelming number of vocabulary words. Consequently, the students had great difficulty in comprehending the new vocabulary words and doing related tasks and activities. The study also highlighted that the number of unfamiliar vocabulary items per lesson was 30, however, according to numerous ELT specialists, the number of vocabulary items to be taught in one lesson for active use should not exceed more than seven (Celce-Murcia, 2001; Cunningsworth, 1995). Hence, 30 new vocabulary items to be taught in one lesson are well above the optimal number. According to Cunningsworth (1995), recycling of vocabulary is an important consideration. He emphasized that learners should come across vocabulary items on several occasions and rather in a different context so that www.irjmets.com @International Research Journal of Modernization in Engineering, Technology and Science [1516] e-ISSN: 2582-5208 International Research Journal of Modernization in Engineering Technology and Science ( Peer-Reviewed, Open Access, Fully Refereed International Journal ) Volume:03/Issue:12/December-2021 Impact Factor- 6.752 www.irjmets.com they can fix those items in memory, gain fluency in using them and achieve a full comprehension of vocabulary. The findings indicated that only about 60% of the participants perceive that the new vocabulary words are repeated in a kind of subsequent lessons to reinforce their meaning and use. Hinkel (2006) emphasized that a word should be encountered 12-20 times in order to be learned from a context. VI. CONCLUSION Byrd (2001) and Sheldon (1998) have argued, textbook evaluation criteria are context-dependent, however, particular needs of both the teachers and students should be considered. The present study was an attempt to evaluate the appropriateness of the level 1 coursebook for teaching and learning vocabulary. To carry out this study the researcher developed two questionnaires; one for teachers and the other for students (see appendices F & G). The criteria addressed in the questionnaires were to some extent similar to the criteria identified by the previous researcher with some adaptations according to the needs of the research (See 3. Data collection). Research showed that although, textbooks gave a reasonable contribution to the teaching-learning process both to the teachers and to learners and it offered a framework of guidance and orientation. However, apart from numerous advantages a single textbook frequently does not meet the diverse needs of the learners. This generates a need for textbook adaptation at the activity, unit, and syllabus levels. Adaptation provides teachers with an opportunity to make greater use of their professional skills and for learners to be involved in the learning process. Most learners, too, acknowledge the importance of vocabulary acquisition the participants rely heavily on the textbook. This further recommends that textbook writers need to check that new items are recycled appropriately in the course. However, as discussed earlier, the number of vocabulary items to be taught in one lesson for active use should not exceed more than seven (Celce-Murcia, 2001; Cunningsworth, 1995), as it leads to a burden for the student in comprehending the reading text which results in doing a lot of dictionary work. In addition, this may also cause de-motivation in reading lessons and learning new vocabulary. On the other hand, teaching such an excessive number of unfamiliar vocabulary words may also demotivate teachers from teaching vocabulary, affect their vocabulary teaching confidence or prevent teachers from using their instructional time effectively. Results revealed that both teachers spent extensive time summarizing the reading text for the students because teaching vocabulary takes most of the class time. Hence, an excessive number of vocabulary items hinder the learners from improving reading, writing, speaking, and listening skills. The findings also revealed that both groups believed that the number of vocabulary exercises and activities was not sufficient to practice the new vocabulary and in addition, vocabulary exercises do not facilitate vocabulary learning as most of the time is consumed in learning the meaning of new vocabulary words in both Arabic and English. Therefore, coursebook writers need to consider, introducing a lesser number of unfamiliar vocabulary words with more exercises to practice them for internalization. The process of materials evaluation can make the evaluators aware of the significance of the material evaluation framework and can also facilitate them to gain a deeper understanding and insights into the process of materials evaluation. Therefore, the framework likely has a vital role to play in the field of applied linguistics. In this study, the process of designing and applying the principled evaluation framework has enabled me to gain insights into the language materials and how they might be developed appropriately. After a comprehensive, intensive, and extensive reading of related literature about materials evaluation, learning and teaching unfamiliar vocabulary words, and research methodology, a comprehensive principled evaluation framework was designed to carry out this study. The framework was then applied to this study which offered a valid and reliable foundation for material evaluation. Hence, by carrying out this study, using this framework revealed the discrepancy between the learners, literature review, and the targeted material i.e., level 1 coursebook. Like any other research, this study also has some limitations. A major limitation of this study was the limited number of participants. The limited number of participants’ evaluation of a textbook cannot be considered as the final judgment to its rejection or acceptance, therefore, more ideas from a diverse number of learners and teachers with different experiences need to be taken into consideration in a longer-term project. In addition, if respondents check answers merely to complete an instrument, they are not reflecting upon the questions or indicating their true preferences (Porte, 2002). Another limitation of this study is, as criticized by Littlejohn www.irjmets.com @International Research Journal of Modernization in Engineering, Technology and Science [1517] e-ISSN: 2582-5208 International Research Journal of Modernization in Engineering Technology and Science ( Peer-Reviewed, Open Access, Fully Refereed International Journal ) Volume:03/Issue:12/December-2021 Impact Factor- 6.752 www.irjmets.com (1998), Rubdy (2003), Tomlinson (2003), regardless of the usefulness of checklists and questionnaires for their purpose i.e., to facilitate materials evaluation, actually make general, impressionistic judgments on the materials and do not offer a comprehensive analysis. Hence, the findings cannot be considered as the final judgment to its rejection or acceptance. By carrying out this study using the designed framework and going through intensive and extensive literature review, as a classroom teacher and researcher I have learnt useful basis to achieve professional development in numerous ways. For example, in past, I was unaware of material evaluation and only focused on self-evaluation as a teacher, however, by undertaking this study I have developed my skills in materials evaluation and comprehend the importance and benefits of carrying out the material evaluation. This study has helped me to develop my principles for materials evaluation and articulating principled evaluation criteria. I have also learnt how to design research instruments and to develop effective frameworks for materials evaluation. Carrying out this study has also enabled me to obtain the utmost comprehension of materials evaluation and language theories effectively. On the other hand, this study has given me the opportunity to see the teaching materials from students’ perspectives which I have not experienced before. This can help me develop teaching materials according to students' needs and interests. Another thing I would like to mention here is that carrying out this study has also helped me in a surprising way i.e., all the teachers in the university never discussed these issues like how everyone is struggling with materials and how this can be improved. Undertaking this study has brought us all on the same platform and I could learn many things from other teachers as well. It is hoped that the ideas expressed herein, and the personal experience shared both contribute to enriching current classroom practice and fruitful research. In the end, it should be noted that textbook evaluation is an ongoing process and according to Sheldon (1988, p.245) “materials evaluation is fundamentally a rule-of-thumb activity, and that no formula, grid or system will ever provide a definite yardstick.” VII. [1] [2] [3] [4] [5] [6] [7] [8] [9] [10] [11] [12] [13] REFERENCES Berne, J. I., & Blachowicz, C. L. Z. (2008). What reading teachers say about vocabulary instruction: Voices from the classroom. The Reading Teacher 62 (4).314-323. Breen, M.P, and Candlin. C.N. (1987). Which materials? A consumer’s and designer’s guide. In L. E. Sheldon (ed), ELT Textbooks and Materials. Problems in Evaluation and Development, (pp. 13-28). Oxford: Modern English Publications. Brown, J. D. (2001). Using surveys in language programs. Cambridge, UK: Cambridge University Press. Byrd, P. (2001). Textbooks: Evaluation for selection and analysis for implementation. In M. CelceMurcia, Teaching English as a second or foreign language (3rd ed.). US: Heinle & Heinle, Thomson Learning Inc, 415-427. Crawford, J. (2002). The role of materials in the language classroom: Finding the balance. In J. Richards & W. Renandya (Eds.), Methodology in Language Teaching: an anthology of current practice (pp. 8090). Cambridge: Cambridge University Press. Cunningsworth, A. (1995). Choosing Your Course Book. Oxford: Heinemann Diamond, L. & Gutlohn, L. (2006). Teaching Vocabulary. Retrieved from http://www.readingrockets.org/article/9943 Dornyei, Z. (2003). Questionnaires in second language research: construction, administration, and processing. Mahwah, N.J: Lawrence Erlbaum Associates. Dudley-Evans, T. and M.J. St John. (1998). Developments in English for Specific Purposes: A Multidisciplinary Approach, Cambridge: Cambridge University Press. Edge, J. & Wharton, S. (1998). Autonomy and development: living in the materials world. In B. Tomlinson (Ed.). Materials development in language teaching (pp. 295-310). Cambridge: Cambridge University Press. Ellis, R. (1997). The Empirical Evaluation of Language Teaching Materials. ELT Journal 51(1) 36-42. Ellis, R. (1998) The evaluation of communicative tasks. In Tomlinson, B. (Ed.). Materials Development in Language Teaching, Cambridge: Cambridge University Press: 217-238. Garton, Sue. & Graves, Karen. (2014). Identifying a Research Agenda for Language Teaching Materials. www.irjmets.com @International Research Journal of Modernization in Engineering, Technology and Science [1518] e-ISSN: 2582-5208 International Research Journal of Modernization in Engineering Technology and Science ( Peer-Reviewed, Open Access, Fully Refereed International Journal ) Volume:03/Issue:12/December-2021 Impact Factor- 6.752 www.irjmets.com [14] [15] [16] [17] [18] [19] [20] [21] [22] [23] [24] [25] [26] [27] [28] [29] [30] [31] [32] [33] [34] [35] [36] [37] [38] [39] [40] Modern Language Journal 98(2), 654-657 Gholami, J., Niko, F., & Soultanpour, A. (2012). A retrospective-comparative evaluation of textbooks developed by native and non-native English speakers. Journal of Academic and Applied Studies, 2(11), 35-42. Ghorbani, M. R. (2011). Quantification and graphic representation of EFL textbook evaluation results. Theory and Practice in Language Studies, 1(5), 511-520. Retrieved from https://doi.org/10.4304/tpls.1.5.511-520 Gillham, B. (2000). Developing a questionnaire. London: Continuum. Grant, M. (1987). Making the most of your textbook. London: Longman. Hinkel, E. (2006). Current perspectives on teaching four skills. TESOL Quarterly 40, (1), 109-125. Hyland, K. (2006). English for Academic Purposes. An advanced resource book. Oxon, Routledge. Islam, C. and C. Mares. (2003). Adapting classroom materials. In Tomlinson, B. (Ed.). Developing Materials for Language Teaching, London: Continuum: 72-100. Jordan, R. R. (1997). English for Academic Purposes. A Guide and Resource Book for Teachers. Cambridge: Cambridge University Press. Lewis, M. (1993). The lexical approach: The state of ELT and a way forward. Hove, UK: Language Teaching Publications. Littlejohn, A. (1998). The analysis of language teaching materials: Inside the Trojan Horse. In Tomlinson, B. (ed) Materials Development in Language Teaching, Cambridge: Cambridge University Press: 190-216. Louis, A. (2001). Level 1 Direct English: Cambridge University Press. McGrath, I. (2002). Materials Evaluation and Design for Language Teaching. Edinburgh: Edinburgh University Press. McGrath, I. (2013). Teaching Materials and the Roles of the EFL/ESL Teachers: Practice and Theory. London Bloomsbury. McDonough, J. & Shaw, C. (2003). Materials and methods in ELT: A teacher’s guide. Malden, MA: Blackwell Publishing Ltd. Mukundan, J. (2004). A Composite Framework for ESL Textbook Evaluation. Unpublished Ph.D. thesis, Serdang: University Putra Malaysia. Mukundan, J. & Ahour, T. (2010). A review of textbook evaluation checklists across four decades (19702008). In B. Tomlinson, and H. Masuhara (Eds.), Research for materials development in language learning: Evidence for best practice (336-352). London, UK: Continuum. Mukundan, J. & Nimehchisalem, V. (2012). Evaluative Criteria of an English language textbook evaluation checklist. Journal of Language Teaching and Research, 3(6), 1128-1134. https://doi.org/10.4304/jltr.3.6.1128-1134 Neuman, S. B. & Dwyer, J. (2009). Missing in action: Vocabulary instruction in pre-k. The Reading Teacher, 62(5), 384-392. Oppenheim, A. N. (1992). Questionnaire design, interviewing, and attitude measurement (New Edition). London: Pinter. Porte, K. (2002). Appraising Research in Second Language Learning. Amsterdam. PA: John Benjamins. Richards, J. (2001). Curriculum Development in Language Education. Cambridge: Cambridge University Press. Richards, J. & Schmidt, R. (2002). Longman Dictionary of Language Teaching & Applied Linguistics (3rd ed.). London: Longman. Rubdy, R. (2003) Selection of materials. In Tomlinson, B. (ed) Developing Materials for Language Teaching. London: Continuum: 37-57. Schmitt, N. (2000). Vocabulary in language teaching. Cambridge: Cambridge University Press. Schmitt, N. (2010). Researching vocabulary: A vocabulary research manual. New York: Palgrave Macmillan. Sheldon, L. E. (1998). Evaluating ELT textbooks and materials. ELT Journal, 42(4), 237-246. Skierso, A. (1991). Textbook selection and evaluation. In M. C. Murcia (ed.), Teaching English as a www.irjmets.com @International Research Journal of Modernization in Engineering, Technology and Science [1519] e-ISSN: 2582-5208 International Research Journal of Modernization in Engineering Technology and Science ( Peer-Reviewed, Open Access, Fully Refereed International Journal ) Volume:03/Issue:12/December-2021 Impact Factor- 6.752 www.irjmets.com [41] [42] [43] [44] [45] [46] [47] [48] [49] [50] [51] [52] [53] [54] [55] [56] [57] [58] second or foreign language (pp. 432-453), Boston: Heinle and Heinle Publishers. St John, M. J. (1996). Business is booming: Business English in the 1990s, English for Specific Purposes (15) 1: 3-18. Timmis, I., Mukundan, J., and Alkhaldi A. A.(2009). Coursebooks: soft or fair targets? Folio 13(2)11-13. Thornbury, S. (2002). How to teach vocabulary. England: Pearson Education Limited. Tomlinson, B. (1998). Glossary of basic terms for materials development in language teaching and introduction. In B. Tomlinson (ed.). Materials development in language teaching. (pp. 1-24). Cambridge: Cambridge University Press. Tomlinson, B., B. Dat, H. Masuhara, and R. Rubdy. (2001). EFL courses for adults. ELT Journal (55) 1: 80-101. Tomlinson, B. & Masuhara, H. (2001). Adults EFL courses. ELT Journal, 55(1), 80-101. Retrieved from https://doi.org/10.1093/elt/55.1.80 Tomlinson, B. (2003) Materials evaluation. In Tomlinson, B. (ed.). Developing Materials for Language Teaching. London: Continuum: 15-36 Tomlinson, B. (ed.). (2011). Materials Development in Language Teaching (2nd ed.). Cambridge, Cambridge University Press. Tomlinson, B. (2012). Materials development for language learning and teaching. Language Teaching, 45(2), 143-179. Retrieved from https://doi.org/10.1017/S0261444811000528 Tomlinson, B. (2013). Materials Evaluation. In: Tomlinson, B. (ed.). Developing Materials for Language Teaching. London, Bloomsbury. Tomlinson, B. & Masuhara, H. (2017). The complete guide to the theory and practice of materials development for language learning. Hoboken, NJ: John Wiley & Sons. Tsagari, D. & Sifakis, N. C. (2014). EFL coursebook evaluation in Greek primary schools: views from teachers and authors. System, 45, 211-226. Retrieved from https://doi.org/10.1016/j.system.2014.04.001 Ur, P. (1996). A Course in Language Teaching: Practice and Theory. Cambridge: Cambridge University Press. Wala, D. A. S. (2003). A coursebook is what it is because what it has to do; an editor’s perspective. In B. Tomlinson (Ed.). Materials development for language teaching (pp. 58-71). London: Continuum Press. Walters, J.M. (2004). Teaching the use of context to infer meaning: ‘A longitudinal survey of L1 and L2 vocabulary research'. Language Teaching, 37(4), pp. 243-252. Wilkins, D. A. (1972). Linguistics in Language Teaching. Hodder & Stoughton Educational. Wong, R. M. (2013). Developing Criteria for Textbook Evaluation. TESL Reporter, 4(1), 52- 74. Zohrabi, M. (2011). Coursebook development and evaluation for English for general purposes course. English Language Teaching, 4(2), 213-222. https://doi.org/10.5539/elt.v4n2p213 www.irjmets.com @International Research Journal of Modernization in Engineering, Technology and Science [1520]