Evaluating the SLATE project

lhs (print) issn 1742–2906 lhs (online) issn 1743–1662 Article Evaluating the SLATE project Ahmar Mahboob*, Angela Chan, Jonathan Webster Abstract his paper presents a broad evaluation of the Scafolding Literacy in Academic and Tertiary Environments project (SLATE) drawing from a range of sources, including (a) feedback from some of the key stakeholders involved, and (b) an analysis of student work. First, the paper presents data collected on students’ perceptions of some of the courses supported by the SLATE team. his data is discussed in relation to some of the issues raised in feedback collected from the course lecturers and the SLATE tutors. he paper then shits its focus and evaluates the project in terms of how the students beneited from the project. his is done by examining how frontloading and supported independent construction helped students in (developing) their use of grammatical metaphor. he indings of this project evaluation have been used to identify issues that need to be considered in developing similar programmes in the future. Keywords: academic writing; feedback; genre pedagogy; grammatical metaphor; project evaluation 1. Introduction Project evaluation is an important component of any project as it examines how project outcomes align with project objectives and thus can help improve projects (e.g. Calder, 1994; Payne, 1994; Allen, 2004). he Scafolding Literacy in Academic and Tertiary Environments (SLATE) project aimed to help non-English speaking background (NESB) undergraduate university students Ailiation University of Sydney, Australia. email: *[email protected] [corresponding author] lhs vol 7 2013 125–139 ©2013, equinox publishing doi : 10.1558/lhs.v7i1-3.125 126 Evaluating the SLATE project develop their English language and literacy needs at an English medium university in Hong Kong. It did so by embedding appropriate language and literacy support within some of their core units of study using online resources (see the Introduction to this Special Issue for a detailed description of the project). he SLATE team provided scafolded support to their students by adapting the teaching learning cycle (Rothery and Stenglin, 1995) to suit the needs of an online literacy support project. In developing this ‘consultative cycle’ (see Mahboob et al., 2010), students were irst provided with models and notes about the nature of their assignments, with explicit references made to the type of language resources they needed to draw on in order to successfully complete their assessment tasks. his phase of the intervention was called frontloading (deconstruction). Students were subsequently asked to drat their assignments, and this drating work was supported by the language coaches through a feedback process, called ‘supported independent construction’ (see Mahboob and Devrim, 2011, this issue, for details). In some courses, the SLATE team also experimented with online joint construction (Dreyfus and Macnaught, this issue) between the frontloading and the supported independent construction phases. he students used the support provided to them in drating and revising their work before submitting their inal assignments to their lecturers. he lecturers then graded these assignments based on their disciplinary criteria. he expected result of this consultative cycle was a gradual and scafolded development of students’ discipline-speciic language ability. It is this project and its expected outcomes that are the focus of this evaluative paper. he SLATE project was a complex project in terms of the various stakeholders involved. In addition to support from University administration and staf (including the Education Development Oice), the project’s main stakeholders included the students, the language coaches, and the subject teachers. he success and implementation of the project was dependent on successful collaboration and engagement between all three of these main stakeholders. Within this group, it was the students who were the primary beneiciary of the project – as the project was geared towards their language and literacy needs. hus, in line with the literature on project evaluation that requires an evaluation of stakeholder feedback (Sengupta and Leung, 2002; Alderson and McIntyre, 2006; Farr, 2008; Barton et al., 2009; Güneş and Dilek, 2009), the Halliday Center at the City University Hong Kong (hereater CityU) was assigned the task to independently monitor the project. In doing so, they collected detailed feedback from the students about the various aspects of their involvement with the project. his paper uses this data for one aspect of evaluation of the SLATE project. In addition, we also collected feedback from the two other main stakeholders in the project (teachers and the SLATE team). his paper Ahmar Mahboob, Angela Chan, Jonathan Webster 127 also reports on the results of this survey and shows that while the students and the other stakeholders were generally in favour of the project and found it useful, their feedback provided useful insights into how the project could be improved in the future. In addition to discussing the stakeholders’ perceptions and attitudes towards the project, a further focus of the study is an examination of it and how the SLATE literacy intervention impacted students’ use of particular language resources. his again, is in line with the literature on project evaluation (Light et al., 2000; Lorenzo et al., 2010), which suggests that project evaluation requires an assessment of whether the goals of the project are met or not. In order to do this, we will draw heavily on Devrim’s (2013) PhD research, which examined students’ use of grammatical metaphor in relation to the modelling and feedback support provided to them. he results of this analysis clearly show that the SLATE intervention achieved a number of the outcomes it had set out to achieve. In presenting the results of this project evaluation, we irst outline the methodology used. We then present the indings of the stakeholders’ feedback before looking at the impact of the intervention on students’ language use. he paper then concludes by summarizing the key indings and making speciic recommendations about how the project can be further improved. 2. Methodology he two primary methods of evaluating the SLATE project including studying the stakeholder evaluation of the project and examining the impact of the intervention on students’ use of language. Each of these required collecting and analysing diferent types of data. Below we irst examine the methodology used to collect and analyse data for the stakeholders’ evaluation, and then the methodology used to understand the impact of the project on student writing. 2.1. Stakeholder evaluation here were three main stakeholders involved in the project: the students, their course lecturers, and the SLATE team. We collected feedback on the project from all three of these stakeholders. Student feedback Students who participated in the project were asked to provide feedback on the support they received. his process was monitored by the Halliday Center at CityU to maintain the integrity of the evaluation process. he Halliday Center collected student feedback by means of a survey. Based on a review of literature on feedback in academic writing (e.g. Cohen, 1987; Ferris, 1995; Hyland and Hyland, 2001), they designed a questionnaire consisting of both closed- 128 Evaluating the SLATE project and open-ended questions. he closed-ended questions asked respondents to indicate the extent to which they agreed/disagreed with a number of statements about various aspects of the SLATE project. Within the closed-ended questions, there were three main categories of questions. he irst category (items 1‒6) focused on the students’ experience of the project in general; the second category (items 7‒9) included items that focused on students’ evaluation of feedback; and the third category (item 10) asked students about how the project impacted their language proiciency (with sub-items focusing on diferent aspects of their language). he data from closed-ended items were converted into numerical scores, where strongly agree = 5; agree = 4; neutral = 3; disagree = 2; and strongly disagree = 1, and tabulated for analysis. he open-ended questions allowed respondents to formulate their own responses about what motivated them to use SLATE, and what they liked about the service, and what they would like to see changed in future implementations. his data was analysed for the range and focus of students’ comments about the project. his paper draws on student responses collected in two courses that were supported by the SLATE team in the inal year of the project: (a) a course in the Department of Biology and Chemistry (BCH; n = 45); and (b) a course in the Department of Chinese, Translation, and Linguistics (CTL; n = 21). An analysis of these indings will help us in understanding some of the issues that led to this diference in perceptions and help provide insights into how the programme can be further improved. Lecturer feedback Feedback from the lecturers was collected during the period in which their courses were being supported. his data included e-mail exchanges between them and the SLATE team as well as notes made during meetings with them at various stages of the project. SLATE team feedback he SLATE team included the project coordinator, coordinators for the various courses, and the tutors. Feedback was collected from all of these participants both during and at the end of each semester. his feedback was based on an open-ended questions sent out via e-mail as well as during various conversations with them. 2.2. Students’ language use he review of students’ use of language presented in this paper is drawn from Devrim’s (2013) PhD research. Devrim’s work reports on the impact of the SLATE intervention on students’ use of the grammatical metaphor, a key lan- Ahmar Mahboob, Angela Chan, Jonathan Webster 129 guage resource used in academic writing. Grammatical metaphor is one of the most important aspects of technical/academic writing because it allows us to make meanings in incongruent ways, resulting in more formal language. he appropriate use of grammatical metaphor allows writers to realize technicality and abstraction, create logical reasoning within clause, use authoritative language while giving opinions, and provide incongruent ways of text structuring (Schleppegrell, 2004: 186). Efective control and use of the various types of grammatical metaphor empowers writers and therefore needs to be explicitly taught to students, which was the focus of Devrim’s work. Based on Halliday’s (1985) work, Devrim argues that there are three types of grammatical metaphor, namely, experiential grammatical metaphor, logical grammatical metaphor, and interpersonal grammatical metaphor. Experiential grammatical metaphor realizes actions/processes (verbs) and qualities of things (adjectives) as things (nouns). By using experiential grammatical metaphor, we can manage the technicality and abstraction in academic writing. Logical grammatical metaphor realizes logical relations between ideas within clause rather than between clauses. And, inally, the interpersonal grammatical metaphor provides us the tools to make our meanings more objective. hese grammatical metaphors are used to create particular types of meanings in text and are oten used in academic writing. hus, it is important to teach students studying in English medium tertiary institutions how to use these linguistic resources. Devrim’s work on grammatical metaphor, included the development of a set of resources to help students use them appropriately, as well as train tutors on how to provide scafolded support to their students on this feature of language. In evaluating the impact of the project on student language, we irst look at how students’ use of grammatical metaphor varied between phases 1 and 3 of Devrim’s PhD project, where one of the main diferences between the three phases was the amount of frontloading and support students were given about the use of grammatical metaphor. Students were given no frontloading on grammatical metaphor in Phase 1 of Devrim’s project, and received the highest amount of support in Phase 3. he diference between students use of grammatical metaphor in the irst drat of their assignments across the three phases therefore helped in tracking whether (and how) students used the resources provided to them in drating their assignments. Ater discussing these results, we will examine how students used feedback provided to them in developing their use of grammatical metaphor. 3. Findings In this section, we will irst discuss the stakeholders’ perceptions and then report on the impact of the intervention on students’ use of language. 130 Evaluating the SLATE project 3.1. Stakeholder evaluation he survey administered to students by the Halliday Center included both closed-ended questions as well as open-ended questions. he responses to the closed-ended questions from the two courses being reviewed here were tabulated to show the distribution of the student responses for each item. hese results – for both of the groups (Linguistics or CTL and Biology or BCH) as well as the average for the two groups ‒ are shared in Table 1. he Table shows that in general students had a positive attitude towards the project: they responded well to the project goals, found the feedback useful, and felt that their language was developing as part of their engagement with the project. hese indings are in line with other research on feedback (e.g. see Hedgcock and Kekowitz, 1994; Ferris, 1995; Hyland and Hyland, 2006). However, there were some students who gave a low rating for the project. On a closer review of the data, it can be seen that the students who gave negative comments were consistently negative in their feedback. Furthermore, in their open comments, these students also wrote that they were taking part in the project because it was a requirement for their course. his suggests that students who felt forced to be a part of this project were generally not very positive about the experience. What is interesting in considering these results is that while about 23% of the students had somewhat negative attitudes towards the project in general (Part A of the survey), only 10% of the students found the feedback problematic (Part B). his suggests that regardless of their attitudes towards the project, they found the feedback that they received helpful in their language development. Table 1: Student responses to the closed-ended questions. * indicates that at least one student in the CTL group did not give a response to this item; ** indicates that at least one student in the BCH group did not give a response to this item. Ahmar Mahboob, Angela Chan, Jonathan Webster 131 Table 1 provides a detailed account of the distribution of students’ comments to the closed-ended items in the survey. hese results show that for many (but not all) of the items, students in the BCH course were more positive towards the project as compared to CTL. his diference was explored further in the discussions with the SLATE team as well as the course lecturers. he feedback from the other stakeholders also showed a similar diference of opinion, i.e. the BCH staf and the SLATE team supporting the BCH group had a more positive perception of this group as opposed to the stakeholders working on the CTL course. he SLATE coordinators reported having diiculties in communicating with the CTL course lecturer and the course lecturer felt that the SLATE project should not be part of their workload. As a consequence, there was limited interaction between the SLATE team and the CTL course lecturer, which impacted the overall functioning of the project. his inding suggests that the attitude of the course lecturer towards the SLATE project inluenced both the SLATE team as well as the students’ engagement with the project. his inding is important as it underscores the importance of strong collaboration between consultancy teams such as the SLATE team and the course lecturer. 132 Evaluating the SLATE project he results of the survey that were most diicult to interpret were those where the students showed dissatisfaction with the project. hese results were further investigated by looking into the open comments provided by the students. he majority of students who were unhappy with the project complained about the amount of work and the tight timelines for the project. hey felt that participating in the project added to their workload considerably. he students who made these comments also wrote that their main motivation for participating in the project was that their lecturers required it and that it contributed to their inal grades. hus, they had somewhat negative attitudes towards their required participation in the project. While these perceptions require careful consideration by the SLATE team in planning for future embedding of literacy within subject learning, the essential consideration is how the students improved their language over time as a result of this intervention. A higher workload may also be an indication of how much time the students had to spend in relecting and responding to the feedback that was given to them. his interpretation was indirectly corroborated in the data collected from the other parts of the student survey: 90% of the students were positively oriented towards the feedback that they received. In addition, in response to an open-ended question where they were asked to state what they did in responding to the feedback, 93% of the students in BCH group and 90% of those in the CTL groups said that they reviewed the feedback carefully and made the changes suggested by their tutors. Making these changes took time and thus impacted the students’ workload. It also impacted the lecturers’ workload, who, at times, were inundated with student e-mails and questions. he increased workload was also a major concern raised by the lecturers and the SLATE tutors. he lecturers were aware of the additional time that the students had to spend on their drats and were also concerned about the time that they had to spend in responding to the student queries. his is an important inding of the project evaluation and is something that needs to be considered in how similar programmes are designed in the future. he SLATE tutors also commented on the workload. hey pointed out that a 48-hour window to review and give meaningful feedback to the students was quite challenging. hey said that, on an average, it took them anywhere between 45 minutes to an hour in responding to a 500 word student assignment. his meant that they could only work with a small number of students each semester. he SLATE coordinators used this feedback in keeping the student-tutor ratio low; the average load was about 10 students per tutor per semester. Regardless of the workload, the students were, in general, in agreement with the observation that the project helped them in their academic writing. here were some diferences in how they saw the project impact their language, but, in general, they found that the project helped them improve their Ahmar Mahboob, Angela Chan, Jonathan Webster 133 paragraph structure, text organization, and grammar much more than their vocabulary. his inding was not a surprise to the SLATE team because the SLATE project was designed to help students to achieve control of the genre of the texts they were writing and, to a certain degree, sentence grammar, but not (subject-speciic) vocabulary. he 3 × 3 (Humphrey et al., 2010), which was the informing theoretical framework for much of the SLATE project, was skewed towards these aspects of language and therefore students’ acknowledgement of improvement in these aspects of their language was a validation of the project goals. In addition to the students’ comments about their language development, we also analysed their actual use of language. he indings of this analysis are presented below. 3.2. Language Use In his doctoral research, Devrim (2013) looked at how students used and incorporated the information included in the literacy resources provided to them during the frontloading phase of the consultative cycle. In order to measure this, he looked at students’ use of grammatical metaphor across 3 phases of intervention. In Phase 1 of his project, students received no frontloading compared with Phase 3, where extended support on grammatical metaphor was provided. he relevant results from Devrim’s analysis of the diferences in students’ use of the targeted language resource are provided in Table 2. Table 2: Students use of experiential and logical metaphors. Research phase Drafts/change Number of experiential Number of logical metaphor metaphor Phase 1 procedural recount Assignment 1 draft -1 instances per 1,000 words 23.1 14.6 Phase 3 consequential explanation Assignment 2 draft 1 – instances per 1,000 words 28.1 14.7 Table 2 gives us the number of instances of experiential and logical grammatical metaphor used per 1,000 words in the irst drats of students’ irst assignments in phases 1 and 3 of the project. We have not included a review of the use of interpersonal or textual grammatical metaphor, because these were not expected in the genres that the students were writing in Phase 3 of the project. he results show that there is a marked increase in the use of the two relevant types of grammatical metaphor in Phase 3, which was the Phase in which Devrim gave detailed task-relevant support to students on how they can use grammatical metaphor in their work before they drated their assignments. hese numbers provide evidence that frontloading worked and that students 134 Evaluating the SLATE project supported by the SLATE team used the material given to them in drating their work. In addition to the frontloading material, tutors worked with the students individually to help them improve their work and develop their ieldspeciic language proiciency. We will look at this in more detail below. Devrim conducted an in-depth analysis of the role of feedback on grammatical metaphor in the SLATE project by tracking what feedback was given to the students and how they responded to it. he example shows how one student (second year second semester) worked on his use of grammatical metaphor across the drats by responding to feedback. (Tutors’ feedback to the student is included in the curly brackets.) Example 1.1: Max – drat 1 (feedback) Nominalization helps us for the purpose of compacting information, organizing information in more logical way, making the style of writing more academic and formal and removing people from the text. {Can you think of a more formal phrase here, example: removal of animate objects. … } In Example 1.1 above, the tutor noticed that the student was using informal language in his writing. he tutor told Max to ‘think of a more formal phrase’. he student responds to this feedback in their revised drat: Example 1.2: Max – drat 2 (response) Nominalization helps to compact information, organize information in more logical way, make the style of writing more academic and formal and removal of animate objects. As can be seen in Example 1.2 above, Max used the suggestion given to him in revising the text. However, there are still problems with this text. herefore, the tutor followed up with additional feedback: Example 1.3: Max - drat 2 (feedback) Nominalization helps to compact information, organize information in more logical way, make the style of writing more academic and formal and removal of animate objects. {he last part of the sentence is nominalized whereas the earlier part still includes processes. he underlined part can be made more formal by using nominalization.} In this feedback, the tutor, using some metalanguage (‘nominalize’, ‘processes’), told Max to nominalize his thesis statement. Max used this feedback to make changes to his inal submission; however, the inal version, while it responded to the feedback, was still not formed correctly. Example 1.4: Max ‒ drat 3 (response) Nominalization is useful in compactness information, re-organization information in more logical way, making the writing style more academic and formal and removal of animate objects. Ahmar Mahboob, Angela Chan, Jonathan Webster 135 he extended example above (1.1–1.4) shows how the tutor and the student worked together through feedback cycles across multiple drats to negotiate text revision. his extended negotiation across drats allowed the students to develop the metalanguage that they needed to understand the feedback and to learn how to use this feedback to change their writings. However, it should be noted that not all changes resulted in grammatically correct sentences; the students, by focusing on one aspect of language, made mistakes in others that they seemed to have had control of previously. his luctuation in the grammatical accuracy at the lexico-grammatical level of the texts signals that the students’ language system is still developing and the student is engaging with the scafolding provided to experiment with new ways of expressing themselves. his support, over time, helps students achieve control of the registers in which they need to write in their disciplines. he results from Devrim’s analysis provided evidence that the frontloading as well as the feedback support helped the students in using and developing genre and discipline-speciic language that they needed to succeed in their work. hus, these indings show that the SLATE project was able to achieve the goals that it had set. 4. Summary and recommendations he goal of this paper was to evaluate the SLATE project in terms of reviewing stakeholder feedback as well as evaluating the impact of the project on students’ language use. his review was undertaken in order to learn more about the project and to use the feedback and analysis to make recommendations about how to plan future projects of a similar nature. In this last section of this paper, we summarize the key indings and make a set of recommendations for future consideration. he review of stakeholder feedback as well as student language use indicated that the project was achieving its outcome. he survey participants noted that the feedback that they received was useful and helped with their language development. his perception was corroborated by the analysis of student writing. An analysis of students’ use of grammatical metaphors before and ater the SLATE intervention showed a marked increase in the students’ use of this language resource. In addition, an analysis of feedback provided to student texts and their response to it showed how students used the feedback in developing their work. hus, the evaluation of the project showed that the project achieved its intended outcomes. However, there were at least three concerns that were brought out during the evaluation of the project and which need some consideration: 136 Evaluating the SLATE project 1. 2. 3. Some students were negative about the project and even while they admitted it helped them, they saw it as additional work; here was an increased workload for the lecturers collaborating with the project; and here were some gaps in communication between the course lecturer and the SLATE team. he irst issue identiied suggests that the students were not particularly aware of how the language support that was being provided to them was intrinsic to helping them understand how the language of their discipline worked. hey were not necessarily aware of the importance of language in construing academic and discipline-speciic language. Although the SLATE team did try to raise their awareness of this by giving them information sheets, etc., it may be necessary for the project team to make the relationship between students’ control over the English language and their academic performance even more explicit. Another possible reason for the negative evaluation by some of the students might have to do with the online nature of the SLATE project. here was no personal face-to-face contact between the SLATE team and the students (or with the course lecturer). All communication was via e-mail and Blackboard. hus, it was diicult for the SLATE team to be aware of all the issues and questions that the students might have and/or what their reactions to the various resources and material were. While the SLATE team did try to predict student queries and responded to these in the support material that they sent to the students, it wasn’t always clear if the students fully comprehended these messages. To improve this situation, it might be useful to have a resource person on site who is familiar with the project and who can visit the classes, talk about the project to students, and answer any on-going questions, etc. In addition, more work could be done collaboratively with the course lecturers to motivate and engage the students in the lectures. Such work on building student understanding and motivation would hopefully result in greater engagement with the project, and thus have a stronger impact on their feelings about literacy intervention. he second problem that needs to be considered is that of the lecturers’ increased workload. his issue is again partly related to the lecturers’ lack of a clear understanding of the role(s) language plays in the construction of knowledge in their disciplines. he lecturers engaged in the project were not shy of the work demands, but they did not fully understand the underlying theory that shaped the SLATE project. Working more closely with the lecturers and providing more support and information to them could remedy this. his strategy might also be helpful in addressing the inal concern listed here. In view of the demands on the staf time and workloads, the SLATE resources Ahmar Mahboob, Angela Chan, Jonathan Webster 137 can be further reined and edited with feedback from the course lecturers so that they include responses to the queries that the lecturer typically receives from students. By doing this, we would be able to decrease the number of student queries and thus lower the demand on lecturer time. Better coordination and more collaboration between the SLATE team and the academic staf will enhance the project further and ofer better support to the students. In summary, the evaluation of the SLATE project showed that it achieved its intended outcomes – which were to provide embedded, discipline-speciic language and literacy support to the students. he large majority of the students participating in the project found the support material and the feedback useful and used it to develop their academic language and literacy. hese indings are very encouraging and show one way in which other courses and institutions may integrate the lessons learnt from the SLATE project in developing discipline-speciic language and literacy needs of their students. While there are a number of lessons that we have learnt from the project, one thing that has been strongly supported through this review is that an online genre-based approach to language support in tertiary environments is not only feasible, but successful. Acknowledgements he evaluation study of the LCC/SLATE project was supported by the English Language Enhanced Fund from the Hong Kong University Grants Committee. About the authors Ahmar Mahboob is Senior Lecturer in the Department of Linguistics at the University of Sydney. His research interests include educational linguistics, language teaching, language policy, teacher education, and World Englishes. Angela Chan is currently Assistant Professor at the University of Hong Kong. She worked as Senior Research Fellow at City University of Hong Kong when the LCC/SLATE evaluation study was conducted. She is interested in conversation analysis, interaction in social and workplace settings, and language learning and teaching. She has published in these areas in Language in Society, Journal of Pragmatics, Journal of Politeness Research, Journal of Asia Paciic Communication and Asia Paciic World. Jonathan J. Webster is Professor, Department of Chinese, Translation and Linguistics, and Director, he Halliday Centre for Intelligent Applications of Language Studies at the City University of Hong Kong. He is also the General Editor of the Equinox journal Linguistics and the Human Sciences and the editor (with Ruqaiya Hasan and Christian Matthiessen) of the two volume Continuing Discourse on Language: A Functional Perspective (Equinox, 2007). 138 Evaluating the SLATE project References Alderson, J. C., and McIntyre, D. (2006) Implementing and evaluating a self-assessment mechanism for the web-based Language and Style course. Language and Literature, 15 (3): 291–306. http://dx.doi.org/10.1177/0963947006066128 Allen, M. J. (2004) Assessing Academic Programs in Higher Education. Bolton, MA: Anker Publishing. Barton, A., Bragg, J. and Serratrice, L. (2009) ‘Discovering language’ in primary school: An evaluation of a language awareness programme. Language Learning Journal, 37 (2): 145– 164. http://dx.doi.org/10.1080/09571730902928029 Calder, J. (1994) Programme Evaluation and Quality: A Comprehensive Guide to Setting up an Evaluation System. London: Kogan Page. Cohen, A. D. (1987) Student processing of feedback on their compositions. In Anita Wender and Joan Rubin (eds), Learner Strategies in Language Learning, 57–69. Englewood Clifs, NJ: Prentice Hall International. Devrim, D. Y. (2013) Development of Grammatical Metaphor in Academic Literacy hrough Online Language Support. Unpublished PhD dissertation. Sydney: University of Sydney. Dreyfus, S. and Macnaught, L. (in press) Joint construction in the SLATE project. Linguistics and the Human Sciences 7 (2). Farr, F. (2008) Evaluating the use of corpus-based instruction in a language teacher education context: Perspectives from the users. Language Awareness 17 (1): 25–43. http://dx. doi.org/10.2167/la414.0 Ferris, D. R. (1995) Student reactions to teacher response in multiple-drat composition classrooms. TESOL 29 (1): 33‒53. http://dx.doi.org/10.2307/3587804 Güneş, T. and Dilek, N. S. (2009) Evaluation of science and technology program according to students' opinions. Procedia – Social and Behavioral Sciences 1 (1): 1396–1401. Halliday, M. A. K. (1985a) Introduction to Functional Grammar. London: Arnold. Hedgcock, J. and Lekowitz, N. (1994) Feedback on feedback: Assessing learner receptivity to teacher response in L2 composing. Journal of Second Language Writing 3 (2): 141–163. http://dx.doi.org/10.1016/1060-3743(94)90012-4 Humphrey, S., Martin, J., Dreyfus, S. and Mahboob, A. (2010) he 3×3: Setting up a linguistic toolbox for teaching and assessing academic writing. In A. Mahboob and N. Knight (eds) Appliable Linguistics: Texts, Contexts, and Meanings, 185–199. London: Continuum. Hyland, F. and Hyland, K. (2001) Sugaring the pill: Praise and criticism in written feedback. Journal of Second Language Writing 10 (3): 185–212. http://dx.doi.org/10.1016/ S1060-3743(01)00038-8 Hyland, K. and Hyland, F. (2006) Contexts and issues in feedback on L2 writing: An introduction. In Ken Hyland and Fiona Hyland (eds), Feedback in Second Language Writing: Contexts and Issues, 1–19. New York: Cambridge University Press. Light, V., Light, P. and Wright, V. (2000) Seeing eye to eye: An evaluation of the use of Ahmar Mahboob, Angela Chan, Jonathan Webster 139 video-conferencing to support collaboration. European Journal of Psychology of Education, XV (4): 467–478. http://dx.doi.org/10.1007/BF03172988 Lorenzo, F., Casal, S. and Moore, P. (2010) he efects of content and language integrated learning in European education: Key indings from the Andalusian bilingual sections evaluation project. Applied Linguistics 31 (3): 418–442. http://dx.doi.org/10.1093/applin/ amp041 Mahboob, A. (under review) Meeting the challenges of English medium higher education in Hong Kong. Mahboob, A. and Devrim, D. Y. (this issue) Supporting independent construction online: he role of feedback and revisions in the SLATE project. Linguistics and the Human Sciences, 7. Mahboob, A. and Devrim, D. Y. (2011) Providing efective feedback in an online environment. SPELT Quarterly 26 (3): 17–30. Mahboob, A., Dreyfus, S., Martin, J. and Humphrey, S. (2010) Appliable linguistics and English language teaching: Scafolding literacy in adult and tertiary environments (SLATE) project. In A. Mahboob and N. Knight (eds) Appliable Linguistics: Texts, Contexts, and Meanings. London: Continuum. Payne, D. A. (1994) Designing Educational Project and Program Evaluations: A Practical Overview Based on Research and Experience. Boston, MA/Dordrecht/London: Kluwer Academic Publishers. Rothery, J. and Stenglin, M. (1995) Exploring literacy in school English (Write it right resources for literacy and learning). Sydney: Metropolitan East Disadvantaged Schools Program. Rubenfeld, S., Sinclair, L. and Clement, R. (2007) Second language learning and acculturation: he role of motivation and goal content congruence. Canadian Journal of Applied Linguistics 10 (3): 309–323. Ryan, R. M. and Deci, E. L. (2000). Self-determination theory and the facilitation of intrinsic motivation, social development, and well-being. American Psychology 55 (1): 68–78. http://dx.doi.org/10.1037/0003-066X.55.1.68 Sengupta, S. and Leung, K. (2002) Providing English language support through collegial mentoring: How do we measure its impact? Assessment & Evaluation in Higher Education 27 (4): 365–381. http://dx.doi.org/10.1080/0260293022000001373 Schleppegrell, M. J. (2004) Technical writing in a second language: he role of grammatical metaphor. In L. J. Ravelli and R. A. Ellis (eds), Analysing Academic Writing: Contextualized Framework, 173–189. New York: Continuum.