Examining the nature and effects of feedback dialogue

Middlesex University Research Repository An open access repository of Middlesex University research ❤tt♣✿✴✴❡♣r✐♥ts✳♠❞①✳❛❝✳✉❦ Ajjawi, Rola and Boud, David ORCID: https://orcid.org/0000-0002-6883-2722 (2018) Examining the nature and effects of feedback dialogue. Assessment & Evaluation in Higher Education, 43 (7). pp. 1106-1119. ISSN 0260-2938 Final accepted version (with author’s formatting) This version is available at: ❤tt♣✿✴✴❡♣r✐♥ts✳♠❞①✳❛❝✳✉❦✴✷✺✽✶✻✴ Copyright: Middlesex University Research Repository makes the University’s research available electronically. Copyright and moral rights to this work are retained by the author and/or other copyright owners unless otherwise stated. The work is supplied on the understanding that any use for commercial gain is strictly forbidden. A copy may be downloaded for personal, non-commercial, research or study without prior permission and without charge. Works, including theses and research projects, may not be reproduced in any format or medium, or extensive quotations taken from them, or their content changed in any way, without first obtaining permission in writing from the copyright holder(s). They may not be sold or exploited commercially in any format or medium without the prior written permission of the copyright holder(s). Full bibliographic details must be given when referring to, or quoting from full items including the author’s name, the title of the work, publication details where relevant (place, publisher, date), pagination, and for theses or dissertations the awarding institution, the degree type awarded, and the date of the award. If you believe that any material held in the repository infringes copyright law, please contact the Repository Team at Middlesex University via the following email address: [email protected] The item will be removed from the repository while any claim is being investigated. See also repository copyright: re-use policy: ❤tt♣✿✴✴❡♣r✐♥ts✳♠❞①✳❛❝✳✉❦✴♣♦❧✐❝✐❡s✳❤t♠❧★❝♦♣② Accepted version of Ajjawi, R. and Boud, D. (2018). Examining the nature and effects of feedback dialogue, Assessment and Evaluation in Higher Education. 43, 7, 1106 - 1119 DOI: 10.1080/02602938.2018.1434128 Examining the nature and effects of feedback dialogue Rola Ajjawi1* and David Boud123 1 Centre for Research in Assessment and Digital Learning, Deakin University, Geelong, Australia 2 Faculty of Arts and Social Sciences, University of Technology Sydney, Australia 3 Work and Learning Research Centre, Middlesex University, London Abstract Research has conventionally viewed feedback from the point of view of the input, thus analysing only one side of the feedback relationship. More recently, there has be an increased interest in understanding feedback-as-talk. Feedback dialogue has been conceptualised as the dynamic interplay of three dimensions: the cognitive, the social-affective and the structural. We sought to explore the interactional features of each dimension and their intermediary effects on students. We analysed students’ feedback dialogue excerpts as cases using interactional analysis. Analysis involved iterative inductive and deductive coding and interpretation of feedback texts generated in an online course. The cognitive, social-affective and structural dimensions were interwoven within excerpts of feedback dialogue with effects on learners that extended beyond the immediate task (e.g. reframing of learners’ ideas, critical evaluation). The interactional features of each dimension include: cognitive (e.g. question asking, expressing oneself); social-affective (e.g. disclosure, expressing empathy); and structural (e.g. longitudinal opportunities for dialogue, invitational opportunities). The study provides evidence that strengthens the call for reconceptualising feedback as a dialogic and relational activity as well as supporting the view that dialogic feedback can be a key strategy for sustainable assessment. Keywords: feedback; dialogue; interactional analysis; higher education 1 Introduction One of the most pervasive critiques of feedback is related to the predominantly monologic way in which it is conceptualised and enacted in higher education (Nicol 2010; Boud and Molloy 2013; Evans 2013). Teachers can spend considerable time writing comments about assignments submitted at the end of a sequence of study, but students may have difficulty understanding and acting upon this information therefore limiting its contribution to their future development (Higgins, Hartley, and Skelton 2002; Sadler 2010; Winstone et al. 2017). Decoding feedback comments is described by students as akin to ‘learning a foreign language’ (Sutton and Gill 2010). Students have often progressed to the next module or assessment activity by the time they receive useful information about their work without opportunities or expectations of dialogue that assists them to construct their understanding in a meaningful way (Carless 2006). These critiques collectively highlight the limitations of information transmission feedback strategies to develop students’ capacities to regulate their learning and to meet their future learning needs. Hence, the call for sustainable and dialogic feedback. There is significant support for the notion that building lifelong learning capabilities should be a key function of higher education (Boud and Falchikov 2006). Sustainable assessment has been proposed as a way of conceptualising this purpose of assessment. It is defined as ‘assessment that meets the needs of the present without compromising the ability of students to meet their own future learning needs’ (Boud 2000, 152). In discussing the importance of sustainable assessment Hounsell (2007), proposed three key aspects of ‘high value’ feedback to achieve this. These were that feedback must: 1) carry impact beyond the task to which it relates; 2) enhance the student role to generate, interpret and engage with feedback; and 3) develop congruence between guidance and feedback by orchestrating teaching and learning environments in which productive dialogue arises. Carless et al. (2011, 397) define sustainable feedback as: ‘dialogic processes and activities which can support and inform the student on the current task, whilst also developing the ability to self-regulate performance on future tasks’. Importantly, dialogic feedback fits with the sustainable assessment agenda because it promotes student engagement with feedback (Price, Handley, and Millar 2011) and self-regulation of learning (Winstone et al. 2017). This study contributes to the empirical literature by exemplifying actual feedback dialogue and examining its intermediary effects on learners and staff involved in the dialogue. Literature review There is a considerable body of work that argues for feedback as dialogue (Nicol 2010; Carless et al. 2011; Boud and Molloy 2013; Ajjawi and Boud 2017), yet empirical research of actual feedback dialogue and its effects is limited. Research has primarily focused on measuring effects of feedback using learning outcomes as proxy and/or student satisfaction (Evans 2013). Otherwise, feedback research has typically coded comments provided by teachers using pre-specified codes derived from the literature (e.g. Brown and Glover 2006; Hughes, Smith, and Creese 2015). This research indicates that the majority of written feedback comments are at the level of task rather than self-regulation (Glover and Brown 2006; Orsmond and Merry 2011; Arts, Jaspers, and Joosten-ten Brinke 2016). In other words, written feedback comments tend to be oriented to how well the task was understood or performed rather than orienting towards prompting students’ self-regulatory behaviours (Hattie and Timperley 2007). One study, based on an analysis of tutor comments by external raters, proposed that feedback comments phrased as questions were more likely to stimulate 2 students’ reflection (Dekker et al. 2013). However, using input measures alone (e.g. timing, frequency, quantity or externally judged product quality) as a way of evaluating feedback effectiveness is problematic because coding the apparent intent of feedback information cannot capture the communicative nature of the phenomenon nor its actual effects on learners (Price et al. 2010). Recently, researchers have started to analyse feedback as interactional, taking into account all speakers and analysing actual feedback dialogue. One such study explored feedback dialogue in the medical workplace (Rizan et al. 2014). Using feedback-as-talk and discourse analysis they highlighted features of language, para-language and non-verbal communication that resulted in the incorporation of corrective feedback comments in their interaction and less loss of face for the student in front of the patient. Steen-Utheim and Wittek (2017) developed a model of feedback dialogue composed of four dimensions: 1) emotional and relational support, 2) maintenance of the dialogue, 3) expressing themselves, and 4) other’s contribution to individual growth. These dimensions shed light on how interaction in feedback dialogue might lead to learning, however, how dialogue is sustained interactionally in order to have these effects is unknown. Also taking an interactional perspective, Esterhazy and Damşa (2017) describe dialogic feedback processes as ‘emerging meaning-making trajectories’ that involve interpretation. They argued that engagement with feedback requires moving between local and wider disciplinary contexts as well as moving between declarative and procedural knowledge. These two dimensions formed the knowledge space within which epistemic meaning-making occurred. The perspective on developmental trajectories and the complexity offered by their model where feedback enables engagement with the broader contexts of learning is refreshing. Considering the theorised value of feedback dialogue, there is a lack of empirical research identifying the interactional features that sustain feedback dialogue. Conceptual framework In our previous work, we put forward a conceptual framework for feedback dialogue arguing that interactional analysis provides a rich lens through which to study feedback dialogue (Ajjawi and Boud 2017). This conception is informed by Interaction Analysis (Jordan and Henderson 1995), an interdisciplinary approach for researching human–human and human–material interactions, that views knowledge and action as fundamentally social in origin, and rooted in particular social and material ecologies. The goal of Interaction Analysis is to identify how ‘participants utilize the resources of the complex social and material world of actors and objects within which they operate’ (Jordan and Henderson 1995, 41). Meaning is always created in dialogue and implies at least two voices (Wegerif 2008). Similarly, learning occurs through feedback dialogue and involves students’ interpretive meaning making about the comments (Steen-Utheim and Wittek 2017). Therefore, in this study we adopt the perspective that feedback is a communicative act and a social process in which power, emotion and discourse impact on how messages are constructed, interpreted and acted upon. One model of feedback dialogue identifies three key dimensions: cognitive (i.e. content), social-affective (i.e. relational) and structural (i.e. curriculum organisation), that dynamically interact within the interaction space (Yang and Carless 2013). The cognitive and social-affective dimensions relate mainly to what teachers and students do within specific learning environments, whilst the structural dimension includes features that are both within and outside the immediate influence of students and teachers. Yang and Carless propose that these three dimensions need to be considered in relation to one another when analysing feedback dialogue. The model highlights the dynamic nature of feedback dialogue and begins to unpack its constituent components. Hence, in this qualitative study we sought to identify the interactional features of each of the cognitive, social-affective and structural dimensions 3 of feedback dialogue from an online course in order to understand how feedback dialogue can be sustained and how it effects student responses in particular self-regulatory behaviours. We used interactional analysis and in the paper present illustrative excerpts to showcase the interplay between these three dimensions of feedback dialogue. Methods Context The study was conducted in a postgraduate online distance course in medical education. Students are typically medical doctors completing the course whilst working as clinicians and medical educators. The current paper utilises data generated from a wider study: the Interactive Assessment and Collaboration via Technology (interACT) project (see Barton et al. 2016). Ethical approval for this study was obtained. InterACT involved the design and embedding of an interactive cover page and longitudinal feedback journal in order to promote feedback dialogue (Barton et al. 2016). The assignment cover page was used to prompt students to evaluate their work qualitatively against the assignment’s criteria, to request specific feedback information from tutors and identify how previous information had informed their current work. Once the assignment and completed cover page were submitted, tutors provided feedback information not only on the assignment but also in response to each student’s evaluation and comments, and hence took part in dialogue on the cover page. Students then uploaded their marked assignments (including the cover page and feedback comments) into their longitudinal feedback journal where they answered questions relating to their interaction with and understanding of the comments they received. The feedback journal was tailored to offer a template for each assignment in the program where students answered four questions that prompted a comparison of the students’ self-evaluation with the tutor’s comments, asked them to reflect on what they had learned, actions to take and if anything was unclear from the feedback process. The relevant tutor was automatically alerted via email when a student posted comments into their longitudinal feedback journal, and they continued feedback dialogue asynchronously together as required. The length of dialogue varied depending on the nature of the students’ comments and if there were questions that needed following up. The feedback journal acted as a repository of all the assessment products, student reflections and feedback dialogue. Data collection Ten students enrolled in the first two core modules and the staff involved in assessing their work took part in the study. These cases provided sufficient depth for initial testing due to the richness of the analytical approach. Specific feedback dialogue excerpts were identified with a single cycle feedback loop defined as an initiation-response interaction between student, tutor and materials. Three actors were present: the tutor, the student and the materials through which feedback dialogue is mediated. Although the notion of the material artefact as actor that ‘takes a turn’ may be perceived as unusual, in interaction analysis turn-taking encompasses more than talk, as participation in an interactional exchange may be constituted by artefacts (Jordan and Henderson 1995). We then extracted and de-identified excerpts of feedback dialogue from students’ assignments, associated cover pages and feedback journal entries. These data files were converted to rtf files and imported into ATLAS.ti for analysis. Data analysis The dialogue excerpts were coded using a mixture of inductive and deductive coding with an interpretative approach informed by interactional analysis. Both authors met severally to negotiate coding and to discuss the developing analytical framework and interpret meaning in relation to our research questions. Across several meetings we worked on developing the analytical coding frame in order to explore the interactional features of the 4 cognitive, social-affective and structural dimensions and their effects. This was an iterative process from feedback literature and theory to data and back again preserving context within the feedback excerpts. First, in order to access the cognitive dimension of feedback-as-talk, the dialogue excerpts were coded deductively using Hattie and Timperley's (2007) model to identify the nature of the feedback comments as related to task, process, self and self-regulation. We also specifically coded the intermediary effects based on the students’ responses relating to: 1) self-evaluation, 2) further engagement in the task, 3) learning about self as learner, 4) learning about learning, and 5) continuing beyond the task (Hattie and Timperley 2007). We chose to use these criteria as markers for the self-regulatory effects of the feedback dialogue on students rather than other models of self-regulation due to their relevance to sustainable assessment and their prominent place in the feedback literature. Second, accessing the social-affective dimension in a meaningful way was more challenging. Here we turned to sociological constructs that take account of the linguistic aspects of communication as well as the social function of talk. In social interactions, participants are typically involved in the ritual maintenance of face (Goffman 1959). Feedback talk being related to judgements about individuals’ work may be seen to be face threatening. Hedges are linguistic devices which can be used to soften criticism and maintain face and interpersonal relations. Hedging includes words such as maybe, sometimes, seemed or a little (Hyland 2001). Pronouns may also be used to influence the impressions of other parties by constructing a proximal or distal sense to their behaviours through talk. For example, the use of I communicates self-agency and intentionality, while we emphasises one’s self as part of a group and you may be distancing indicating others outside the conversational space (Mercer 2004). Such linguistic features can provide important clues into how feedback is being interpreted and its effects on learners and teachers. Third, in order to access the structural dimension we took a helicopter view of the curricular conditions within which the dialogue was created taking into account understandings from the feedback literature. The first author was an insider of the curriculum development team at the time and so was very familiar with the curriculum design features, whilst the second author is published in this field. Findings We found that episodes of feedback dialogue extended from the cover page to the feedback journal for a single assignment, across assignments in a single module and even across modules. Dialogue was initiated by the prompt questions in the materials namely the cover page and feedback journal or by tutors in the margin of the assignment or on the cover page. All five criteria for self-regulatory feedback effects, highlighted above, were also present in our data. Each of our feedback excerpts contained the three dimensions and we identified specific features of each dimension from the data (see Table 1). The cognitive, social-affective and the structural dimensions were dynamically intermingled in the feedback dialogue to prompt self-regulatory behaviours. All interactional features were not necessarily present in each episode of dialogue but the three dimensions were. <insert table 1> We now present three specific excerpts and their interpretations. We do this to highlight the dynamic interplay between the cognitive, social-affective and structural dimensions and show nuanced features of dialogue that prompt students’ self-regulation. The excerpts chosen identify three ways in which feedback dialogue may extend beyond the task, and hence fit the sustainable feedback agenda: 1) induction into the broader educational discourse; 2) reflection on self as student and educator; 3) advice on utilisation of the assessment artefact in the workplace. These excerpts are presented in table format, each 5 displaying the turns taken for the particular dialogue interaction, the actors involved (pseudonyms used) and direct excerpts of the dialogue text. Excerpt 1: induction into the broader educational discourse The assignment instructions were: ‘Develop a one-sheet peer evaluation sheet. Use this to evaluate a lecture...’ This excerpt involves two loops of feedback dialogue, it was tutor-initiated in the margin of the assignment (turn 2) and extends from the assignment to the feedback journal. Interchanges in this excerpt (Table 2) are conversational and informal. The dialogue goes beyond the immediate requirements of the assignment starting within the local context and then moving between the local and the broader contexts. The tutor, Julie, foreshadows this in Turn two with a question about educational decision making and a shift in pronoun to the inclusive: ‘we’ and ‘our’ (turn 2). The shift in pronoun from ‘I’ to ‘we’ can be seen to imply solidarity between participants (Brown and Levinson 1987). Melanie reflects on her choices and then elaborates on her reasons why she chose that online lecture (turn 4). She shifts to the broader context picking up on the inclusive ‘we’ to elaborate on motivations to view online lectures generally (turn 4). Julie invites Melanie into a broader conversation occurring within the discipline of medical education about the value of putting lectures online when she highlights the differing views on this within the community: ‘yet some colleagues think putting lectures up online will toll the death knell of face to face lectures’ (turn 5). Julie then self-discloses her views by admitting to attending live theatre beamed to the local cinema screen paralleling the conversation about attending lectures live versus viewing them online. Melanie continues with elaborating her position and meaning making in relation to the value of attending lectures live (e.g. forming networks) and closes the dialogue with a friendly comment ‘have fun at the opera!’ that acknowledges the tutor’s disclosure (turn 6). This feedback conversation takes on a broader purpose than correction. As can be seen in the student response, the feedback dialogue leads to self-evaluation (why she chose the particular resource), continuing beyond the task (dialogue about online lectures) and learning about learning (the value of social learning). <insert table 2> The three dimensions of cognitive, social-affective and structural are interwoven in this excerpt in the following ways. The cognitive dimension is initially elicited through questions from the tutor related to the student’s motivation to attend a specific lecture (relevant to the task at hand) then as the dialogue continues shifting to a broader conversation about the social aspects of learning. The social-affective dimension is sustained through personal disclosure, informal language choice and use of inclusive pronouns evoking a common educational identity. The structural dimension through the feedback journal allows the student to respond to a comment on her assignment and for an asynchronous dialogue to continue beyond the task. Excerpt 2: reflection on self as student and educator The following excerpt (Table 3) involves two loops of feedback dialogue, initiated by the cover page extending to the feedback journal. The dialogue which occurred between the tutor Julie and learner Freda is initiated by a question on the cover page intended to prompt self-evaluation and invite dialogue (turn 1). The learner, in turn 2, is seeking reassurance and expresses frustration ‘!!’ and a loss of confidence. Julie starts with praise about the assignment ‘well handled’ then takes this opportunity to extend Freda’s learning while also providing emotional support. She does this through asking questions that prompt Freda’s selfevaluation and re-application to her own context (turn 3). Julie also acknowledges Freda’s feelings through encouragement ‘Hope this brings back a little confidence’ mirroring the 6 word ‘little’ potentially to save face and promote trust. The materials in turn 4 enable Freda to elaborate the judgements about her work ‘did not recognise some of the good work I did’ (turn 5). As can be seen in her response, the feedback dialogue leads to self-evaluation (of why she felt frustrated and her judgements about her work), learning about self as learner (recognising her own response to the task) and learning about learning (the role of emotions in learning and how she might support her own students) (turn 5). There is also reciprocal learning action by the tutor in terms of adjusting the assessment instructions to explain the imposed restrictive word limit and acknowledgement of the dialogue in closing it down (turn 6). <insert table 3> The cognitive dimension is sustained through elaboration, learning and learning and self-evaluation. The social-affective dimension involves self-disclosure by the students when she articulates her loss of confidence with the task using emotional talk (e.g. suffering, struggling) and seeks reassurance. The tutor takes this opportunity to go beyond praise, acknowledging the motivational value of feedback as well as prompting a self-regulatory and cognitive re-evaluation linking with Freda’s role as an educator. This move potentially minimises loss of face from Freda as she can then focus on her own students and reflect on her learning as an educator. The structural dimension allows for sustained asynchronous dialogue across the cover page and feedback journal and creates opportunity for Freda to express herself. Excerpt 3: utilisation of the assessment artefact in the workplace As before, the assignment instructions were: ‘Develop a one-sheet peer evaluation sheet. Use this to evaluate a lecture...’ The following excerpt (Table 4) involves two loops of feedback dialogue, initiated by the cover page extending to the feedback journal. For this particular task, Paul developed a peer evaluation sheet which he used to evaluate a peer’s lecture. The dialogue which occurred between the tutor Julie and learner Paul is initiated by a question on the cover page intended to prompt self-evaluation and invite dialogue (turn 1). The learner, in turn 2, asks whether there are any improvements which he could make to this tool before using it in his workplace. Julie responds to the question by asking him to reflect on the merits of the tool having used it for the assignment. She displays some ambiguity about her interpretation of his question providing tips on how it could be adapted to suit Paul’s needs (turn 3). Her response highlights different factors involved in educational design processes. Julie prompts his reevaluation of the tool, which he does and reports back on in turn 5 in the feedback journal bringing in his experiential knowledge and shows re-engagement with the merits of the tool. Julie acknowledges his reflections in Turn 6 with a reminder about the design matching the purpose. <insert table 4> The cognitive dimension is elicited through a question on the cover with the effect of Paul re-evaluating his experience with the tool (the assessment product) and with previous evaluation tools he has used more broadly, continuing beyond the task by applying what he is learning to his own educational practice. It seems that question asking by the tutor and the materials plays a role in eliciting Paul’s response. The social-affective dimension is sustained through respectful listening and responding. The dialogue here remains on point, with few personal markers. The structural dimension enables Paul’s responses to be tracked from the assignment to the effects of the feedback. Discussion 7 Findings from this study highlight the interdependence of the cognitive, socialaffective and structural dimensions in the feedback dialogue space and the subsequent effects on students’ self-regulatory behaviours moving beyond the immediate task. Our findings lend empirical support to the utility of the framework proposed by Yang and Carless (2013). Specifically, we extend their work by highlighting specific interactional features of dialogue within each dimension and the intermediary effects of these on learners. We also extend our own work in showing how interactional analysis may be used to examine feedback dialogue and its effects rather than focusing on feedback as input, hence recasting feedback as a social act involving learners, tutors, contexts and relationships (Ajjawi and Boud 2017). The cognitive dimension Interactional features of the cognitive dimension included question asking, expressing oneself, encouraging reframing of ideas, promoting critical evaluation and engagement beyond the task. Within the excerpts of feedback dialogue we saw how students’ selfregulatory behaviours may be mobilised or supported. We hypothesise that question-asking within the dialogue plays an important role in shifting beyond the immediate task to a more self-regulatory frame. The questions posed by the tutor and the materials created opportunities for students to reflect on their development and to articulate their own understanding and interpretations of the feedback comments which are then made available for the tutor to further comment on (enabled and sustained through the structural and socialaffective dimensions). This supports Dekker et al.’s (2013) hypothesis that question asking and using a positive tone are more likely to prompt reflection. This is perhaps not surprising as dialogue which prompts reflection is an important factor in the development of selfregulatory practices (Orsmond et al. 2013) with self-reflection being an important phase of self-regulation of learning (Zimmerman 2002). The participants who engaged with feedback at the self-regulatory level often naturally, or through prompting, considered their roles and work as medical educators in relation to the study material. The evocation of role or identity within the feedback dialogue can be seen in the excerpts above when the student was inducted into the broader disciplinary discourse of online lectures, through reflection on self as student and educator, or in scaffolding the use of an assessment artefact in the workplace. Self-regulatory feedback dialogue might be one approach that can be purposefully used to promote bridging of learning to the workplace and to induct newcomers into the broader academic discourses of their discipline. Few have highlighted the role of dialogic feedback in the professional socialisation of students into a discipline (Molloy 2009; Eriksson and Mäkitalo 2015; Esterhazy and Damşa 2017). This bodes well for a sustainable assessment agenda. The social-affective dimension The interactional features of the social-affective dimension included linguistic strategies (e.g. use of inclusive pronouns, mirroring), self-disclosure, acknowledging student emotional responses and providing social support. Yang and Carless (2013) argued that the social affective dimension – for establishing and maintaining social relationships – is foundational to the content of the feedback not being derailed. Indeed, students’ judgements about the strength of the educational relationship and how trusting they are of their tutor’s intentions seem to have an influence on their acceptance and use of feedback in the present and future (Telio, Regehr, and Ajjawi 2016). Interaction is important for building trust, this involves a mutual investment in faith, as well as respectful and empathic listening and responding (Carless 2013). The sharing of (relevant) personal information and experiences between staff and students indicates that trust was fostered in some of these interactions, however without asking students about this directly we cannot be certain. Our tutor, Julie, explicitly acknowledged students’ emotional responses e.g. loss of confidence, showing empathy and providing support and encouragement. Loss of confidence 8 may have a detrimental effect on students acting on feedback, indeed research shows that emotional reactions play a significant part in determining how students will act on the feedback they receive (Pitt and Norton 2017). Others have also highlighted the importance of such attributes and behaviours for effective feedback dialogue (Steen-Utheim and Wittek 2017). The structural dimension The interactional features of the structural dimension is through the particular questions in the cover page and the longitudinal feedback journal that provided space for the dialogue to continue. The questions in the cover page acted as an external prompt to encourage self-regulatory actions such as self-evaluation and feedback seeking. The design also enabled learners to seek feedback that addresses their goals and to continue a dialogue with their tutor. Arguably, if a student has sought feedback about a specific aspect of their work they are more invested in the response. The authentic nature of assignments with learners who are also embedded in the workplace (and studying part-time) means that these learners have opportunities to apply and think about the content of the module in relation to their own work responsibilities and contexts also potentially a motivating factor for engagement in feedback. Adaptation of the context, curriculum, assessment and feedback materials appears to have initiated opportunities for ‘high value’ dialogic feedback (as per Hounsell 2007) to occur rather than remaining task focused. In the previous feedback approach self-regulatory feedback questions would have ended on the piece of work itself with the hope that the feedback questions would prompt the intended reflection by the learner. Making feedback dialogic with explicit self-regulatory prompts, enabled follow up of the effects of these questions through seeing whether there was initial uptake or not. Some learners are likely to be quite adept at actively self-regulating their learning and the educational design here makes it explicit to the tutors, while for other learners their self-regulation strategies may remain passive and implicit requiring such explicit scaffolding to bring these strategies to the surface for both the learners and their tutors to see. We should be cautious in assuming that these features will lead to a positive effect on student learning. They are important design elements but in themselves they do not guarantee learning. Our findings highlight that designing the structural dimension alone is not sufficient and that the cognitive and social-affective dimensions should be considered to sustain dialogue. Invitations to dialogue need to be taken up by the student. And, for the dialogue to be maintained, the responses of the student must in turn be taken up by the tutor. Conclusion Our research shows that feedback dialogue is enabled through the interplay of the cognitive, social-affective and structural dimensions, where all three are necessary for sustained dialogue. Further, this paper exemplifies interactional features of actual feedback dialogue and shows student and tutor responses to the dialogue including engagement beyond the immediate task. This heralds a shift from feedback being about hopefully useful information to a dialogic process that enables tracking the intermediary effects of feedback. It would be inappropriate to ignore the real time implications of such curriculum redesign on the tutors in particular in relation to workload. However, this extra burden may be offset through efficiencies in rationalising the total number of assignments that are to be reviewed and through shifting the feedback to earlier in the process. Because the design enabled tutors to see the effects of their feedback comments and promoted interaction with students within a distance course, this was found to be more satisfying for tutors than information transmission (Barton et al. 2016) and hence we argue more likely to be sustainable. Furthermore, our study demonstrates the potential for tutors to learn from the 9 feedback interactions with their learners. These strategies may be useful in implementing feedback dialogue in courses with deadlines and shorter turnaround times. A strength of this study is that it deals with real feedback events primarily designed for pedagogic rather than research purposes. The effect of the feedback on both teacher and learner are apparent through the dialogue so that an evaluation of the feedback effect can be interpreted. The effects shown are intermediary in the sense that they show engagement with the feedback and cognitive and/or affective reframing, what we cannot see in the data is the longer-term effect of how feedback is used in future work. The written nature of our data means more sophisticated forms of para-linguistic interaction are generally absent. Further research should explore patterns of linguistic use that promote cognitive, social-affective and structural dimensions of interaction in relation to longitudinal (sustainable) effects on students. Acknowledgements We would like to acknowledge Jisc for their funding, and the students and staff who took part in the study. Thanks to Karen Barton who collected the data, Jo Tai, Phill Dawson, Margaret Bearman and Rachelle Esterhazy for their comments on earlier versions of this paper. Disclosure statement No potential conflict of interest was reported by the authors. Funding This work was done by Joint Information Systems Committee [grant number 5/11 – Assessment and Feedback Programme]. Notes on contributors Rola Ajjawi is a Senior Research Fellow at the Centre for Research in Assessment and Digital Learning at Deakin University. She holds a Bachelor's Honours degree in physiotherapy and worked as a physiotherapist and clinical educator before moving into academia. She has since led a program of research centred on workplace learning, with a particular interest in assessment and feedback. Current projects include: feedback that makes a difference, academic failure and persistence, and what makes for successful research environments. David Boud is Professor and Foundation Director of the Centre for Research in Assessment and Digital Learning, Deakin University, Professor of Work and Learning at Middlesex University and Emeritus Professor at the University of Technology Sydney. He has been a pioneer in developing learning-centred approaches to assessment across the disciplines, particularly in student self-assessment, building assessment skills for long-term learning and new approaches to feedback. References Ajjawi, R., and D. Boud. 2017. "Researching feedback dialogue: an interactional analysis approach." Assessment & Evaluation in Higher Education 42 (2):252-65. doi: 10.1080/02602938.2015.1102863. Arts, J. G. , M. Jaspers, and D. Joosten-ten Brinke. 2016. "A case study on written comments as a form of feedback in teacher education: so much to gain." European Journal of Teacher Education 39 (2):159-73. 10 Barton, K. L., S. J. Schofield, S. McAleer, and R. Ajjawi. 2016. "Translating evidence-based guidelines to improve feedback practices: the interACT case study." BMC Medical Education 16 (1):1-12. doi: 10.1186/s12909-016-0562-z. Boud, D. 2000. "Sustainable Assessment: Rethinking assessment for the learning society." Studies in Continuing Education 22 (2):151-67. doi: 10.1080/713695728. Boud, D., and N. Falchikov. 2006. "Aligning assessment with long-term learning." Assessment & Evaluation in Higher Education 31 (4):399-413. doi: 10.1080/02602930600679050. Boud, D., and E. K. Molloy. 2013. "Rethinking models of feedback for learning: the challenge of design." Assessment & Evaluation in Higher Education 38 (6):698-712. doi: 10.1080/02602938.2012.691462. Brown, E., and C. Glover. 2006. "Evaluating written feedback." In Innovative Assessment in Higher Education, edited by C. Bryan and K. Clegg, 81-91. Oxford: Routledge. Brown, P., and S. C. Levinson. 1987. Politeness: Some Universals in Language Usage. Cambridge: Cambridge University Press. Carless, D. 2006. "Differing perceptions in the feedback process." Studies in Higher Education 31 (2):219-33. doi: 10.1080/03075070600572132. Carless, D. 2013. "Trust and its role in facilitating dialogic feedback." In Feedback in Higher and Professional Education, edited by David Boud and Elizabeth Molloy, 90-103. Abingdon: Routledge. Carless, D., D. Salter, M. Yang, and J. Lam. 2011. "Developing sustainable feedback practices." Studies in Higher Education 36 (4):395-407. doi: 10.1080/03075071003642449. Dekker, H., J. Schönrock-Adema, J. W. Snoek, T. van der Molen, and J. Cohen-Schotanus. 2013. "Which characteristics of written feedback are perceived as stimulating students’ reflective competence: an exploratory study." BMC Medical Education 13 (1):1-7. doi: 10.1186/1472-6920-13-94. Eriksson, A., and A. Mäkitalo. 2015. "Supervision at the outline stage: Introducing and encountering issues of sustainable development through academic writing assignments." Text & Talk 35 (2):123-53. Esterhazy, R., and C. Damşa. 2017. "Unpacking the feedback process: an analysis of undergraduate students’ interactional meaning-making of feedback comments." Studies in Higher Education:1-15. doi: 10.1080/03075079.2017.1359249. Evans, C. 2013. "Making Sense of Assessment Feedback in Higher Education." Review of Educational Research 83 (1):70-120. doi: 10.3102/0034654312474350. Glover, C., and E. Brown. 2006. "Written Feedback for Students: too much, too detailed or too incomprehensible to be effective?" Bioscience Education 7 (1):1-16. doi: 10.3108/beej.2006.07000004. Goffman, E. 1959. The Presentation of Self in Everyday Life. London, UK: Penguin Books. Hattie, J., and H. Timperley. 2007. "The Power of Feedback." Review of Educational Research 77 (1):81-112. doi: 10.3102/003465430298487. Higgins, R., P. Hartley, and A. Skelton. 2002. "The Conscientious Consumer: Reconsidering the role of assessment feedback in student learning." Studies in Higher Education 27 (1):53-64. doi: 10.1080/03075070120099368. Hounsell, D. 2007. "Towards more sustainable feedback to students " In Rethinking Assessment in Higher Education: Learning for the Longer Term, edited by D. Boud and N. Falchikov, 101-13. London: Routledge. Hughes, G., H. Smith, and B. Creese. 2015. "Not seeing the wood for the trees: developing a feedback analysis tool to explore feed forward in modularised programmes." 11 Assessment & Evaluation in Higher Education 40 (8):1079-94. doi: 10.1080/02602938.2014.969193. Hyland, F. 2001. "Providing Effective Support: Investigating feedback to distance language learners." Open Learning: The Journal of Open, Distance and e-Learning 16 (3):23347. doi: 10.1080/02680510120084959. Jordan, B., and A. Henderson. 1995. "Interaction Analysis: Foundations and Practice." The Journal of the Learning Sciences 4 (1):39-103. Mercer, N. 2004. "Sociocultural discourse analysis: analysing classroom talk as a social mode of thinking." Journal of Applied Linguistics 1 (2):137-68. Molloy, E. K. 2009. "Time to Pause: Feedback in Clinical Education." In Clinical Education in the Health Professions, edited by C. Delaney and E. K. Molloy. Sydney: Elsevier. Nicol, D. 2010. "From monologue to dialogue: improving written feedback processes in mass higher education." Assessment & Evaluation in Higher Education 35 (5):501 - 17. Orsmond, P., S. J. Maw, Julian R. Park, Stephen Gomez, and Anne C. Crook. 2013. "Moving feedback forward: theory to practice." Assessment & Evaluation in Higher Education 38 (2):240-52. doi: 10.1080/02602938.2011.625472. Orsmond, P., and S. Merry. 2011. "Feedback alignment: effective and ineffective links between tutors’ and students’ understanding of coursework feedback." Assessment & Evaluation in Higher Education 36 (2):125-36. doi: 10.1080/02602930903201651. Pitt, E., and L. Norton. 2017. "‘Now that’s the feedback I want!’ Students’ reactions to feedback on graded work and what they do with it." Assessment & Evaluation in Higher Education 42 (4):499-516. doi: 10.1080/02602938.2016.1142500. Price, M., K. Handley, and J. Millar. 2011. "Feedback: focusing attention on engagement." Studies in Higher Education, 36 (8):879-96. Price, M., K. Handley, J. Millar, and B. O'Donovan. 2010. "Feedback: all that effort, but what is the effect?" Assessment & Evaluation in Higher Education 35 (3):277-89. doi: 10.1080/02602930903541007. Rizan, C., C. Elsey, T. Lemon, A. Grant, and L. V. Monrouxe. 2014. "Feedback in action within bedside teaching encounters: a video ethnographic study." Medical Education 48 (9):902-20. doi: 10.1111/medu.12498. Sadler, D. R. 2010. "Beyond feedback: Developing student capability in complex appraisal." Assessment and Evaluation in Higher Education 35 (5):535-50. doi: 10.1080/02602930903541015. Steen-Utheim, A., and A. L. Wittek. 2017. "Dialogic feedback and potentialities for student learning." Learning, Culture and Social Interaction. doi: https://doi.org/10.1016/j.lcsi.2017.06.002. Sutton, P., and W. Gill. 2010. "Engaging feedback: meaning, identity and power." Practitioner Research in Higher Education 41 (1):3-13. Telio, S., G. Regehr, and R. Ajjawi. 2016. "Feedback and the educational alliance: examining credibility judgements and their consequences." Medical Education 50 (9):933-42. doi: 10.1111/medu.13063. Wegerif, R. 2008. "Dialogic or dialectic? The significance of ontological assumptions in research on educational dialogue." British Educational Research Journal 34 (3):34761. Winstone, N. E., R. A. Nash, M. Parker, and J. Rowntree. 2017. "Supporting learners' agentic engagement with feedback: A systematic review and a taxonomy of recipience processes." Educational Psychologist 52 (1):17-37. doi: 10.1080/00461520.2016.1207538. 12 Yang, M., and D. Carless. 2013. "The feedback triangle and the enhancement of dialogic feedback processes." Teaching in Higher Education 18 (3):285-97. doi: 10.1080/13562517.2012.719154. Zimmerman, B. J. 2002. "Becoming a Self-Regulated Learner: An Overview." Theory Into Practice 41 (2):64. 13 Table 1: Dimensions and interactional features of feedback dialogue Dimensions Interactional Features Cognitive Question asking Enabling students to elaborate Encouraging reframing of ideas Promoting critical evaluation Engaging beyond the task Social-affective Linguistic strategies (e.g. mirroring, inclusive pronoun use) Expressing empathy Disclosure Acknowledging student emotional responses Providing support and encouragement Structural Invitations to the dialogue Opportunities for sustained dialogue longitudinally Sufficient time for dialogue to occur before next task (for students and staff) Invitations for learners to seek feedback Authentic / contextualised assessment 14 Table 2 Excerpt of feedback dialogue: inducting into the broader educational discourse Turn 1 Name Learner: Melanie Date 31/08/12 Context T&L 4 text in the assignment The tutor adds a comment into the assignment margin Feedback journal 2 Tutor: Julie 18/09/12 3 Course materials 18/09/12 4 Melanie 18/09/12 Feedback Journal 5 Julie 18/09/12 Feedback Journal 6 Melanie 19/09/12 Feedback Journal Quote ‘As a refresher for the purposes of this evaluation, I have viewed the lecture online a second time more recently’ ‘How did you know where to find the link? … Would you have reviewed it if you weren’t doing an evaluation? If we want people to view online lectures [generally] do we need to provide them with a task or do we rely on internal motivation? What might our decision rely on?’ ‘What did you learn from the feedback process?’ ‘- re: comment 5 -- I had known that the lectures from this particular conference are uploaded annually, and I would have reviewed it even outside the context of this assignment. However, I did review this lecture specifically for this assignment as well … if we wanted people to view online lectures, how we compel them to do so would depend on what tends to motivate the learners, and how motivated we anticipate them to be on the whole …’ ‘Hi Melanie thanks for your feedback. I think then another thing to ponder is why you attended the lecture in person knowing it was online. I have my own thoughts, relating to the social aspect of learning … yet some colleagues think putting lectures up online will toll the death knell of face to face lectures. Of course if that were the case again perhaps we need to re-examine why we put lectures on. Julie (who is going to an opera beamed live from the MET [Metropolitan Theatre, London] to our local cinema [in Dundee] next month)’ ‘Yes, I think you're quite right about the social aspect of learning. To me, attending these conferences is a way to keep in touch with colleagues and mentors, and to gain their insights on clinical and professional matters. I also feel that you appreciate some nuances by attending the lecture live that are harder to appreciate with only audio … Have fun at the opera!’ 15 Table 3 Excerpt of feedback dialogue: reflection on self as student and educator Turn 1 Name Course materials Date 30/08/12 Context T&L3 cover page Quote ‘Which aspect(s) of your assignment would you specifically like feedback on?’ 2 Learner: Freda 30/08/12 T&L3 cover page 3 Tutor: Julie 17/09/12 T&L3 cover page 4 18/09/12 5 Course materials Freda Feedback journal Feedback journal ‘Mostly whether I have succeeded in answering the question, or whether I have focused too much on other less relevant things … I’m very happy with the feedback to date and I was gaining a little more confidence with each assignment, until now!!’ ‘Well handled. Hope this brings back a little confidence, though do reflect on how your confidence was dented and what effect this had on your work. Does this apply to your students? Did I give you enough guidance? What in particular was so stressful? Was it taking you out of your comfort zone? Is that always a bad thing?’ ‘What did you learn from the feedback process?’ 6 Julie 18/09/12 18/09/12 Feedback journal ‘I felt I had taken a long time to get my head around this assignment … as a result I think my confidence was a little dented from the start and I perhaps did not recognise some of the good work I did in relation to the final assignment. I was interested to note the comment regarding confidence and how this might affect my own students and this is something I will take on board. If I am able (as a tutor) to identify students with a lack of confidence early on, perhaps I can reassure and encourage them in the hope that they can still produce their best work, rather than their work suffering as a result of dented confidence … Useful feedback, thanks.’ ‘Hi Freda, perhaps I need to include in the assignment the rationale for such a challenging word count. I enjoyed reading your reflection. Julie’ 16 Table 4: Excerpt of feedback dialogue: utilisation of the assessment artefact in the workplace Turn Name Date Context Quote 1 Course materials Learner: Paul 19/08/12 T&L4 cover page T&L4 cover page 3 Tutor: Julie 04/09/12 4 Course materials ‘Which aspect(s) of your assignment would you specifically like feedback on?’ I adapted this feedback form from the one I currently use, in itself an amalgamation of multiple ones I’d seen before. I’d like any tips on how it can be improved before I start using it in teaching sessions. The best way to evaluate the tool itself is to use it as you have here (as the observer) and also as the teacher. In this exercise you have been asked to use it to think about ways you could improve your own lectures. Do you think the exercise fulfilled its remit? If using it to evaluate a student’s presentation skills make sure it reflects the objectives (e.g. if asking questions is one of the criteria, you would need to include how effectively questions were handled, if keeping to time was a criteria adding a start and end time would be useful). A careful balance needs to be made between ease of use and usefulness. ‘What actions, if any, will you take in response to the feedback process?’ 5 Paul Date notdocumen ted Feedback journal I've taken another look at the form and will refine it a little. I'd previously used different models with a 3 & 7 point scales but they were either too simplistic or, in case of the latter, had a lot of ambiguity. 6 Julie Date notdocumen ted Feedback Journal Hi Paul, I like your reflections. Remember, this tool is for you to use yourself so does not need the rigour of e.g. a student evaluation form. Julie 2 19/08/12 T&L4 cover page Feedback journal 17