EDINchapter

5 From Pivot to Foothold: Facilitating an Institutional Approach to Online Teaching and Learning at Ireland’s First Technological University Author Details Frances Boylan, Derek Dodd, Jennifer Harvey, Ana Elena Schalk, City Campus Learning Teaching and Technology Centre, TU Dublin, Ireland Corresponding Author [email protected] Abstract Over the spring and summer period of 2020, the Learning, Teaching and Technology Centre (LTTC) at Technological University Dublin (TU Dublin) employed a range of strategies to help staff respond to the unique challenges associated with an initial period of ‘emergency remote teaching’ and a succeeding ‘temporary online pivot’ (Nordmann et al., 2020, p. 4) precipitated by the COVID-19 pandemic. During this period the ‘TU Dublin VLE Baseline Checklist’, was designed to provide non-prescriptive, VLE-agnostic, clear, actionable guidance for ensuring quality and consistency in online provision across the University. The checklist contains a set of good practice recommendations for the design and delivery of a quality, student-centred online learning experience against which staff may benchmark their module design and teaching approaches within the VLE. This paper will give an account of the Centre’s development of the baseline checklist, including the impetus for its creation and the LTTC’s efforts to engage a range of internal stakeholders to promote its formal adoption as part of the University’s quality assurance and enhancement processes. Two efforts by the LTTC to embed the VLE baseline checklist in its CPD modules on technology-enhanced teaching, learning and assessment are outlined as are the initial findings of a survey carried out to ascertain lecturers’ perspectives on, and experience in, implementing the baseline checklist in their own teaching practice. It is hoped that the case study presented here will be of relevance to any higher education institution seeking to develop a strategic approach to academic development for the benefit of learners engaging in online and blended delivery. For practitioners, the findings may provide some insights into the practicalities of working collaboratively to design and then implement a whole-institution inclusive approach to online learning across a multi-platform, dispersed, multi-campus large Higher Education Institution. Keywords VLE, eLearning, Guidelines, Baseline, Academic Development, Student Centred 72 73 Introduction TU Dublin: The Institutional Context Virtual Learning Environments: Lecturer Usage and Student Attitudes TU Dublin was formally established as the first Technological University of Ireland on 1st January 2019. This was the culmination of more than seven years of collaboration between its three constituent institutions – Institute of Technology, Blanchardstown, Dublin Institute of Technology and Institute of Technology, Tallaght. The new University now represents the largest higher education institution in the state with approximately 30,000 enrolled students and is at an advanced stage in realising the vision for its new Dublin city campus at Grangegorman, currently the largest campus development project in Europe. VLEs are now a ubiquitous and mature part of the general Higher Education learning experience that it is now difficult to ‘imagine a time when access to a set of learning materials, particularly in higher education, did not involve logging on to the institutional VLE’ (Thomas, 2012, p. xvi). However, VLEs, despite often being heralded as transformative learning platforms or disruptive drivers of pedagogical innovation, tend to fall into patterns of use variably described as repository and communication (Farrelly, Raftery and Harding, 2018, p. 12), notes bank (O’Rourke, Rooney and Boylan, 2015, p 1) or ‘content storage’ (Flavin, 2020, p. 44) models. This would seem to indicate a gulf between the discourses that often underpin discussions of VLEs and their transformative, paradigm-shifting potential with the real, and often rather more limited, uses to which these technologies are put by staff working at the chalkface. In January 2020, TU Dublin launched its new strategic plan, at the heart of which lies an ambitious set of targets for the realisation of an agile, technology-enabled university offering flexible pathways and transformational learning opportunities for digitally-literate graduates who will live and work in a landscape characterised by a rapid and exponential pace of technological development (TU Dublin, 2020). Currently in the middle of an extensive organisation design process, the university has also signalled an ambition to develop a cutting-edge, student-centred learning environment with a major €500 million infrastructural development plan that includes the establishment of new, state-of-the-art, technology-enabled facilities to provide a quality, technology-enhanced learning experience for all students (TU Dublin, 2020). This action is more than just about making digital technologies available, but about prioritising digital capacitybuilding within a range of institutional activities across the university (The National Forum, 2018) and for these to be appropriately resourced and supported within an associated organisation design and strategy (JISC, 2019). Learning environments are, by any measure, complex systems that often defy our efforts to describe, direct or design them but are commonly regarded as comprising structural conditions such as class sizes and student-faculty ratios, and the physical and virtual learning spaces which mediate the social, physical, psychological and pedagogical contexts in which learning occurs (Fraser, 1998, p. 3). As one of the main points of engagement between learners, lecturers, and instructional content, the virtual learning environment (VLE) represents an integral component of any University’s general learning environment. In earlier research conducted into VLE usage at one of TU Dublin’s constituent institutions, it was reported that while the VLE enjoyed ‘high levels of usage’ amongst academics and had become ‘an integral part of student and lecturer expectation’, engagement with technology had had very little impact, by lecturers’ accounts, on models of teaching and learning or ‘pedagogical innovation’ (O’Rourke, Rooney and Boylan, 2015, p 1). Historically, the university’s antecedent institutions had followed their own distinct trajectories in terms of their development, adoption and acceptance of VLE platforms with the result that the newly merged Technological University Dublin is not only geographically spread across several physical campuses, but also offers different VLE solutions and different virtual classroom integrations to students at each campus. Furthermore, and in common with most Irish HEIs, there are mixed levels of acceptance and exploitation of VLEs by staff at each campus with multiple, complex factors influencing uptake. Generally, institutions operate an ‘opt-in rather than mandatory approach’ (O’Rourke, Rooney and Boylan, 2015, p. 3) but the resultant professional independence afforded to staff can mean ‘ambiguous HEI online contexts’ in which there exist diverse goals, values and often ‘tensions between managerial and professional values’ (Jarzabkowski, Sillince and Shaw, 2010, p. 225). This general milieu presents significant challenges for any effort to develop or advocate for a wholeinstitution approach to technology-supported, online, and blended learning, not least because of the complexity of the infrastructures, operations and practices but also as an outcome of the nature of the rapid, pandemic-precipitated shift to online teaching and learning. However, despite such an apparent diversity within level of staff adoption, a national survey of HE students, published in May 2020, seems to strongly suggest that many learners regard the VLE positively and share a desire to see a greater utilisation of digital technologies in their HEIs. The Irish National Digital Experience (INDEx) findings from a 2019 survey of over 25,000 students on ‘the digital engagement, experiences and expectations’ of students and staff across the Irish higher education sector reported that ‘just under 50%’ of respondents expressed a preference for digital technologies to be utilised ‘more than they are now’ (INDEx, 2020, p. 7) within their courses, and that ‘universal, effective and consistent use of the VLE’ was one of students’ ‘top requests for improving their experience of digital teaching and learning’ (INDEx, 2020, p. 11). 74 75 The authors of the INDEx survey report also remark that students ‘could never have imagined what was to come’ (INDEx, 2020, p. 7) in spring 2020. Indeed, not long after its formal establishment as a university – and like all other HEI’s – TU Dublin’s faculty and students found themselves subject to the unprecedented disruption engendered by the COVID-19 pandemic. With staff and students abruptly forced into a period of ‘emergency remote teaching’ and a subsequent ‘temporary online pivot’ (Nordmann et al., 2020, p. 4) because of the continuing impact of the pandemic, the university’s VLEs offered a lifeline that ensured a continuity in teaching and learning that might otherwise not have been possible. Developing the Baseline Checklist as a Response to the ‘Online Pivot’ Like many of our colleagues in educational development and education more generally, TU Dublin LTTC staff were instrumental in supporting University staff to make the rapid ‘online pivot’. This work primarily comprised the development and curation of supportive resources, and the enhancement of direct academic and technical supports, particularly in the areas of online assessment, VLE content development, and the virtual classroom. One of the key challenges associated with supporting lecturers through the various uncertainties within ‘online pivot’, was an identified need to develop robust but rapidly actionable practical guidelines for enhancing online teaching and learning, however provisional such arrangements may prove to be. It is in this context that the authors of this paper sought to develop the ‘VLE baseline checklist’ as a non-prescriptive and VLE-agnostic set of good practice recommendations for ensuring a quality learning experience within TU Dublin’s multiple VLEs. But also as a practical instrument for lecturers to evaluate and iterate their own approaches to technology-supported learning. The baseline is now also supplemented by the ‘VLE Baseline Plus’, which offers an additional set of practical recommendations to guide the further development of modules in TU Dublin’s virtual learning environments. It is hoped that, through using the respective checklists, lecturers can – by their own preference – contribute to the development of a more consistent approach to the design, delivery, and management of online and blended learning at TU Dublin, irrespective of the specific platform used, based on a shared set of criteria for evaluating quality in online teaching. Literature Review The question of what constitutes ‘quality’ in online education has been the subject of considerable debate since the emergence of the first, internet-enabled distance courses. Today, this remains a vexed question, not least because ‘finding appropriate comparators for the efficacy of any particular mode of delivery is difficult when the broader questions of quality assurance in higher education are far from settled’ (Parker, 2008, p. 306). Though Irish HEI’s are required to ‘have regard’ to Quality and Qualifications Ireland (QQI) quality assurance guidelines for blended learning, published in 2018, the limited number of explicitly designed blended programme offered 76 by universities and institutes of technology make this a matter of relatively niche concern. The pandemic-induced scramble to shift to online and blended delivery has, however temporary, had the effect of throwing the absence of institutional policies around quality in online learning into sharp relief. In a meta-synthesis of quality in online education (QQE) measurement approaches and quality frameworks, Esfijani (2018) writes that, where quality indicators are concerned, there is – as one might expect – a considerable variance in terms of detail and emphasis and no ‘universally applicable’ set of standards for quality in online or eLearning. She also describes how the growing international body of knowledge on QQE is ‘still fragmented and lacking coherence’ (p.70). She recommends that quality assurance indicators and frameworks be developed and employed in ways which consider the specific contextual requirements of institutions and educational cultures, and which place an emphasis on the primary role to be played by key stakeholders – such as instructors and learners – in the development of ‘general quality frameworks’ (p.70). Examples of systematic efforts to establish standards for quality online can be found in publications such as South African National Association of Distance Education and Open Learning (NADEOSA) thirteen ‘Quality Criteria for Distance Education’ in 1996 and in ‘Quality on the Line: Benchmarks for Success in Internet-based Distance Education (Merisotis and Phips/IHEP, 2000), which lists 24 benchmarks ‘essential to ensure quality’ in online courses, and the ‘Exemplary Course Programme’ (2000), established by Blackboard Inc. for the purpose of ‘identifying and disseminating best practices for designing high-quality courses’ and offering a set of quality standards against which online courses can be benchmarked using an associated rubric with numerical point values (Blackboard, 2020). In the early two-thousands, the European Foundation for Quality in e-Learning (EFQUEL), developed a set of quality standards for its ‘UNIQUe’ Certification for Quality in E-learning and in the US, the Accreditation and Assuring Quality in Distance Learning report (US Council for Higher Education Accreditation, 2002) developed a set of accreditation standards for distance learning providers. Elsewhere, The European Institute for e-Learning (EIfEL) and LIfIA (2004) created Open eQUality’ learning standards as a framework of quality outcomes for online learners in adult and higher education and the Online Learning Consortium (OLC) Quality Scorecard (2005) produced a set of criteria and benchmarking tools within a suite of quality scorecards for course design, instructional practice, digital courseware, and online student support. More recently the Australasian Council on Open, Distance and e-Learning created ‘benchmarks for technology-enhanced learning’ to assist institutions in ‘their practice of delivering a quality technology enhanced learning experience for their students and staff’ (ACODE, 2014). Other frameworks have been produced by the Asian Association of Open Universities (AAOU), the European Foundation for Quality in e-Learning (EFQUEL) ‘Open ECBCheck’ quality improvement scheme; the German development agency Deutsche Gesellschaft für Internationale Zusammenarbeit (GIZ) quality assurance and international accreditation of ‘electronically supported learning’ (GIZ, 2020); and the international organisation for standardisation (ISO) offers a range of standards germane to online or eLearning including ISO/IEC 40180 (ISO, 2017). 77 In 2017, Martin, Polly, Jokiaho and May identified nine main categories that tend to be prioritised in ‘learning standard documents’, with instructional design appearing to be the most emphasised, followed respectively by ‘students attributes, satisfaction institutional mission, structure and support’ (Martin et al., 2017, p. 7). More recently in 2019, the revised ‘National Standards for Quality Online Courses’ (NSQOL) produced by the US-based Quality Matters (QM), in partnership with the Virtual Learning Leadership Alliance (VLLA), now incorporate quality standards that broadly focus on issues such as Course Overview and Support, Content, Instructional Design, Learner Assessment, Accessibility and Usability, Technology and Course Evaluation. Similarly, ‘Quality matters’, a US-based body offers a course review process based a benchmarking process involving its own QM rubric with criteria including course design, delivery, content, institutional infrastructure, the LMS, ‘faculty readiness’, and ‘student readiness’ (Quality Matters, 2020). The development of the VLE baseline checklist reported in this paper was informed by comparable efforts to establish benchmarking criteria for the promotion and evaluation of quality and best practice in online and blended learning in the context of an unprecedented, accelerated, and unplanned-for explosion in the use of VLE’s for teaching and learning. These include the Online Learning Consortium’s ‘Quality Scorecard for Online Learning’ (OLC, 2011), the evidence-based decision-making Framework for Online Learning (Sandars et al., 2020, p.4), the conceptual framework for responsive online teaching in crises (Whittle et al., 2020, p. 313) and UCL Connected Learning Baseline (UCL, 2020). Though the university is currently in the process of developing its own institution-wide set of quality assurance guidelines for online and blended learning – based on QQI QA guidelines – the VLE baseline checklist has the more modest aim of providing lecturers with a set of quality indicators that they can use to self-evaluate their own online modules. The LTTC also intends to continuously review and update the baseline based on feedback offered by lecturers and other stakeholders so that – if formally adopted as part of the university’s QA frameworks – it will adequately reflect its diverse disciplinary and educational cultures and practices, and the preferences of lecturers themselves. Institutional Context: Creating and Promoting the VLE Baseline Following the closure of TU Dublin’s physical campuses due to the COVID-19 pandemic, April 2020 saw the establishment of an institutional strategic planning group with a remit for ensuring that the university had the capacity and capability to deliver the remainder of its 2020/21 secondsemester programmes – including synchronous live classrooms and online assessment – through its current VLEs. The initial focus of this group was to maintain learning continuity for students and the provision of support to colleges and schools in the preparation of alternative online assessment approaches. This necessitated a rapid and unprecedented acceleration in VLE adoption combined with a more strategic and coordinated approach to training, support and technology to enable staff and students to properly engage in remote and blended learning contexts. Against this backdrop, the TU Dublin VLE baseline was initiated in response to calls for direction from many stakeholders including students, academic staff and leaders; and staff at the Learning, Teaching and Technology Centre (LTTC). 78 The Student Perspective In May 2020, a survey was conducted by TU Dublin’s Campus Life office to capture the main concerns of students looking forward to the new academic year. Campus Life is a student-facing unit within the university which aims to provide high quality, responsive and integrated student support services with the overarching aim of creating an excellent student experience at the university’s largest city campus. From the 1511 responses (209 PT and 1320 FT) to this survey, students’ main concerns were financial. Almost half of those who had been in employment had lost their jobs in 2020 and were worried about being able to cover College costs, including fee payments. With the shift to online delivery, students were also surveyed about their experiences, with only ten percent of part-time learners reporting a positive experience, dropping to 1% for students in traditional, full-time delivery. Both student groups (PT 16% and FT 14%) expressed reservations about the effectiveness of online vis a vis face-to-face study, noting a lack of consistency between modules in the use of the VLE across their programmes. The possible lack of interaction with staff and peers (such as through group work) was also highlighted (14%) as well as concerns about being able to manage workload. 13% of part-time students surveyed reported being concerned about being able to find suitable study spaces. The next most frequently cited concern for full-time students was an anticipated difficulty in getting back to the college routine, and finding the motivation and focus required on their eventual return to campus (9%). On a national level, data captured from the Irish INDEx survey (Index, 2020) identified the virtual learning environment (VLE) as the digital tool found ‘most useful’ by students and staff and that universal, effective and consistent use of the VLE and provision of lecture recordings were two of students’ top requests for improving their experience of digital teaching and learning. A majority of students also expressed a preference for seeing digital tools and technologies used within their courses ‘more than they are now’ (INDEx, 2020, p. 7). If the experience of 2020 has taught us anything, a more coordinated approach to the VLE is required if online and remote teaching and learning is to be effective for learners and one that is carefully informed, discussed and collaboratively agreed (National Forum, 2020, p. 6). To give an indication of the scale of the challenge faced by Irish HEI’s during this year’s online pivot, at the time of the INDEx Survey, (INDEx, 2020, p. 37) 70% of staff who teach reported having no experience of teaching in a live online environment. LTTC Actions Much of the early focus of the LTTC response to the CoVID pandemic was to provide online workshops and training sessions for academic staff across the University. For many staff this was the first time they had taught online. That schedule soon changed however and grew to incorporate additional sessions that addressed other needs raised by staff as they experienced their ‘new normal’. In addition, a suite of new online resources was developed and made available from the LTTC Keep Teaching, Teaching Online, and Assessing Online webpages from March 2020 onwards. These evolved into a ‘Teaching Remotely’ resource portal for teaching, learning and assessment in online and blended modules, made available from September 2020 onwards. The 79 centre also developed a considerable number of resources for D2L Brightspace, the city campus VLE, designed explicitly to align with the VLE baseline, which were made available through workshops and the LTTC website. Additional teaching resources were also provided to TU Dublin staff to use through LinkedIn learning. Timely responsive sessions tailored to the expressed needs and preferences of teaching staff were identified through the webinar registration form. However, as is evident from the analysis of 450 qualitative form responses not only were academics upskilling to handle their immediate challenges (27%), but they were also thinking beyond the ‘pivot’ period between March and June of 2020, out of a concern for the upcoming academic year. It is perhaps worth noting that in a survey conducted by the LTTC in May 2020, 58 percent of TU Dublin staff replied (n=68) that the move online had resulted in significant changes in their assessment practices with over 50% reporting considerable changes to their teaching. However, many staff reported concerns about teaching their online and blended modules effectively and being able to move seamlessly between different permutations of blended and online delivery as circumstances and rolling ‘lockdowns’ dictated (65%). Staff were also concerned about incoming first-year students in September 2020 and were preparing for how they could support them adequately, through extended online inductions or orientations, to engage them in their studies in an online context (17%). While, as a team, the LTTC had reacted quickly and pre-emptively to TU Dublin’s institutional closure, providing a range of supports that were urgently needed by academics for ‘remote emergency teaching’ in the immediate term, it became clear that before the commencement of the next academic year that there was an emergent need to reorganise support materials so as to produce an easy-to-follow ‘path’ for lecturers through using our LTTC resources to develop and refine their approach to using the VLE. The VLE baseline emerged, in part, as a response to this need to guide teaching staff though available support resources in a way that is aligned with the practical steps involved in pivoting to online delivery. TU Dublin’s Academic Leadership Forum (ALF) as a Driver A key concern articulated by university leaders and staff was to ensure that all online provision met a baseline quality standard. So, the challenge facing the LTTC was to rapidly design and build an easy-to-use and non-prescriptive framework, or model, that staff could use as a guide for reviewing and redeveloping their online modules for the academic year 2020-21 and also provide a foundation for future quality online indicators for the new University. This framework would also need to be closely linked to LTTC produced resources, workshops, training sessions and CPD offerings. Comments from key internal stakeholders indicated that a VLE-agnostic tool would be most useful, given the multiple platforms in use at TU Dublin’s constituent campuses; this was corroborated by key staff members whose feedback was sought at each stage of the baseline’s development, and after its completion. A baseline model was chosen as it was appropriate to the dual-mode delivery and multi-platform nature of TU Dublin’s online programmes. Key aims included the development of a common terminology for virtual delivery, a set of baseline quality indicators for blended and online delivery, 80 and a framework for ensuring consistency of approach across the university independent of the VLE platform used. Alongside this, a need was recognised to encourage the adoption of the VLE baseline across as many programmes as possible before the commencement of the new academic year. Early collaboration was helpful to inform decisions, develop a shared understanding for a working model that could facilitate a consistent and coherent implementation process across all three University campuses. To this end in July 2020, the LTTC, in conjunction with the Academic Leadership Forum (ALF) working group prepared a review paper entitled ‘responding to the Covid-19 challenge: TU Dublin eLearn project’. This document provided an overview of current supports for staff and students’ digital skills development proposing an institutional strategic approach to ensuring quality across our programme provision. Within this there was a request for additional resources necessary for the University to be able to achieve a baseline provision for all our staff and students. Creating the Baseline Phase one of the baseline development involved a literature search, review of online resources and consultation with staff, students and Brightspace users with the help of our D2L Customer Success Partner. We also investigated a range of national and international approaches to the development of quality indicators for online delivery and associate frameworks for their measurement and evaluation. Our intention in developing the new checklist was to step the university’s academic staff through a series of categorised good practice recommendations for the design and management of modules in our institutional VLEs. From this review, we drew upon the format of the University College London (UCL) Connected Learning Baseline to inform an initial framework structure for our ‘VLE Baseline Checklist’ redesign.. It was our hope that if this checklist was embraced by programme teams, as well as by individual academic staff, it would help standardise pedagogically-sound approaches to the design and management of online modules across all our programmes and provide some much-needed guidance for staff and consistency for students. Undertaking a redesign also enabled us to link with other university learning and teaching related projects such as the First Year Framework for Success, the Student Success Portal, the Learning from and Engaging with Assessment and Feedback (LEAF) project, and the Technology Enhanced Learning Teaching and Assessment (TELTA) award-winning module on our MSc programme. We could also draw upon extensive data gathered from both staff and students through the recent consultation and procurement process conducted prior to migrating to our new VLE Brightspace in Sept 2019. A series of ‘brainstorming’ sessions were used to pull together every idea and approach that we felt warranted possible inclusion in the checklist. Then, from that list we drew out themes or categories of ideas settling finally on (1) Student Orientation, (2) Structure your Content, (3) Live Lectures and Tutorials, (4) Communication, (5) Assessment and Feedback, (6) Resources, (7) Accessibility, and (8) Quality Assurance. From there, we categorised each of the different ideas and approaches in our list under those headings before reviewing each category again. This reviewing process was very important and led to the removal of some ideas and/or the repositioning or merging of others. 81 Finally, we engaged in a word-smithing exercise to improve the clarity and style of the text of each idea within each category. We found that starting each idea, where possible, with a short statement in bold text followed by a concise explanation was effective, for example: 1.6 A short ‘communication statement’ – include a section which details: how you will communicate with your students; their expectations with regards to your availability, ‘virtual office hours’ and response times; contact info for relevant support staff; class ‘netiquette, i.e., acceptable standards of communication and expectations of participation in the virtual classroom, discussion forums etc. 4.2 Use the announcements tool to communicate important updates to learners, such as key dates, upcoming online classes, or new module information. Encourage students to enable email notifications. 7.2 Navigation and linking – Make sure that module content is clearly organised and labelled and use a consistent navigational style; verify that all links provided are live and not broken, use descriptive link titles, if links will open in another tab or window, make this clear. Finally, following some feedback from colleagues within the LTTC and those involved in Learning Development within the colleges that make up the university, some final tweaks were made to individual items within the checklist. The presentation of the checklist was as important as the text included in it and, as such, we decided to limit it to two A4 pages to make it manageable to use when printed, and we included checkboxes down the side next to each idea to encourage staff to use this checklist in a very active way to evaluate their online modules against it. Aware that some of the academic staff using this checklist would be new to our university and our VLEs and therefore would be less confident in implementing its recommendations, we also created a lengthier companion guide that was linked to from the VLE baseline checklist. Should they need it, full instructions on how to implement any of the ideas on the checklist in their module was set out there for them and, where appropriate, samples were provided. For example, a communication statement template was prepared that lecturers could copy and edit for their own modules. The baseline was finally complete by the end of May 2020. At this point it was presented at several senior leadership fora, tabled and approved at the Academic Quality Assurance Committee, and finally approved and adopted by the university’s President’s group as the approved model for online module design and management at the University. Promoting the Baseline At the commencement of the 2020-21 academic year, the LTTC began to promote the VLE baseline more actively to staff using an array of internal communications both at the university level and across its constituent campuses, colleges, and schools. The baseline was explicitly referenced in all relevant LTTC university-wide emails promoting our online workshops and training schedule. Similarly all workshop and training session participants were alerted to the Checklist (with workshops carefully aligned with the recommendations contained with the baseline where appropriate). 82 The centre also linked to the baseline in various places on its website where support resources are curated. Further to this, each of the university’s college-level Heads of Learning Development were alerted to the availability of the baseline and encouraged to disseminate it to other staff in their respective schools. Through the various senior leadership fora, college directors were introduced to the baseline as part of a response to their previous-semester requests for further supports for staff, particularly in relation to their approach to the design and management of pedagogically sound and student-engaging online and blended modules in the academic year ahead. Additionally, the LTTC redeveloped two of its own courses on technology-enhanced teaching and assessment practice to align them explicitly with the baseline checklist. Firstly, in October 2020, the centre designed and ran a one-week online mini-module for TU Dublin academic staff, developed to give staff a chance to experience being an online student but also to witness an exemplar module incorporating many of the good practice recommendations as set out in the VLE baseline checklist. The content covered over the course of the week gave further direction on important aspects of teaching and assessing online and included baseline topics such as module orientation, building communities of learners, curating content, and engaging learners. Finally, the assessment for the mini-module was also based on the VLE checklist – with participants asked to print out the baseline, use it to evaluate one of their own online modules, noting where their modules fell short of the baseline. A final assignment was a 300-word reflective piece on the application to their own teaching practice. Figure 1: The Course Homepage for the ‘TELTA Engage’ Mini-module, Hosted on the D2L ‘Brightspace’ VLE In January 2020, the eight-week 5 ECTS module ‘Technology-Enhanced Teaching, Learning & Assessment’ (TELTA) was also re-developed and extended to align with the recommendations of the VLE baseline checklist. This redesign was undertaken both to strengthen the module and to ensure that it was adhering to TU Dublin best practice, but also so that the module could be presented as an example of best practice to participants in a manner similar to the ‘TELTA Engage’ mini-module. Both courses were also used to promote the VLE baseline model to staff and as further opportunities to gather feedback from participants on their experience of evaluating their own modules against its recommendations. 83 Figure 2: A Screengrab of the TELTA Module Which Illustrates the Alignment Between One of its Core Units the VLE Baseline Checklist Findings and Discussions – Initial Impressions To date, forty-two survey responses have been collected over a four-month period, with staff encouraged to complete the survey via general university-wide emails, as well as targeted communications directed at participants who had completed the ‘TELTA Engage’ mini-module and full TELTA CPD online course. As respondents were drawn from across TU Dublin’s campuses, just over half of those surveyed (54%) indicated that they were currently using Brightspace as their main VLE while 43% were using Moodle. From survey feedback, 73% were from staff who had been teaching for over eleven years in Higher Education, 79% had attended a training workshop/course related to teaching and learning, while 45% had completed a fully online training course in the last five years with less than half (48%) having previously taught/assessed a cohort of students on a fully online course. The majority of those who responded (79%) had already heard of the VLE baseline, although substantially less (52%) had previously made use of the Checklist to support the (re)design or development of one of their VLE modules. Those staff already familiar with the checklist had become aware of the resource either through all-staff emails (38%) or attendance at an LTTC webinar (40%). The remainder had received the Checklist via College or School distribution lists. VLE Baseline Perceived Usefulness Data Gathering In addition to gathering data from our different professional development activities, it was decided to develop a short survey that could reach all staff across all university campuses. The survey in Google Forms comprised four sections: Section one served as an introduction and gathered general information about the respondents’ teaching background, completed professional development, VLE preferences and use of technology within their teaching. Section two established where the respondent had heard about the baseline, and if they had already used it, generally, to help redesign or develop an online module. Section three concentrated on practical uses of the baseline checklist. Survey respondents were asked to rate the usefulness of the topics covered, the format and structure, the bulleted recommendations, and the companion guide that had been developed. They were also asked their opinion on what topics they felt did not need to be included in the checklist, and what – from their perspective – should have been included but was omitted. Respondents were also given an opportunity to volunteer additional comments on the checklist. The survey was disseminated to all staff by email. It was also promoted at workshops and training sessions, and amongst staff who had registered to take part in the two modules discussed above: ‘TELTA’ and ‘TELTA Engage’. The response rate was lower than ideal, with just 42 participants in total, meaning that the findings presented here cannot be considered in any way representative or conclusive. However, the data does provide an interesting, if partial, qualitative snapshot of the experience of TU Dublin staff members’ perspectives on the baseline. With just over half of respondents reporting having used the checklist to support the design or redevelopment of their online modules, the data that was gathered is meaningful. The following section of this paper will discuss our findings in more detail. 84 All four features, (Topics, Format, Recommendations and Accompanying guide) listed in the survey were deemed useful by respondents, with the topics and recommendations being rated slightly more highly. Some respondents commented about the checklist being a point of reference or a guide to improve online module design, for example one respondent (R 20) commented that ‘I am definitely going to use it as a point of reference against which I will check my VLE modules’, while another (R 33) offered that ‘I will definitely use the guidelines to improve my module pages on Moodle’. Other respondents felt that they did not need a reference guide, did not have the time to undertake a redesign, or preferred grounding their design directly on theoretical models, for example, respondent 27 commented that ‘A lot of the check list was already being used in my modules’ while another (R 16) indicated that they already ‘base(d) their own design of Gilly Salmon’s ‘5-stage’ model’. By and large, most respondents reported seeing the value of having a TU Dublin VLE benchmark, with one (R 5) commented that ‘Making it a requirement for all staff that there is minimum engagement with principles of online delivery and/or requiring all to be familiar with a checklist for good practice would go a long way towards enhancing quality in online delivery’. Another, (R 24) commented that it was a ‘Very useful baseline for best practice’ while another (R 38) stated that it was ‘Great to be setting the bar high for us, not sure we’re all at that level (yet)’. Similarly, respondent (R 7) commented ‘Thank you for improving the general rules, it will improve overall quality immensely’. 85 The VLE as an Initiator of Change For those respondents who had tried to use the baseline as a guide for implementing change, the outcomes were mixed. Some respondents reported feeling positive about the baseline’s recommendations and anticipated implementing them in their future online teaching. For example, one respondent (R 8) stated ‘I like the idea of releasing content incrementally and I intend to use this in a few of my courses ‘. Respondent 19 alluded to the difficulty in implementing changes in online course design and evaluating their effectiveness in the short term by stating ‘I have tried to create a sense of connection and will keep trying, I am not certain if is working yet but I will keep on trying until it does, I chat to the classes I know from last year, for newer classes they mainly silent’. In the same vein, respondent 22 reported that ‘I really liked the idea of a communications statement but none of my students have responded to the idea. They still do what they have always done and email me directly’. Elsewhere, one Respondent (R 21) expressed a preference for not following the baseline’s recommendations in the area of communication, and establishing instructor presence, offering that ‘I have consciously avoided the recommendations in sense of connection. I find notifications intrusive and contributing to the sense of “always on”’. Table 1: Comparison Between Most and Least Frequently Implemented VLE Recommendations VLE ref %N=42 Highest application VLE ref %N=42 Lowest application 2.2 86 Break up your content 1.5 40 Assessment overview/feedback opportunities 1.1 76 Welcome message to greet your learners 3.2 40 Notify learners – use VLE communications tools to remind learners of classes etc. 5.1 76 Provide a clear assessment schedule and overview 1.6 38 A short ‘communication statement’ outlining your availability etc. 3.1 69 Provide a clear schedule of live classes/lectures in advance, 4.4 38 Establish presence participating actively in module discussion forum etc. 3.3 69 Link to recordings 1.2 33 Staff information page to introduce yourself, provide contact information 2.1 69 Use clear and consistent terms 4.1 33 Provide a communication ‘statement’ in orientation unit, your virtual hours etc. Prioritisation of VLE Recommendations 2.6 67 3.4 29 The VLE checklist combines recommendations for both design and practices under eight headings. When surveyed about the design of the checklist, there were comments that some of the practicebased recommendations could be omitted. For example, Respondent 5 commented that ‘the checklist is too long’ and suggested ‘leav(ing) out section 7 (accessibility) entirely’ while another (R 29) felt that ‘The process of interacting with students’, i.e. recommendations for informal icebreaking under orientation in live lectures and tutorials, ‘should be separate’. Respondent 42 pointed to perceived issues with the scope of the baseline, suggesting that ‘Parts of 5 – assessment are more about constructive alignment, rather than the VLE. It’s good to have this as a reference but impossible to have it all implemented’. Provide a reading and resources list Orient new learners with a session for troubleshooting, icebreakers and orientation 4.2 67 Use the announcements tool to communicate important dates 7.4 26 Adhere to TU Dublin accessibility guidelines issued by the disability service Reference was made to the current working situation by several respondents alongside some of the challenges of rapidly moving online without the requisite time and resource required for the explicit redesign or development of their modules. Respondent 21 commented that ‘the overwhelming workload of having to move all teaching activities online at short notice means that I cannot claim to have done a thorough “redesign” of modules’, with respondent 22 similarly reporting that they lacked the ‘Time required to implement and sustain the checklist especially with large classes’. Some respondents expressed a desire to make more extended use of the checklist but reported a similar concern that current workloads were not conducive to this, at least in the short to medium term, for example when respondent 22 commented that ‘While I aim to adapt things to the suggested guidelines and standards, it is not feasible right now to re-think everything for every module, while teaching is going on. Not least because this is possibly a temporary situation’. It is interesting to note that this respondent also expressed reservations about the investment required in re-developing their course given the likely provisional nature of online delivery in the context of the current pandemic. 86 If a comparison is made between the most and least frequently implemented VLE good practices, those related to the VLE design appeared to be the most likely to have been completed, while conversely recommendations around establishing lecturer presence or engaging directly with students were less so (see table below). 87 Conclusion References This paper has provided an overview of work undertaken within TU Dublin to support lecturers to make a move online as part of an institutional response to challenges arising from the COVID-19 pandemic. Key within this approach was the collaborative design of a VLE-agnostic baseline checklist providing a set of good-practice recommendations for the design and delivery of quality, student-centred online learning. The design of the checklist was informed by current research and consideration of other national and international best practice models. AAOU. (2016) Asian Association of Open Universities. Available at: https://www.aaou.org/quality-assurance-framework/. As part of the pilot evaluation study, the checklist was reviewed by a small but representative sample of lecturers using the Moodle and Brightspace VLEs at TU Dublin. It was disappointing that almost half of the staff surveyed as part of our evaluation study were unfamiliar with the resource. This is likely to be a result, at least partially, of the way the checklist was disseminated and academic staff workloads in the context of the current pandemic. LTTC webinars appeared to be the most successful way to engage or raise awareness and is noted as a recommendation for the next stage of the project. Staff who did respond to the survey reported a varied amount of time and experience of time teaching in HE or the use of blended or online learning. Irrespective of their campus VLE platform, almost all those surveyed agreed that the new resource provided a useful guide to module design. A small number felt that the checklist was too long, while almost a quarter of respondents suggested the inclusion of additional topics or requirements. CHEA Institute for Research and Study of Accreditation and Quality Assurance. (2002) Accreditation and Assuring Quality in Distance Learning, Washington: Council for Higher Education Accreditation. Perhaps not unexpectedly, recommendations directly related to design and set up of a module, e.g., links to lectures and resources, were the most likely to be reported as having been implemented. The least likely appeared to relate to establishing teacher presence within an online environment or to strategies for promoting engagement with students. A distinction between the design and their personal professional practice seemed to be a conscious decision made by the lecturer and perhaps reflected their overall approach to teaching in an online space. This suggests that it might be helpful to try and engage staff through tailored webinars and workshops and modules to help raise awareness of the value of online staff /student presence rather than simply circulate the checklist and accompanying guide in isolation. Subsequent to this study, a Baseline plus checklist has now been designed and accompanying Moodle and Brightspace guides developed. The aim, following its approval at Academic Quality Assurance (AQA) committee and its adoption by the University President’s group as the approved model for online module design and management at the University, is to embed the baseline within the university’s new Quality Enhancement framework as a requirement for all TU Dublin programmes. As with similar enterprises, it is often the conversations around the design and application processes that will be the initiators of subsequent longer-term changes in practices. It is hoped that any such VLE related discussions between different stakeholder groups can build towards a shared understanding of what constitutes a well-created and distinctive TU Dublin quality online learning experience for all of our students. 88 ACODE. (2014) Benchmarks for Technology Enhanced Learning, Canberra: ACODE. Blackboard. (2000) Exemplary Course Program. Available at: https://www.blackboard.com/resources/are-your-courses-exemplary. EFQUEL. (2012) Certification of Quality in eLearning. [Online] Available at: http://unique.efquel.org/. EFQUEL. (2015) ECBCheck. Available at: http://www.ecb-check.org/7/ecbcheck-e-learning-programme-certification/. Esfijani, A. (2018) ‘Measuring Quality in Online Education: A Meta-Synthesis’, American Journal of Distance Education, vol. 32, no. 1, pp. 57-73. European University Association. (2018) Learning and teaching in the European Higher Education Area. Geneva: European University Association. Farrelly, T., Raftery, D., and Harding, N., (2018) “Exploring lecturer engagement with the VLE: findings from a multi-college staff survey,” The Irish Journal of Technology Enhanced Learning, vol. 3, no. 2, pp. 11–23. Fraser, B.J., (1998) “The birth of a new journal: Editor’s Introduction,” Learning Environments Research, vol. 1, no. 1, pp. 1–5. Flavin, M. (2020). Re-imagining Technology Enhanced Learning: Critical Perspectives on Disruptive Innovation. Springer International Publishing. GIZ. (2020) GIZ. Available at: https://www.giz.de/en/ourservices/99581.html. ISO/IEC. (2017) ISO Organisation. Available at: https://www.iso.org/standard/62825.html. Jarzabkowski, P., Sillince, J. A.A. and Shaw, D. (2010) “Strategic ambiguity as a rhetorical resource for enabling multiple interests,” Human Relations, vol. 63, no. 2, pp. 219–248. Keegan, D.A., and Bannister, S.L. (2020) ‘More than moving online: Implications of the COVID pandemic on curriculum development’, Medical Education, vol. 55, no. 1, pp. 101-103. Langer-Crame, M., Killen, C., Francis, J., Beetham, H., Knight, S. and Newman, T. (2019) JISC Digital Experience Insights survey 2019: findings from teaching staff in UK further and higher education. Available at: https://www.jisc.ac.uk/reports/digital-experience-insights-survey-2019-staff-uk. LIfIA & EIfEL. (2004) Open eQuality Learning Standards, Stanford, CA: LIfIA & EifEL. LTTC. (2020) VLE Baseline Checklist. Available at: https://www.tudublin.ie/explore/about-theuniversity/academic-affairs/teaching/lta-supports/teaching-remotely/checklists/. 89 LTTC. (2020) Teaching Remotely. Available at: https://www.tudublin.ie/explore/about-theuniversity/academic-affairs/teaching/lta-supports/teaching-remotely/. The National Forum. (2018) Ireland’s National Professional Development Framework Summary Findings of the Initial Implementation. Dublin: The National Forum. Martin, F., Polly, D., Jokiaho, A. and May, B. (2017) ‘Global Standards for enhancing quality in online learning’, The Quarterly Review of Distance Education, vol. 8, no. 2, pp. 1-10. The National Forum for the Enhancement of Teaching and Learning. (2020) INDEX Findings from students and staff who teach in Higher Education. Dublin: The National Forum. Merisotis, J. and Phipps, R. (2000) Quality on the line: Benchmarks for success in internet-based distance education, Washington: National Education Association. Thomas, M. (2012) ‘Preface’ in Thomas, M. (ed.) Design, Implementation, and Evaluation of Virtual Learning Environments. Pennsylvania: IGI global. pp. xvi - xxv. NADEOSA. (1996) Designing and Delivering Distance Education: Quality Criteria and Case Studies from South Africa, South Africa: NADEOSA. University College London. (2020) UCL Connected Learning Baseline. Available at: https://www.ucl.ac.uk/teaching-learning/publications/2020/may/ucl-connected-learning-baseline. Nordmann, E., Horlin, C., Hutchison, J., Murray, J-A., Robson, L., Seery, M.K. and MacKay, J.R.D. (2020) 10 simple rules for supporting a temporary online pivot in higher education. Available at: https://doi.org/10.31234/osf.io/qdh25. Virtual Learning Leadership Alliance. (2011) National Standards for Quality Online Teaching. Vienna: INACOL. Whittle, C. et al. (2020) ‘Emergency remote teaching environment: a conceptual framework for responsive online teaching in crises’, Information and Learning Sciences, vol. 121, no. 5/6, pp. 311-319. Online Learning Consortium. (2011) The Quality Scorecard for online learning. Boston: Online Learning Consortium. O’Rourke, K.C., Rooney, P. and Boylan, F. (2015) ‘What’s the Use of a VLE?’, Irish Journal of Academic Practice, vol. 4, no. 1, Article 10. Parker, N.K. (2008) “The quality dilemma in online education revisited,” in Anderson, T (ed.) The Theory and Practice of Online Learning. 2nd edn. Edmonton, AB: Athabasca University Press, pp. 305–342. QQI. (2018) Quality Assurance Guidelines for Blended Learning Programmes, Dublin: QQI. QQI. (2020) The Impact of COVID-19 Modifications to Teaching, Learning and Assessment in Irish Further Education and Training and Higher Education. Dublin: QQI. Quality Matters (2020). Quality Matters. Available at: https://www.qualitymatters.org/reviews-certifications/course-design-reviews. Redmond, P., Abawi, L-A., Brown, A., Henderson, R. and Heffernan, A. (2018). ‘An Online Engagement Framework for Higher Education’, Online Learning, vol. 22, no. 1, pp. 183-204. Sandars, J., Correia, R., Dankbaar, M., de Jong, P., Goh, P., Hege, I., Masters, K., Oh, S., Patel, R., Premkumar, K., Webb, A. and Pusic, M. (2020) ‘Twelve tips for rapidly migrating to online learning during the COVID-19 pandemic’, MedEDPublish, vol. 9, pp. 1-14. Technological University of Dublin. (2019) Strategic Plan 2030. Dublin: TU Dublin. The National Forum for the Enhancement of Teaching and Learning (2018) Building Digital Capacity in Irish Higher Education 2013–18. Dublin. The National Forum. 90 91