JASAAN NATIONAL SENIOR HIGH SCHOOL

GRADE 11 TVL-SMAW

TITLE

Impact of the COVID-19 pandemic on teaching and learning in health

professional education: a mixed methods study protocol

Submitted by:
GROUP 2
Grade and Section:
GRADE 11 STRENGTH
Submitted to:
FELOMINA DAGUPLO FLORES
Abstract

Background
Due to the complex nature of healthcare professionals’ roles and
responsibilities, the education of this workforce is multifaceted and
challenging. It relies on various sources of learning from teachers,
peers, patients and may focus on Work Integrated Learning (WIL). The
COVID-19 pandemic has impacted many of these learning
opportunities especially those in large groups or involving in person
interaction with peers and patients. Much of the curriculum has been
adapted to an online format, the long-term consequence of which is
yet to be recognized. The changed format is likely to impact learning
pedagogy effecting both students and teachers. This requires a
systematic approach to evaluation of online teaching and learning
adaptation, in comparison to the previous format, where, in person
education may have been the focus.

Methods
The proposed study is a broad based evaluation of health professional
education in a major Australian University. The protocol describes a
mixed methods convergent design to evaluate the impact of online
education on students and teachers in health professional courses
including Medicine, Nursing, Allied Health and Biomedical Science. A
framework, developed at the university, using Contribution Analysis
(CA), will guide the evaluation. Quantitative data relating to student
performance, student evaluation of units, quantity of teaching
activities and resource utilization will be collected and subjected to
relevant statistical analysis. Data will be collected through surveys
(500 students and 100 teachers), focus groups (10 groups of students)
and interviews of students and teachers (50 students beyond
graduation and 25 teachers, for long term follow up to 12 months).
Application of CA will be used to answer the key research questions on
the short term and long-term impact of online education on teaching
and learning approaches.

Discussion
The protocol describes the study, which will be widely implemented
over the various courses in Health Professional Education and
Biomedical Science. It will evaluate how students and teachers
engage with the online delivery of the curriculum, student
performance, and resources used to implement these changes. It also
aims to evaluate longitudinal outcome of student learning attributes
and impact on graduate outcomes, which is poorly reported in
educational literature.
Background
The preparation of the future healthcare workforce is a key priority
for governments around the world[1]. Traditionally this education has
mainly been supported through a variety of campus-focused activities,
which involve face-to-face interactions of students with peers, and
work-integrated learning (WIL). For decades, doctors, nurses and
allied health professionals have been trained by observing and
learning from experienced clinical practitioners through work-
integrated learning, similar to the “apprenticeship model”[2]. Student
learning typically takes place in locations like lecture theatres,
hospital wards, operating theatres, practitioners’ clinics and the
community. Occasionally it occurs informally in social places like
hospital tearooms, cafeterias or in workplace corridors through tutor-
student and peer interactions, but is typically in person[3].

In person education has been shown to create more student-tutor and

student-student interaction, which can promote better engagement
[4]. As explained through the Social learning Theory, new patterns of
behavior arise from a direct interaction with peers or by observing
behavior of peers[5]. However, due to the COVID-19 pandemic all in
person opportunities for formal and informal learning have ceased
and health professional courses have been required to move to
exclusive delivery through online education[6]. The impact of this
change to teaching and learning, on both learners and teachers, is
largely unknown.

Online education is the delivery of learning materials using internet

for student-student and student-teacher interaction and for
distributing educational materials. Over the last decade, with
advancement of technology-assisted learning, teachers have started
using online learning platforms to promote self-directed learning and
assessment in students. Use of online education also assists in
engaging a large group of students at one time (where lectures may
not be possible) with options of both synchronous and asynchronous
learning [7]. While synchronous learning ensures that all students
learn the same content in a similar way, asynchronous learning
facilitates information to be communicated across sites and
campuses, with students engaging in learning at their own pace, and
where feasible in their own time. Similarly, online delivery of
education as a component of blended learning, allows tutors to
flexibly adjust to student learning styles and assess them [8]. It may
 assist teachers to provide the necessary support required for the
individual student while feedback may not be possible in a large
group face-to-face session [9].

During the COVID-19 pandemic, universities across the world have

transitioned to distance education, most of which, is planned for
online delivery [6, 10, 11]. Health professional courses may use
variable tools of blended learning for this process, which may include
synchronous online tutorials, E-learning in simulation sessions,
asynchronous activity in moderated discussion forums, formative
quizzes and other teacher-directed or self-directed learning activities.
Engaging with these learning methods may be perceived differently
from conventional classroom-based teaching. Online learning has
required adjustment by both teachers and learners to adapt to new
learning styles with focus on active learning and technological
support required for delivery of teaching [12, 13].

Engaging with a curriculum that has been transitioned from in person

to online is likely to impact how students learn and how they can
contextualize that learning into clinical practice. Understanding the
impact of these initiatives on student engagement, learning and
behavior (both positive and negative), will provide important
information for teaching and learning practice into the future[14], in
particular the influence of online education on the development of
practical skills and graduate readiness to practice. In this paper, we
present a protocol developed in order to study the process and impact
of student and teacher adaptation to the learning pedagogies
developing as a result of the pandemic. The primary aim of this
protocol is to study the impact of the change to teaching and learning
 approaches. This includes evaluation of the online education and
changes to work-integrated learning, during the COVID-19 pandemic
and its impact on students and teachers, both in the short and long
term.

Methodological approach
A convergent mixed methods research design [15] will be employed
using both quantitative and qualitative data, which will be collected
together longitudinally analyzed and interpreted together. Mixed
methods has been frequently applied to health education research
[16] due to the complexity of learning programs and multiple
interactions involved, making it difficult and inaccurate to use a single
evaluation method. This mixed methods approach will facilitate
evaluation of the various elements and factors that influence student
learning and online curriculum delivery by teachers. A framework
developed by the same faculty on the factors that contribute to health
professions student learning will be used to guide the evaluation. This
framework used Contribution Analysis[17] to identify the proximal
and distal factors influencing graduate outcomes (Table 1). A range of
teacher and student related factors was identified as influencing
learning. In addition, the role of WIL in supporting graduate outcomes
was highlighted.
 Data collection - Quantitative data
Assessment data monitoring
Student performance data before and after COVID times will be
collected with possible confounders controlled for.

Assessment data will be collected over the duration of the COVID-19

pandemic including the in-semester, work-based and end of the year
assessment. The data collected will be compared to similar data from
the year 2019 to control for possible confounders. The analysis will
provide a direct comparison of student performance in the year prior
(when in person learning was present) with the changes to teaching
and learning that were introduced during the pandemic (partial/
complete replacement of in person with online learning).

Teacher and student surveys

The teacher and student surveys, to be administered, will capture
participants’ perspectives of the teaching and learning experienced
during the COVID-19 pandemic. Reflecting on the factors contributing
to achieving the course learning outcomes (Table 1), students and
teachers will be asked to rate each of the factors on a 5-point scale
ranging from ‘no influence’ to ‘major influence’.

During the COVID-19 pandemic, both teachers and students have

experienced a unique situation. This unique experience may likely
yield some uncertainty and stress and require them to adapt and
adjust to the new way of being during the pandemic and its
unforeseen challenges. Considering this, some of the factors outlined
 in Table 1 will be examined further in the surveys (e.g. adaptability
and resilience). For example, in addition to asking participants about
their perceived effectiveness to adapt to changes during the
pandemic, they will also be asked to complete a validated nine-item
adaptability scale [18] yielding a measure of their adaptability in
response to the pandemic.

The surveys will also examine students’ and teachers’ previous

experience of, preparedness for, interests in, supports received for,
and the challenges encountered in, learning and teaching remotely
during the pandemic. The student survey will ask to provide the time
students spend typically on learning online and offline including
synchronous (e.g. ‘live’ tutorials) and asynchronous activities (e.g.
posting to a discussion forum) at the peak of the pandemic. From an
economics perspective, the teacher survey will seek to establish an
estimate of the total additional hours that teachers have dedicated to
transitioning their teaching and assessments to be deliverable over
this duration against four categories: (a) considering change
(meeting, planning and problem solving); (b) creating change
(constructing the required teaching resources or processes, including
learning new systems to be able to do so); (c) delivering change (any
increase/decrease in teaching delivery time); and (d) supporting
change (managing and answering student academic queries and
welfare).

In addition, the surveys will include a handful of open-ended

questions. For example, the teacher survey will ask participants to
describe if they had any experience of providing online/remote
education prior to the COVID-19 pandemic and how that experience
prepared them for teaching during the pandemic. The student survey
will ask participants to elaborate on the reasons for their preference
for synchronous and asynchronous learning activities.

Both surveys will ask a range of demographic questions (e.g. sex,

cultural and ethnic background, course and year of study/teaching) in
order to define characteristics of the sample. The surveys will also
collect an expression of interest to participate in longitudinal
interviews (teachers) or focus groups (students).

Data analysis – quantitative data

Quantitative data relating to student performance, student evaluation
of units, quantity of teaching activities and resource utilization will be
tested for normality summarized and presented using descriptive
statistics. Where appropriate, data will be converted from natural
units to financial units (i.e. staff hours converted to salary
expenditure). Student demography, unit completion status, grades
and student evaluation of units will be analyzed using Analysis of
Variance with Bonferroni post-hoc comparison. Differences in these
outcomes will be compared across the 2019 and 2020 calendar years
using independent t-test. Multivariable logistic regression analysis,
adjusting for student demography and period of enrolment, will be
used to assess predictors of student performance. Differences in
student performance and evaluation of units will be analyzed using
inferential statistics including mean differences, ANOVA, linear and
logistic regression. A sensitivity analyses will be conducted to explore
different permutations of basic assumptions, such as varying the
academic level attributed to the staffing hours..
Data collection - Qualitative data
Longitudinal interviews and focus groups
It is anticipated that the extended duration of the pandemic may have
a lasting impact on student learning and confidence in undertaking
clinical responsibilities. The lack of clinical exposure for many months
at the peak of the pandemic and sustained changes to how students
learn (e.g. large group teaching replaced by online learning, minimal
peer and teacher interaction occurring in person) may impact
students’ learning styles and possibly effect preparedness for future
clinical practice. Identifying any barriers to learning early enough may
provide an opportunity to institute remediation changes to assist
student confidence and competence. Hence, a longitudinal follow up
following graduation provides insights if added supervision and
support is needed for new graduates completing the course during the
pandemic.

To assist in addressing the gaps in learning, an interview and focus

group guide was developed [19] with consensus from all researchers.
The initial qualitative exploration will focus on experience of teaching
and learning during COVID-19 and impact on students and outcomes.
A selection of final year students and academic teaching staff who
volunteer to participate in the focus groups from all health
professional programs, will be invited to participate based on a range
of demographics, courses, academic levels etc. (estimated total 50
students and 25 academics for interviews). The members of the
research team, with no prior relationship with the student groups, will
conduct the focus group and interviews. The students will be
reassured of data being de-identified for reporting. A small sample of
students and teachers will be selected from the total sample that
participate in the initial interviews/focus groups for follow up over
time (approximately six students and six teachers).

Follow up interviews will be conducted at the conclusion of the year

(graduate point for students) and in 2021 (after graduation). The
purpose of the follow up is to explore longer-term impacts on teachers
and students. In particular, what have been the permanent changes to
teaching practice and for students/graduates, do they feel prepared
for practice. These interviews will take a narrative approach asking
participants to reflect on experiences of learning and the personal and
professional impacts on them. Data will follow graduates from 2020
into their first year of work aiming to capture any impact (positive or
negative) COVID-19 has had on their employability or work-readiness.

Data analysis - Qualitative data

Data from focus groups and interviews will be transcribed verbatim.
Interview data will be analyzed using Ritchie and Spencer’s [20] five-
stage framework analysis, which include familiarization, identifying
the coding framework, indexing, charting, and mapping and
interpretation. We will ensure that we achieve sufficient information
power through: [1] focused research questions; the specificity of
participants; [3] high quality interview dialogue; [4] large amount of
data; and [5] a structured, team-based approach to analysis [21].

In order to ensure trustworthiness in the analysis of qualitative data,

we will employ the team-based five-stage framework analysis
approach [20] with the use of NVivo. Each stage of the analysis will
involve discussion in several rounds of team meetings to compare,
contrast and negotiate our interpretations of the data. In addition, we
are aware of our positioning in the research through completing a
team reflexivity exercise [22] at the beginning of the study. This
provide us with a valuable opportunity to understand our diverse
background and perspectives that will support more rigorous data
interpretation with team members contributing different perspectives
and insights into the data collection, analysis and reporting.
Data analysis - Interpretation
The synthesized findings from each element of the study will be
examined side by side by the research team. Comparisons and
connections between findings from each data set, including
similarities and differences will be identified, discussed and debated
[15].
Discussion
The study aims to evaluate the impact of online education on health
science and health professions education in Australia. It is a broad-
based faculty-wide evaluation that encompasses the key teaching and
learning initiatives introduced in health professions education at the
university. Along with being widely implemented over the various
courses in biomedical science and health professional education, it
also aims to evaluate longitudinal outcome of student learning
attributes and impact on graduate outcomes. While there is much
being published about the impact of COVID-19 on teaching and
learning, this study will be unique in that it will follow the longer-term
impacts.

With the introduction of online education delivered over a prolonged

duration (up to many months with or without the reintroduction of
 face-to-face learning), the evaluation will also address how the
university workforce engages with a modified curriculum design. It
may allow faculty to co-teach units with experienced staff, and hence
promote mentorship. Flexible organization of online lessons, with off-
site teaching of content may generate a change in curriculum delivery
that is worthy of a detailed review. The faculty may be confronted
with challenges to create these learning resources; some of them may
be quite novel and innovative. It will be valuable to study if the
anticipated results are achieved after introducing these variations to
course delivery. This can only be achieved through a systematic
approach to evaluation, of these educational initiatives during and
after the pandemic.

There is a need to assess the learning outcomes achieved through

online education. Evidence on feasibility, benefits, shortcomings and
modifiable drivers of economic impact of the different types of
initiatives that are introduced in online education (and whether they
work or not) is needed. If they are found be helpful by learners and
teachers, possibly, some of these may be considered for integration in
courses in the long term after the pandemic has ceased, and not just
be limited to the current situation with enforced distance education
[23].

The strengths of this study lie in the large and diverse sample of
students, the use of a framework for the selection of variables to
examine in detail and the mixed method design that will allow both
real and relative concepts to be identified.0..0