Papers by Prof Elizabeth (Liz) Archer
Frontiers in Education: Assessment & Evaluation in Higher Education, 2023
Grit has been associated with student success in a plethora of higher education (HE) studies. In ... more Grit has been associated with student success in a plethora of higher education (HE) studies. In order to measure and evaluate the predictive validity of grit in student success in a particular context, an appropriate, reliable and valid instrument is required. This paper examines the short grit scale's (Grit-S) psychometric properties among a sample of 837 postgraduate students enrolled at the University of South Africa (UNISA). This appears to be the first study to explore the psychometric rigour of the scale among postgraduates within a South African distance education (DE) setting. An exploratory factor analysis (EFA) followed by partial confirmatory factor analysis (PCFA) revealed that the same two factors commonly reported in the grit literature could be detected in the items tested (i.e. 'passion' and 'perseverance' for long-term goals), and in combination explained 57.22% of the variance. Model fit indices for the two-factor solution were satisfactory (NFI = 0.984; TLI = 0.981; CFI = 0.991; RMSEA = 0.038). Further results from the item analysis indicated that both factors (passion and perseverance) were soundly reliable, as was the overall GritS scale, yielding Cronbach Alpha values of ≥0.77. Overall, the study provides promising evidence that the GritS scale can validly and reliably be used to operationalise grit (and the separate dimensions of perseverance and passion) among postgraduates pursuing their DE studies. Further theoretical and practical implications are discussed.
The COVID-19 pandemic has led to rapid change, unprecedented in higher education. One such change... more The COVID-19 pandemic has led to rapid change, unprecedented in higher education. One such change has been the almost complete shift to online assessment. The simultaneous employment of online assessment and proctoring has not enjoyed the rigorous academic debate and research traditionally associated with such shifts in academia. This engagement is essential and this article aims to discuss aspects of social justice, ethics and the validity of digital proctoring to the burgeoning debate. Digital proctoring is a lucrative industry (Coghlan Miller & Paterson, 2021), notwithstanding the admitted opportunities for cheating, irrespective of the intensity of overwatch. Digital proctoring is marketed and has become entangled with issues of institutional reputation and the legitimacy of qualifications. The student seems to be a secondary consideration compared to the technocratic digital proctoring arena. However, the introduction of online assessment, specifically with digital proctoring, impacts the assessment's validity by introducing intervening variables into the process. The drive to detect and prevent online cheating has led to algorithmic proliferation. This technologically driven approach has embedded social injustice and questionable ethics and validity into the assessment systems. This article examines the social justice, ethical and validity issues around technological proctoring under the grouped themes: Emotional factors; Racial and/or skin colour; Digital literacy and Technology; and Disability. However, the COVID-19 pandemicdriven shifts have provided the unprecedented opportunity to elevate assessment from recall to critical thinking and applicationbased assessment. An opportunity to ensure that our assessment is valid, assesses higher-order learning, and truly evaluates the concepts we wish to assess.
I am an established academic, NRF rated researcher, leader and innovator. I have two decades of e... more I am an established academic, NRF rated researcher, leader and innovator. I have two decades of experience in Higher Education. I am passionate and motivated, dedicated to student and institutional success Cape Town,
The Journal for Transdisciplinary Research in Southern Africa, 2013
There has been an increased emphasis on providing schools with feedback on performance of learner... more There has been an increased emphasis on providing schools with feedback on performance of learners with the aim of improving the quality of education. However, if feedback on learner performance is to be effectively used by schools, then it is important to understand what the informational needs of the schools are and whether schools can access the information. Thus, one research question is posed in this article namely: How can the current presentation of performance data captured in the school reports and feedback sessions be improved? The conceptual framework for the study draws on evaluation studies focusing on the use and usefulness of information. Methodologically, design research using mixed methods was employed. A needs analysis of what information was essential was undertaken. Six primary and secondary schools were purposively sampled to participate in the needs analysis from which the first prototype of the report and feedback sessions was developed. This needs analysis wa...
The International Review of Research in Open and Distributed Learning, 2014
Student success and retention is a primary goal of higher education institutions across the world... more Student success and retention is a primary goal of higher education institutions across the world. The cost of student failure and dropout in higher education is multifaceted including, amongst other things, the loss of revenue, prestige, and stakeholder trust for both institutions and students. Interventions to address this are complex and varied. While the dominant thrust has been to investigate academic and non-academic risk factors thus applying a "risk" lens, equal attention should be given to exploring the characteristics of successful students which expands the focus to include "requirements for success". Based on a socio-critical model for understanding of student success and retention, the University of South Africa (Unisa) initiated a pilot project to benchmark successful students' habits and behaviours using a tool employed in business settings, namely Shadowmatch®. The original focus was on finding a theoretically valid measured for habits and behaviours to examine the critical aspect of student agency in the social critical model. Although this was not the focus of the pilot, concerns regarding using a commercial tool in an academic setting overshadowed the process. This paper provides insights into how academic-business collaboration could allow an institution to be more dynamic and flexible in supporting its student population.
The International Review of Research in Open and Distributed Learning, 2015
Open Educational Resources (OER) emerged within the context of open education which is typically ... more Open Educational Resources (OER) emerged within the context of open education which is typically characterized by the sharing of knowledge and resources and the exchange of ideas. Unisa as a mega open distance learning (ODL) university has publicly communicated its intention to take part in the use and creation of OER. As global and local university research on OER is limited, this prompted an investigation to gauge the uptake of OER at Unisa, by staff, with the purpose of institutional information gathering for decision making and planning in this area. During 2014, a survey was undertaken for this reason. The survey examined knowledge of OER, Intellectual Property (IP) Rights and Licensing, participation in OER, barriers to OER and OER in the Unisa context with a view to determining the stage at which the institution is in terms of adopting and engaging with the OER initiative. The results indicated that although there is knowledge and understanding of OER, this has not been conve...
The International Review of Research in Open and Distributed Learning, 2015
In the context of the hype, promise and perils of Big Data and the currently dominant paradigm of... more In the context of the hype, promise and perils of Big Data and the currently dominant paradigm of data-driven decision-making, it is important to critically engage with the potential of Big Data for higher education. We do not question the potential of Big Data, but we do raise a number of issues, and present a number of theses to be seriously considered in realising this potential. The University of South Africa (Unisa) is one of the mega ODL institutions in the world with more than 360,000 students and a range of courses and programmes. Unisa already has access to a staggering amount of student data, hosted in disparate sources, and governed by different processes. As the university moves to mainstreaming online learning, the amount of and need for analyses of data are increasing, raising important questions regarding our assumptions, understanding, data sources, systems and processes. This article presents a descriptive case study of the current state of student data at Unisa, as well as explores the impact of existing data sources and analytic approaches. From the analysis it is clear that in order for big(ger) data to be better data, a number of issues need to be addressed. The article concludes by presenting a number of theses that should form the basis for the imperative to optimise the harvesting, analysis and use of student data.
Assessment & Evaluation in Higher Education, 2019
Abstract Assessment and learning analytics both collect, analyse and use student data, albeit dif... more Abstract Assessment and learning analytics both collect, analyse and use student data, albeit different types of data and to some extent, for various purposes. Based on the data collected and analysed, learning analytics allow for decisions to be made not only with regard to evaluating progress in achieving learning outcomes but also evaluative judgments about the quality of learning. Learning analytics fall in the nexus between assessment of and for learning. As such it has the potential to deliver value in the form of (1) understanding student learning, (2) analysing learning behaviour (looking to identify not only factors that may indicate risk of failing, but for opportunities to deepen learning), (3) predicting students-at-risk (or identifying where students have specific learning needs), and (4) prescribing elements to be included to ensure not only the effectiveness of teaching, but also of learning. Learning analytics have underlying default positions that may not only skew their impact but also impact negatively on students in realising their potential. We examine a selection of default positions and point to how these positions/assumptions may adversely affect students’ chances of success, deepening the understanding of learning.
Globally, a wealth of educational data has been collected on learner performance in a bid to impr... more Globally, a wealth of educational data has been collected on learner performance in a bid to improve and monitor the quality of education. Unfortunately, the data seem to have had only limited influence on learning and teaching in classrooms. This thesis aimed to bridge this gap between the availability of learner performance data and their use in informing planning and action in schools. A design research approach was used to optimise the feedback system for the South African Monitoring system for Primary schools (SAMP). Design research aims to produce both an intervention to address a complex real-world challenge and to develop design guidelines to support other designers faced with similar challenges in their own context. In this research, the process of developing and improving the feedback system was also used to examine ways of facilitating the use of the feedback. Multiple cycles of design, implementation and evaluation of four different prototypes of the feedback system were...
Perspectives in Education, 2010
Reform and improvement are imperative in the current South African education system. Monitoring o... more Reform and improvement are imperative in the current South African education system. Monitoring of school and learner achievement is an essential for establishing praxis for school improvement. Diversity of cu lture and South Africa’s 11 official l anguages m ake it difficult to develop valid monitoring systems. Limited resources, time constraints and the need to redress neglect of large portions of the education infrastructure from the apartheid era make it problematic to develop new monitoring systems for all official languages. Adaptation and translation of existing international monitoring instruments provide a lternative solutions to developing new monitoring systems. Adaptation and translation of existing instruments is a daunting process, which balances statistical analysis, translation processes and user and expert evaluations. We investigate how to balance these different processes in order to create an instrument that provides valid data for educational decisions. The proc...
Transforming Research Methods in the Social Sciences
Transforming Research Methods in the Social Sciences, 2019
Reexamining dashboard development: putting the horse back in front of the cart During this age of... more Reexamining dashboard development: putting the horse back in front of the cart During this age of data proliferation, heavy reliance is placed on data visualisation to support users in making sense of vast quantities of information: Finding the signal in the noise (Silver 2012). Informational Dashboards have become the must have accoutrement for Higher Education institutions with various stakeholders jostling for development priority. Due to the time pressure and user demands, the focus of development process is often on designing for each stakeholder and the visual and navigational aspects. Dashboards are designed to make data visually appealing and easy to relate and understand; unfortunately this may mask data issues and create an impression of rigour where it is not warranted. This paper explores an alternative approach to data visualisation and dashboard design. It suggests that the first step should be the development of an enriched database which integrates key indicators from various data sources and then acts as a database for various dashboards. The emphasis is thus on the underlying value-added database which can then be overlaid with any user interface for dashboard generation for a multitude of stakeholders. The enriched central database thus becomes a menu of available quality assured data which can easily be drawn into various dashboards at various levels of granularity ensuring improved data quality management, responsiveness, efficiency and flexibility. A case study is employed illustrating the benefits, showcasing various views developed for diverse stakeholders employing this approach. Reexamining dashboard development: putting the horse back in front of the cart Presentation Reexamining dashboard development: putting the horse back in front of the cart The aim of the paper is to explore sound approaches of meeting stakeholder data visualisation requirements in the era of overwhelming data demand and supply, while maintaining data integrity, consistency and flexibility. Traditionally higher education institutions always had access to relatively large data sets and tools for analysis. This is growing exponentially with the ever increasing amount of digital student data that can be harvested and analysed (Swain, 2013), as well as increased technological and analytical capabilities (Wishon & Rome, 2012). Learning analytics has been described as the "new black" (Booth, 2012), and student data as the "new oil" (Watters, 2013). The 2013 NMC New Horizon report: Higher Education Edition (New Media Consortium, 2013) identifies learning analytics as one of the key emerging technologies to enter mainstream use from 2015-2016. The report (New Media Consortium, 2013) also identifies dashboards as a key technology in leveraging the power of data at all stakeholder levels. Rationale With the exponential growth of data, with multiple complex relationships, differing constraints, the ever changing environment, uncertainty and time pressure, problem-solving and decision-making can be overwhelming. Dashboards have emerged as a technology to enable stakeholders to engage with the data effectively through various data visualisations (Abd-elfattah, Alghamdi, & Amer, 2014). A dashboard can be defined as: "… a visual display of the most important information needed to achieve one or more objectives; consolidated and arranged on a single screen, so the information can be monitored at a glance" (Few, 2006, p. 26). Dashboards started to gain traction in the 1980's in the business world and have since gained popularity in Higher Education (Newman, Thomas, & Webber, 2014). Many tools have been developed for the purpose of designing dashboards and disseminating information in graphic formats, these include IBM Cognos, Oracle BI Foundation Suite, SAS Enterprise Intelligence Platform, SAP Business Object BI Platform, MicroStrategy, QlikView and Web Focus, amongst others.
This chapter forms part of the book: Educational Design Research "Plomp, T & Nieveen, N (Eds... more This chapter forms part of the book: Educational Design Research "Plomp, T & Nieveen, N (Eds.), Educational design research – Part B: Illustrative cases (pp. 71-93). Enschede, the Netherlands: SLO. The book "Educational design research" comprises two parts: Educational design research - Part A: An introduction Part A provides an introduction into educational design research as a suitable research approach either to address complex problems in educational practice or to develop or validate theories (e.g.) about learning processes, learning environments and the like. A new edition of this book is planned for October 2013. Educational design research - Part B: Illustrative cases Part B contains a collection of 51 examples of successful educational design research projects written by researchers from more than 20 countries. These examples enable graduate students and novice researchers to learn how to design and conduct a project utilizing an educational design research a...
The International Review of Research in Open and Distributed Learning
During this age of data proliferation, heavy reliance is placed on data visualisation to support ... more During this age of data proliferation, heavy reliance is placed on data visualisation to support users in making sense of vast quantities of information. Informational Dashboards have become the must have accoutrement for Higher Education institutions with various stakeholders jostling for development priority. Due to the time pressure and user demands, the focus of development process is often on designing for each stakeholder and the visual and navigational aspects. Dashboards are designed to make data visually appealing and easy to relate and understand; unfortunately this may mask data issues and create an impression of rigour where it is not justified. This article proposes that the underlying logic behind current dashboard development is limited in the flexibility, scalability, and responsiveness required in the demanding landscape of Big Data and Analytics and explores an alternative approach to data visualisation and sense making. It suggests that the first step required to address these issues is the development of an enriched database which integrates key indicators from various data sources. The database is designed for problem exploration allowing users freedom in navigating between various data-levels, which can then be overlaid with any user interface for dashboard generation for a multitude of stakeholders. Dashboards merely become tools providing users and indication of types of data available for exploration. A Design Research approach is shown, along with a case study to illustrate the benefits, showcasing various views developed for diverse stakeholders employing this approach, specifically the the Digital Decision Network Application (DigitalDNA) employed at Unisa.
Frontiers in Education
This article introduces the concept of the assessment purposes triangle to illustrate the balanci... more This article introduces the concept of the assessment purposes triangle to illustrate the balancing of purposes that needs to be considered in assessment for educational quality. The triangle is a visual aid for stakeholders making value judgments about equitable resource allocation in an environment of scarcity. This article argues that each of the three basic purposes of assessment, assessment to support learning; assessment for accountability; assessment for certification, progress, and transfer need to enjoy appropriate attention to support quality education. Accountability and certification, progress, and transfer are inherently high stakes due to systemic pressures such as institutional funding, accreditation with various bodies, legislative requirements, national and international competition, and public and media pressure. It is thus likely that resources will be diverted to these purposes from the lower stakes assessment to support learning. This article illustrates the inherent dangers of overemphasizing certain purposes to the detriment of others in any educational system. It also discusses the power inherent in assessment and argues for a harnessing of this power for quality education through catering for the three basic assessment purposes in an equitable manner. All actors in an educational system should be cognizant of the goals of the education system as a whole when developing and engaging with assessment. This article provides a conceptual tool for stakeholders in educational assessment to engage with these goals when considering the allocation of appropriate resources to each of the assessment purposes.
Monitoring the Quality of Education in Schools, 2017
Uploads
Papers by Prof Elizabeth (Liz) Archer
This syllabus covers a training that is meant to assist health professionals in using qualitative evaluation skills in sound and rigorous evaluation of their program. The sessions go beyond basic concepts to explore important considerations of qualitative methods in the context of rigorous evaluation. Through session content and participatory exercises, participants will gain basic skills in rigorous qualitative data collection, analysis, and use.
This syllabus provides an overview of the ten-day (8.5 working days) training workshop, including presentations, facilitator and participant guides, practical sessions, case studies, and sample agendas.
A complete collection of resources for MEASURE Evaluation's Qualitative Methods in Evaluation of Public Health Programs Curriculum can be found here.
Filed under: Data Demand and Use, Gender, Curriculum, Data Quality Review, Data Quality, Evaluation, Data use, Qualitative Evaluation, Gender Norms