OUDES

Increasingly, researchers are being asked to make statements about the impact of their work. The idea of impact varies across university disciplines, and this session explored the range of researcher perspectives on impact from the humanities; medical sciences; mathematical, physical and life sciences; and social sciences. How researchers interpret the value of their work is an important subject for education, particularly since research in education is usually expected to strive for improvement and effect.

This article is a reflection on an area of particular interest in the current research environment, but which has not yet been explored satisfactorily in the education literature: the evaluation of educational research. The particular focus is on the UK context, but the article is informed by comparative evidence from six countries (gathered through analysis of policy and administrative documents, literature review, informal discussion and written requests for information from key persons). It identifies eight recent trends in the evaluation of education research (from performance-based funding and institutionalisation of assessment, to the de-sensitivisation of research assessment) and it explores the benefits and perils of three types of assessment procedures (peer review, bibliometrics and econometrics) as they operate at a micro, meso and macro level. The article argues that current evaluations of educational research (particularly those aimed at supporting funding decisions) tend to operate from an instrumental standpoint that largely ignores the epistemic specificity of the various fields, modes or genres of research, the assumptions about knowledge with which they work, and the cultural and social dimensions of research evaluation as a practice.

This paper explores recent public debates around research assessment and its future as part of a dynamic landscape of governance discourses and practices, and organisational, professional and disciplinary cultures. Drawing reflectively on data from RAE 2001, RAE 2008 and REF 2014 (reported elsewhere), the paper highlights how recent debates around research assessment echo longer-term changes in research governance. The following changes, and several critiques of their implications, are discussed: shifts in the principles for governing research and the rise of multipurpose assessment; the spread of performance-based funding and external accountability for research; the use of metrics and indicators in research assessment; the boundary work taking place in defining and classifying units or fields for assessment; the emphasis on research impact as a component of research value; organisational recalibration across the sector; and the specialisation of blended professional practice. These changes are underpinned by persistent tensions around accountability; evaluation; measurement; demarcation; legitimation; agency; and identity in research. Overall, such trends and the discursive shifts that made them possible have challenged established principles of funding and governance and have pushed assessment technologies into a pivot position in the political dynamics of renegotiating the relationships between universities and the state. Jointly, the directions of travel identified in this paper describe a widespread and persistent regime of research governance and policy that has become embedded in institutional and individual practices.

Incentives for improving research productivity at universities prevail in global academia. However, the rationale, methodology, and impact of such incentives and consequent evaluation regimes are in need of scrutinization. This paper explores the influences of financial and career-related publishing incentive schemes on research cultures. It draws on an analysis of 75 interviews with academics, senior university administrators, and journal editors from China, a country that has seen widespread reliance on international publication counts in research evaluation and reward systems. The study focuses on humanities and social sciences (HSS) as disciplinary sites, which embody distinct characteristics and have experienced the introduction of incentive schemes in China since the early 2000s. Findings reveal tensions between internationalization and indigenization, quality and quantity, integrity and instrumentalism, equity and inequity in Chinese academia. In particular, we argue that a b...

This paper presents a systematic review of scholarly efforts that uniquely emerged at the onset of the COVID-19 pandemic and focused primarily on higher education teachers’ perspectives on technology use and on associated changes in the relationship between teachers and students amidst the transition to emergency remote teaching worldwide. Our narrative synthesis of 32 studies, the majority of which come from lower-and middle-income countries/regions, suggests that numerous factors interact to shape academics’ technology use in emergency remote teaching across higher education contexts. We report strong findings of teachers’ resilience and resourcefulness in their self-exploration of various technologies and teaching strategies in response to the continued severity of the pandemic. Ultimately, this review suggests directions for further research on engaging educational leaders and faculty in reimagining teaching as not only a core academic function of higher education, but also, and...

This study explored conceptually and empirically the ways in which those engaged with university-based arts and humanities research (researchers, managers, partners, beneficiaries) construct and respond to the challenges of generating, interpreting, and demonstrating the cultural value of research. Cultural value is a contested concept, beset by philosophical, practical and political tensions. We argue that interpretations of valuecultural or otherwiseare part of complex ecologies of cultural life, creation and understanding, while at the same time underpinning economies of description, prescription, inscription and ascription. Meaning, expression, narrative and practice, combined and recombined in experience, are core themes in our participants' description of the arts and the humanities. However, more needs to be done across all levels of the research governance system so that meaningful engagement is sustained and narratives in cultural terms are not perceived as a risk in accountability contexts.

Increasingly, researchers are being asked to make statements about the impact of their work. The idea of impact varies across university disciplines, and this session explored the range of researcher perspectives on impact from the humanities; medical sciences; mathematical, physical and life sciences; and social sciences. How researchers interpret the value of their work is an important subject for education, particularly since research in education is usually expected to strive for improvement and effect.

This article introduces a methodological approach for articulating and communicating the impact and value of research: qualitative network analysis using collaborative configuration tracing and visualization. The approach was proposed initially in Oancea (Interpretations and Practices of Research Impact across the Range of Disciplines Report, Oxford, Oxford University, 2011) and was refined and tested in a 2013-14 study funded by the Arts and Humanities Research Council. It uses co-constructed qualitative network diagrams to enable the systematic elicitation and visualization of information from participants (such as researchers, administrators, facilitators, partners, users, and beneficiaries of research) about the different flows and relationships that they see as relevant to creating, articulating, and demonstrating impact and value from research. Unlike quantitative network studies, the emphasis here is on the process of construction and interpretation of qualitative network maps by the participants. Subject to further testing and refinement and to critical understanding of the conceptual, technical, practical, and political limitations of measurement in this area, the approach that we have developed can be adapted for use in research, evaluation, communication, engagement, knowledge exchange, and developmental work in higher education institutions and funding organizations. Key words: qualitative network analysis; network mapping; cultural value; research impact; arts and humanities research. 1. Background 'For the purposes of the REF, impact is defined as an effect on, change or benefit to the economy, society, culture, public policy or services, health, the environment or quality of life, beyond academia' (REF 2014, para 140-our emphasis) 'Research across the arts and humanities (understood in their broadest definition) has consequences for individuals and groups in the UK and internationally, challenging imaginations and enriching lives economically, culturally, spiritually and educationally. The impact of such research is powerful, pervasive and ubiquitous, influencing civil society and the quality of life' (REF 2014 Panel D, para 83-our emphasis) 'Institutions will echo a top-down rhetoric in order to make a case for resources but it's not how people describe what they do' (interview participant, Oancea, Florez-Petour and Atkinson 2015).

Incentives for improving research productivity at universities prevail in global academia. However, the rationale, methodology, and impact of such incentives and consequent evaluation regimes are in need of scrutinization. This paper explores the influences of financial and career-related publishing incentive schemes on research cultures. It draws on an analysis of 75 interviews with academics, senior university administrators, and journal editors from China, a country that has seen widespread reliance on international publication counts in research evaluation and reward systems. The study focuses on humanities and social sciences (HSS) as disciplinary sites, which embody distinct characteristics and have experienced the introduction of incentive schemes in China since the early 2000s. Findings reveal tensions between internationalization and indigenization, quality and quantity, integrity and instrumentalism, equity and inequity in Chinese academia. In particular, we argue that a blanket incentive scheme could reinforce a managerial culture in higher education, encourage performative objectification of academics, and jeopardize their agency. We thereby challenge 'one-size-fits-all' policymaking, and suggest instead that institutions should have the opportunity to adopt an ethical and 'human-oriented' approach when developing their research incentives and evaluation mechanisms.

This paper explores recent public debates around research assessment and its future as part of a dynamic landscape of governance discourses and practices, and organisational, professional and disciplinary cultures. Drawing reflectively on data from RAE 2001, RAE 2008 and REF 2014 (reported elsewhere), the paper highlights how recent debates around research assessment echo longer-term changes in research governance. The following changes, and several critiques of their implications, are discussed: shifts in the principles for governing research and the rise of multipurpose assessment; the spread of performance-based funding and external accountability for research; the use of metrics and indicators in research assessment; the boundary work taking place in defining and classifying units or fields for assessment; the emphasis on research impact as a component of research value; organisational recalibration across the sector; and the specialisation of blended professional practice. These changes are underpinned by persistent tensions around accountability; evaluation; measurement; demarcation; legitimation; agency; and identity in research. Overall, such trends and the discursive shifts that made them possible have challenged established principles of funding and governance and have pushed assessment technologies into a pivot position in the political dynamics of renegotiating the relationships between universities and the state. Jointly, the directions of travel identified in this paper describe a widespread and persistent regime of research governance and policy that has become embedded in institutional and individual practices.

This study examines Chinese universities' incentive schemes for international publication in the humanities and social sciences (HSS). It analysed 172 incentive documents collected at 116 research-intensive universities in China, including monetary bonus schemes and career-related incentives. This analysis is complemented by interviews with six senior administrators at six Chinese universities and four journal editors to explore the policy-making process and impacts of incentivisation. The study found that most universities actively promoted international publications, with variations and similarities in their incentive schemes. In general, more benefits were granted for SSCI (Social Sciences Citation Index) and A&HCI (Arts and Humanities Citation Index) journal publications than domestic publications. We conclude such incentivisation showcased the influence of western standards and global benchmarking in the internationalisation of Chinese HSS, and that the incentive schemes may create a 'Matthew Effect' enabling SSCI and A&HCI journals to flourish, while deepening the divide between these and other journals.

This paper is a reflection on an area of particular interest in the current research environment, but which has not yet been explored satisfactorily in the education literature: the evaluation of educational research. The particular focus is on the United Kingdom context, but the paper is informed by comparative evidence from six countries (gathered through analysis of policy and administrative documents, literature review, informal discussion, and written requests for information from key persons). It identifies eight recent trends in the evaluation of education research (from performance-based funding and institutionalisation of assessment, to the de-sensitivisation of research assessment) and it explores the benefits and perils of three types of assessment procedures (peer review, bibliometrics, economic metrics) as they operate at a micro, meso and macro level. The paper argues that current evaluations of educational research (particularly those aimed at supporting funding decisions) tend to operate from an instrumental standpoint that largely ignores the epistemic specificity of the various fields, modes or genres of research, the assumptions about knowledge with which they work, and the cultural and social dimensions of research evaluation as a practice.

This paper uses empirical research data and theoretical insights from the literature on governance to problematise some of the arguments presented in the research assessment literature, in particular, the description of the UK RAE/REF as a mechanism for top-down control with strongly negative, blanket-impacts on disciplines, institutions and individual researchers. The concepts of performativity, accountability and governmentality are employed to unpack normative claims about negative impacts and conflicts of values, and empirical claims about the nature of changes in behavior, attitude and interpretation, as reported by the researchers surveyed. The paper argues that inherent, multiple ambivalences of the RAE as a governance technology operate at the transition points between traditional and contemporary forms of governing and account for the mixed picture of its impacts at system, field, institutional, and individual level.