Using semi-structured interviews with n=69 global research stakeholders, , this research explores... more Using semi-structured interviews with n=69 global research stakeholders, , this research explores the ways in which stakeholders within system-level research governance organisations conceptualised, responded to, and reasoned the realities of disruption caused by the COVID-19 pandemic, and how they positioned procedural changes to their governance mechanisms. Given that shocks to systems present critical challenges to established practices and embedded institutional norms, we use neo-institutional theory as a heuristic device to examine the relationship between the exogenous shock of COVID-19, trajectories of institutional norms and cultures, and the role institutional stakeholders play in managing responses. Across all the research systems studied (with particular focus on the UK, Australia, Norway, New Zealand, Hong Kong SAR and Italy), participants were concerned about how the shock provided by COVID-19 had both revealed and entrenched deep inequalities (between individuals and b...
There are two key questions at the heart of the ongoing debate about education and training for a... more There are two key questions at the heart of the ongoing debate about education and training for all young people, irrespective of background, ability or attainmen
Zeitschrift Fur Erziehungswissenschaft, Nov 1, 2014
This paper uses empirical research data and theoretical insights from the literature on governanc... more This paper uses empirical research data and theoretical insights from the literature on governance to problematise some of the arguments presented in the research assessment literature, in particular, the description of the UK RAE/REF as a mechanism for top-down control with strongly negative, blanket-impacts on disciplines, institutions and individual researchers. The concepts of performativity, accountability and governmentality are employed to unpack normative claims about negative impacts and conflicts of values, and empirical claims about the nature of changes in behavior, attitude and interpretation, as reported by the researchers surveyed. The paper argues that inherent, multiple ambivalences of the RAE as a governance technology operate at the transition points between traditional and contemporary forms of governing and account for the mixed picture of its impacts at system, field, institutional, and individual level.
A panel session reflecting on research impact and knowledge exchange from different angles, from ... more A panel session reflecting on research impact and knowledge exchange from different angles, from user perspectives and wide public debates, through institutional contexts and the interfaces with different funding bodies, and to international experiences. The aim of the panel session was to reflect on research impact and knowledge exchange from different angles, from user perspectives and wide public debates, through institutional contexts and the interfaces with different funding bodies, and to international experiences. The session drew to a close a seminar series that included contributions from academics, funding bodies, and research managers, nationally and internationally, in the social sciences and beyond. The series included discussion of metrics, as well as of narrative approaches to articulating impact; of the REF and institutional responses to it, as well as of individual academics' engagement with impact, and the challenges, benefits and dilemmas arising from it; of practical, as well as conceptual and critical, aspects of research impact. In this final session of the series, each panellist made a brief opening contribution, followed by 40 minutes of discussion with the audience
This study explored the diverse ways in which those engaged in university-based arts and humaniti... more This study explored the diverse ways in which those engaged in university-based arts and humanities research (researchers, administrators, partners, beneficiaries) construct and respond to the challenges of interpreting, enacting, and demonstrating the cultural value of research. We refined and tested a methodological approach, proposed initially in Oancea(2011), that is based on qualitative network analysis, or configuration tracing and analysis. The data collected include well over 70 hours of interviews, 24 co-produced configuration drawings,email follow-up and qualitative answers to an online questionnaire. In addition, we reviewed clusters of theoretical resources that had either been mobilised in related research, or we deem ripe for further exploitation in this field. These resources include theories of the valuation of cultural goods; critiques of cultural valuation; and philosophical, cultural-psychological and ecological approaches to cultural values/s. Our review of these resources highlights the limits of dualist thinking, but also the theoretical and methodological relevance of concepts such as networks, interaction, intersubjectivity, configurations, texture, flows, and ecology. The study has furthered our understanding of cultural value as a contested concept, beset by conceptual, philosophical, practical and political tensions. It has taken us through many crisscrossing discourses and has shown us that interpretations of value – cultural or otherwise –underpin economies of description, prescription, inscription and ascription, while at the same time being part of complex ecologies of cultural life, creation and understanding. Meaning,expression, narrative and practice, combined and recombined in experience through the interaction between self and others, sit at the core of our participants’ description of the arts and the humanities. We set off on this study with a theoretical and methodological interest in tracing conceptual and empirical configurations of value from research, as articulated and practiced by our respondents. Our evolving design enabled us to take guidance from the participants in order to follow such configurations flexibly. This process has showed us the fruitfulness, as well as the challenges, of the form of qualitative network analysis that we have developed. Subject to further testing and refinement, this approach is amenable to use in research, evaluation,communication and developmental work.
International Journal of Research & Method in Education, Nov 1, 2007
This paper is a reflection on an area of particular interest in the current research environment,... more This paper is a reflection on an area of particular interest in the current research environment, but which has not yet been explored satisfactorily in the education literature: the evaluation of educational research. The particular focus is on the United Kingdom context, but the paper is informed by comparative evidence from six countries (gathered through analysis of policy and administrative documents, literature review, informal discussion, and written requests for information from key persons). It identifies eight recent trends in the evaluation of education research (from performance-based funding and institutionalisation of assessment, to the de-sensitivisation of research assessment) and it explores the benefits and perils of three types of assessment procedures (peer review, bibliometrics, economic metrics) as they operate at a micro, meso and macro level. The paper argues that current evaluations of educational research (particularly those aimed at supporting funding decisions) tend to operate from an instrumental standpoint that largely ignores the epistemic specificity of the various fields, modes or genres of research, the assumptions about knowledge with which they work, and the cultural and social dimensions of research evaluation as a practice.
This study examines Chinese universities' incentive schemes for international publication in the ... more This study examines Chinese universities' incentive schemes for international publication in the humanities and social sciences (HSS). It analysed 172 incentive documents collected at 116 research-intensive universities in China, including monetary bonus schemes and career-related incentives. This analysis is complemented by interviews with six senior administrators at six Chinese universities and four journal editors to explore the policy-making process and impacts of incentivisation. The study found that most universities actively promoted international publications, with variations and similarities in their incentive schemes. In general, more benefits were granted for SSCI (Social Sciences Citation Index) and A&HCI (Arts and Humanities Citation Index) journal publications than domestic publications. We conclude such incentivisation showcased the influence of western standards and global benchmarking in the internationalisation of Chinese HSS, and that the incentive schemes may create a 'Matthew Effect' enabling SSCI and A&HCI journals to flourish, while deepening the divide between these and other journals.
This paper explores recent public debates around research assessment and its future as part of a ... more This paper explores recent public debates around research assessment and its future as part of a dynamic landscape of governance discourses and practices, and organisational, professional and disciplinary cultures. Drawing reflectively on data from RAE 2001, RAE 2008 and REF 2014 (reported elsewhere), the paper highlights how recent debates around research assessment echo longer-term changes in research governance. The following changes, and several critiques of their implications, are discussed: shifts in the principles for governing research and the rise of multipurpose assessment; the spread of performance-based funding and external accountability for research; the use of metrics and indicators in research assessment; the boundary work taking place in defining and classifying units or fields for assessment; the emphasis on research impact as a component of research value; organisational recalibration across the sector; and the specialisation of blended professional practice. These changes are underpinned by persistent tensions around accountability; evaluation; measurement; demarcation; legitimation; agency; and identity in research. Overall, such trends and the discursive shifts that made them possible have challenged established principles of funding and governance and have pushed assessment technologies into a pivot position in the political dynamics of renegotiating the relationships between universities and the state. Jointly, the directions of travel identified in this paper describe a widespread and persistent regime of research governance and policy that has become embedded in institutional and individual practices.
1. The English White Paper, The Importance of Teaching, was discussed at the February 2011 meetin... more 1. The English White Paper, The Importance of Teaching, was discussed at the February 2011 meeting of the Universities‟ Council for the Education of Teachers (UCET Research and Development Committee. It seemed possible that proposed changes in initial teacher ...
Using semi-structured interviews with n=69 global research stakeholders, , this research explores... more Using semi-structured interviews with n=69 global research stakeholders, , this research explores the ways in which stakeholders within system-level research governance organisations conceptualised, responded to, and reasoned the realities of disruption caused by the COVID-19 pandemic, and how they positioned procedural changes to their governance mechanisms. Given that shocks to systems present critical challenges to established practices and embedded institutional norms, we use neo-institutional theory as a heuristic device to examine the relationship between the exogenous shock of COVID-19, trajectories of institutional norms and cultures, and the role institutional stakeholders play in managing responses. Across all the research systems studied (with particular focus on the UK, Australia, Norway, New Zealand, Hong Kong SAR and Italy), participants were concerned about how the shock provided by COVID-19 had both revealed and entrenched deep inequalities (between individuals and b...
There are two key questions at the heart of the ongoing debate about education and training for a... more There are two key questions at the heart of the ongoing debate about education and training for all young people, irrespective of background, ability or attainmen
Zeitschrift Fur Erziehungswissenschaft, Nov 1, 2014
This paper uses empirical research data and theoretical insights from the literature on governanc... more This paper uses empirical research data and theoretical insights from the literature on governance to problematise some of the arguments presented in the research assessment literature, in particular, the description of the UK RAE/REF as a mechanism for top-down control with strongly negative, blanket-impacts on disciplines, institutions and individual researchers. The concepts of performativity, accountability and governmentality are employed to unpack normative claims about negative impacts and conflicts of values, and empirical claims about the nature of changes in behavior, attitude and interpretation, as reported by the researchers surveyed. The paper argues that inherent, multiple ambivalences of the RAE as a governance technology operate at the transition points between traditional and contemporary forms of governing and account for the mixed picture of its impacts at system, field, institutional, and individual level.
A panel session reflecting on research impact and knowledge exchange from different angles, from ... more A panel session reflecting on research impact and knowledge exchange from different angles, from user perspectives and wide public debates, through institutional contexts and the interfaces with different funding bodies, and to international experiences. The aim of the panel session was to reflect on research impact and knowledge exchange from different angles, from user perspectives and wide public debates, through institutional contexts and the interfaces with different funding bodies, and to international experiences. The session drew to a close a seminar series that included contributions from academics, funding bodies, and research managers, nationally and internationally, in the social sciences and beyond. The series included discussion of metrics, as well as of narrative approaches to articulating impact; of the REF and institutional responses to it, as well as of individual academics' engagement with impact, and the challenges, benefits and dilemmas arising from it; of practical, as well as conceptual and critical, aspects of research impact. In this final session of the series, each panellist made a brief opening contribution, followed by 40 minutes of discussion with the audience
This study explored the diverse ways in which those engaged in university-based arts and humaniti... more This study explored the diverse ways in which those engaged in university-based arts and humanities research (researchers, administrators, partners, beneficiaries) construct and respond to the challenges of interpreting, enacting, and demonstrating the cultural value of research. We refined and tested a methodological approach, proposed initially in Oancea(2011), that is based on qualitative network analysis, or configuration tracing and analysis. The data collected include well over 70 hours of interviews, 24 co-produced configuration drawings,email follow-up and qualitative answers to an online questionnaire. In addition, we reviewed clusters of theoretical resources that had either been mobilised in related research, or we deem ripe for further exploitation in this field. These resources include theories of the valuation of cultural goods; critiques of cultural valuation; and philosophical, cultural-psychological and ecological approaches to cultural values/s. Our review of these resources highlights the limits of dualist thinking, but also the theoretical and methodological relevance of concepts such as networks, interaction, intersubjectivity, configurations, texture, flows, and ecology. The study has furthered our understanding of cultural value as a contested concept, beset by conceptual, philosophical, practical and political tensions. It has taken us through many crisscrossing discourses and has shown us that interpretations of value – cultural or otherwise –underpin economies of description, prescription, inscription and ascription, while at the same time being part of complex ecologies of cultural life, creation and understanding. Meaning,expression, narrative and practice, combined and recombined in experience through the interaction between self and others, sit at the core of our participants’ description of the arts and the humanities. We set off on this study with a theoretical and methodological interest in tracing conceptual and empirical configurations of value from research, as articulated and practiced by our respondents. Our evolving design enabled us to take guidance from the participants in order to follow such configurations flexibly. This process has showed us the fruitfulness, as well as the challenges, of the form of qualitative network analysis that we have developed. Subject to further testing and refinement, this approach is amenable to use in research, evaluation,communication and developmental work.
International Journal of Research & Method in Education, Nov 1, 2007
This paper is a reflection on an area of particular interest in the current research environment,... more This paper is a reflection on an area of particular interest in the current research environment, but which has not yet been explored satisfactorily in the education literature: the evaluation of educational research. The particular focus is on the United Kingdom context, but the paper is informed by comparative evidence from six countries (gathered through analysis of policy and administrative documents, literature review, informal discussion, and written requests for information from key persons). It identifies eight recent trends in the evaluation of education research (from performance-based funding and institutionalisation of assessment, to the de-sensitivisation of research assessment) and it explores the benefits and perils of three types of assessment procedures (peer review, bibliometrics, economic metrics) as they operate at a micro, meso and macro level. The paper argues that current evaluations of educational research (particularly those aimed at supporting funding decisions) tend to operate from an instrumental standpoint that largely ignores the epistemic specificity of the various fields, modes or genres of research, the assumptions about knowledge with which they work, and the cultural and social dimensions of research evaluation as a practice.
This study examines Chinese universities' incentive schemes for international publication in the ... more This study examines Chinese universities' incentive schemes for international publication in the humanities and social sciences (HSS). It analysed 172 incentive documents collected at 116 research-intensive universities in China, including monetary bonus schemes and career-related incentives. This analysis is complemented by interviews with six senior administrators at six Chinese universities and four journal editors to explore the policy-making process and impacts of incentivisation. The study found that most universities actively promoted international publications, with variations and similarities in their incentive schemes. In general, more benefits were granted for SSCI (Social Sciences Citation Index) and A&HCI (Arts and Humanities Citation Index) journal publications than domestic publications. We conclude such incentivisation showcased the influence of western standards and global benchmarking in the internationalisation of Chinese HSS, and that the incentive schemes may create a 'Matthew Effect' enabling SSCI and A&HCI journals to flourish, while deepening the divide between these and other journals.
This paper explores recent public debates around research assessment and its future as part of a ... more This paper explores recent public debates around research assessment and its future as part of a dynamic landscape of governance discourses and practices, and organisational, professional and disciplinary cultures. Drawing reflectively on data from RAE 2001, RAE 2008 and REF 2014 (reported elsewhere), the paper highlights how recent debates around research assessment echo longer-term changes in research governance. The following changes, and several critiques of their implications, are discussed: shifts in the principles for governing research and the rise of multipurpose assessment; the spread of performance-based funding and external accountability for research; the use of metrics and indicators in research assessment; the boundary work taking place in defining and classifying units or fields for assessment; the emphasis on research impact as a component of research value; organisational recalibration across the sector; and the specialisation of blended professional practice. These changes are underpinned by persistent tensions around accountability; evaluation; measurement; demarcation; legitimation; agency; and identity in research. Overall, such trends and the discursive shifts that made them possible have challenged established principles of funding and governance and have pushed assessment technologies into a pivot position in the political dynamics of renegotiating the relationships between universities and the state. Jointly, the directions of travel identified in this paper describe a widespread and persistent regime of research governance and policy that has become embedded in institutional and individual practices.
1. The English White Paper, The Importance of Teaching, was discussed at the February 2011 meetin... more 1. The English White Paper, The Importance of Teaching, was discussed at the February 2011 meeting of the Universities‟ Council for the Education of Teachers (UCET Research and Development Committee. It seemed possible that proposed changes in initial teacher ...
BACKGROUND: Funders of medical research the world over are increasingly seeking, in research asse... more BACKGROUND: Funders of medical research the world over are increasingly seeking, in research assessment, to complement traditional output measures of scientific publications with more outcome-based indicators of societal and economic impact. In the United Kingdom, the Higher Education Funding Council for England (HEFCE) developed proposals for the Research Excellence Framework (REF) to allocate public research funding to higher education institutions, inter alia, on the basis of the social and economic impact of their research. In 2010, it conducted a pilot exercise to test these proposals and refine impact indicators and criteria.
METHODS: The impact indicators proposed in the 2010 REF impact pilot exercise are critically reviewed and appraised using insights from the relevant literature and empirical data collected for the University of Oxford's REF pilot submission in clinical medicine. The empirical data were gathered from existing administrative sources and an online administrative survey carried out by the university's Medical Sciences Division among 289 clinical medicine faculty members (48.1% response rate).
RESULTS: The feasibility and scope of measuring research impact in clinical medicine in a given university are assessed. Twenty impact indicators from seven categories proposed by HEFCE are presented; their strengths and limitations are discussed using insights from the relevant biomedical and research policy literature.
CONCLUSIONS: While the 2010 pilot exercise has confirmed that the majority of the proposed indicators have some validity, there are significant challenges in operationalising and measuring these indicators reliably, as well as in comparing evidence of research impact across different cases in a standardised manner. It is suggested that the public funding agencies, medical research charities, universities, and the wider medical research community work together to develop more robust methodologies for capturing and describing impact, including more valid and reliable impact indicators.
Uploads
Papers by Alis Oancea
METHODS: The impact indicators proposed in the 2010 REF impact pilot exercise are critically reviewed and appraised using insights from the relevant literature and empirical data collected for the University of Oxford's REF pilot submission in clinical medicine. The empirical data were gathered from existing administrative sources and an online administrative survey carried out by the university's Medical Sciences Division among 289 clinical medicine faculty members (48.1% response rate).
RESULTS: The feasibility and scope of measuring research impact in clinical medicine in a given university are assessed. Twenty impact indicators from seven categories proposed by HEFCE are presented; their strengths and limitations are discussed using insights from the relevant biomedical and research policy literature.
CONCLUSIONS: While the 2010 pilot exercise has confirmed that the majority of the proposed indicators have some validity, there are significant challenges in operationalising and measuring these indicators reliably, as well as in comparing evidence of research impact across different cases in a standardised manner. It is suggested that the public funding agencies, medical research charities, universities, and the wider medical research community work together to develop more robust methodologies for capturing and describing impact, including more valid and reliable impact indicators.