Kathryn Oliver
Currently working on a Leverhulme-funded project aiming to collate and analyse covert network data
Supervisors: Frank de Vocht, Martin Everett, Annemarie Money, and John Scott (External)
Supervisors: Frank de Vocht, Martin Everett, Annemarie Money, and John Scott (External)
less
InterestsView All (46)
Uploads
Papers by Kathryn Oliver
Background.
This report describes the findings and methods of a systematic review of research which explores the relationship between obesity and educational attainment. It has been conducted at a time of great concern about levels of obesity in the UK, and the negative physical, psychological and social impacts of obesity. Current research suggests that there may be a relationship between obesity and poor educational attainment. It is likely that obesity and poor school performance are elements of a broader picture of inequalities in health and education, whereby disadvantaged socio-economic groups tend to have poorer health and lower levels of education. However, it is possible that other factors influence obesity and attainment, such as gender, discrimination and poor mental and emotional well-being.
This systematic review was therefore commissioned to address the question: What do we know about the relationship between childhood obesity and educational attainment, from the research literature? In order that our review might be informed by those closest to observing any interaction between obesity and attainment, we sought the perspectives of teachers and young people to identify the causal pathways that seemed most pertinent to them.
However, little is known about what types of evidence policymakers working in local settings prefer to use. This
study aims to evaluate policymakers’ needs and sources of information, at regional and local levels. Methods: An
electronic survey with telephone follow-up was carried out among PH policymakers and evidence producers
(n = 152) working in a large UK city. Respondents were asked which types of evidence they used regularly,
found most useful and what were their main sources of information. Semi-structured interviews (n = 23) added
were analysed quantitatively in addition to the categorical data generated by the survey. Results: Policymakers use
a much greater range of evidence and information than is often indicated in the literature on evidence-based
policy. Local data were by far the most used (n = 95%) and most valued (n = 85%) type of information, followed by
practice guidelines. The main sources of information were Government websites (84%), followed by information
obtained through personal contacts (71%), including PH professionals, council officers and politicians. Academics
were rarely consulted and research evidence was rarely seen as directly relevant. Conclusions: Policymakers use a
wider range of evidence types than previously discussed in the literature. Although local data were most valued by
policymakers, results suggest that these were accessed through personal contacts, rather than specialized organizations.
Systems to provide local high-quality evidence for PH policy should be supported.
Arguments supporting the involvement of users in research have even more weight when involving the public in systematic reviews of research. We aimed to explore the potential for public involvement in systematic reviews of observational and qualitative studies.
Methods
Two consultative workshops were carried out with a group of young people (YP) aged 12–17 years to examine two ongoing reviews about obesity: one about children's views and one on the link between obesity and educational attainment. YP were invited to comment on the credibility of themes, to propose elements of interventions, to suggest links between educational attainment and obesity and to comment on their plausibility.
Results
Researchers had more confidence in review findings, after checking that themes identified as important by YP were emphasised appropriately. Researchers were able to use factors linking obesity and attainment identified as important by YP to identify limitations in the scope of extant research.
Conclusion
Consultative workshops helped researchers draw on the perspectives of YP when interpreting and reflecting upon two systematic reviews. Involving users in judging synthesis credibility and identifying concepts was easier than involving them in interpreting findings. Involvement activities for reviews should be designed with review stage, purpose and group in mind
Initial assessment Routinely collected general practice data in one UK region suggested a CKD prevalence of 4.1%, compared with an estimated national prevalence of 8.5%. Of patients on CKD registers, ∼30% were estimated to have suboptimal management according to Public Health Observatory analyses.
Choice of solution An evidence-based framework for implementation was developed. This informed the design of an improvement collaborative to work with a sample of 30 general practices.
Implementation A two-phase collaborative was implemented between September 2009 and March 2012. Key elements of the intervention included learning events, improvement targets, Plan-Do-Study-Act cycles, benchmarking of audit data, facilitator support and staff time reimbursement.
Evaluation Outcomes were evaluated against two indicators: number of patients with CKD on practice registers; percentage of patients achieving evidence-based blood pressure (BP) targets, as a marker for CKD care. In Phase 1, recorded prevalence of CKD in collaborative practices increased ∼2-fold more than that in comparator local practices; in Phase 2, this increased to 4-fold, indicating improved case identification. Management of BP according to guideline recommendations also improved.
Lessons learned An improvement collaborative with tailored facilitation support appears to promote the uptake of evidence-based guidance on the identification and management of CKD in primary care. A controlled evaluation study is needed to rigorously evaluate the impact of this promising improvement intervention.
The gap between research and practice or policy is often described as a problem. To identify new barriers of and facilitators to the use of evidence by policymakers, and assess the state of research in this area, we updated a systematic review.
Methods
Systematic review. We searched online databases including Medline, Embase, SocSci Abstracts, CDS, DARE, Psychlit, Cochrane Library, NHSEED, HTA, PAIS, IBSS (Search dates: July 2000 - September 2012). Studies were included if they were primary research or systematic reviews about factors affecting the use of evidence in policy. Studies were coded to extract data on methods, topic, focus, results and population.
Results
145 new studies were identified, of which over half were published after 2010. Thirteen systematic reviews were included. Compared with the original review, a much wider range of policy topics was found. Although still primarily in the health field, studies were also drawn from criminal justice, traffic policy, drug policy, and partnership working. The most frequently reported barriers to evidence uptake were poor access to good quality relevant research, and lack of timely research output. The most frequently reported facilitators were collaboration between researchers and policymakers, and improved relationships and skills. There is an increasing amount of research into new models of knowledge transfer, and evaluations of interventions such as knowledge brokerage.
Conclusions
Timely access to good quality and relevant research evidence, collaborations with policymakers and relationship- and skills-building with policymakers are reported to be the most important factors in influencing the use of evidence. Although investigations into the use of evidence have spread beyond the health field and into more countries, the main barriers and facilitators remained the same as in the earlier review. Few studies provide clear definitions of policy, evidence or policymaker. Nor are empirical data about policy processes or implementation of policy widely available. It is therefore difficult to describe the role of evidence and other factors influencing policy. Future research and policy priorities should aim to illuminate these concepts and processes, target the factors identified in this review, and consider new methods of overcoming the barriers described.
Background.
This report describes the findings and methods of a systematic review of research which explores the relationship between obesity and educational attainment. It has been conducted at a time of great concern about levels of obesity in the UK, and the negative physical, psychological and social impacts of obesity. Current research suggests that there may be a relationship between obesity and poor educational attainment. It is likely that obesity and poor school performance are elements of a broader picture of inequalities in health and education, whereby disadvantaged socio-economic groups tend to have poorer health and lower levels of education. However, it is possible that other factors influence obesity and attainment, such as gender, discrimination and poor mental and emotional well-being.
This systematic review was therefore commissioned to address the question: What do we know about the relationship between childhood obesity and educational attainment, from the research literature? In order that our review might be informed by those closest to observing any interaction between obesity and attainment, we sought the perspectives of teachers and young people to identify the causal pathways that seemed most pertinent to them.
However, little is known about what types of evidence policymakers working in local settings prefer to use. This
study aims to evaluate policymakers’ needs and sources of information, at regional and local levels. Methods: An
electronic survey with telephone follow-up was carried out among PH policymakers and evidence producers
(n = 152) working in a large UK city. Respondents were asked which types of evidence they used regularly,
found most useful and what were their main sources of information. Semi-structured interviews (n = 23) added
were analysed quantitatively in addition to the categorical data generated by the survey. Results: Policymakers use
a much greater range of evidence and information than is often indicated in the literature on evidence-based
policy. Local data were by far the most used (n = 95%) and most valued (n = 85%) type of information, followed by
practice guidelines. The main sources of information were Government websites (84%), followed by information
obtained through personal contacts (71%), including PH professionals, council officers and politicians. Academics
were rarely consulted and research evidence was rarely seen as directly relevant. Conclusions: Policymakers use a
wider range of evidence types than previously discussed in the literature. Although local data were most valued by
policymakers, results suggest that these were accessed through personal contacts, rather than specialized organizations.
Systems to provide local high-quality evidence for PH policy should be supported.
Arguments supporting the involvement of users in research have even more weight when involving the public in systematic reviews of research. We aimed to explore the potential for public involvement in systematic reviews of observational and qualitative studies.
Methods
Two consultative workshops were carried out with a group of young people (YP) aged 12–17 years to examine two ongoing reviews about obesity: one about children's views and one on the link between obesity and educational attainment. YP were invited to comment on the credibility of themes, to propose elements of interventions, to suggest links between educational attainment and obesity and to comment on their plausibility.
Results
Researchers had more confidence in review findings, after checking that themes identified as important by YP were emphasised appropriately. Researchers were able to use factors linking obesity and attainment identified as important by YP to identify limitations in the scope of extant research.
Conclusion
Consultative workshops helped researchers draw on the perspectives of YP when interpreting and reflecting upon two systematic reviews. Involving users in judging synthesis credibility and identifying concepts was easier than involving them in interpreting findings. Involvement activities for reviews should be designed with review stage, purpose and group in mind
Initial assessment Routinely collected general practice data in one UK region suggested a CKD prevalence of 4.1%, compared with an estimated national prevalence of 8.5%. Of patients on CKD registers, ∼30% were estimated to have suboptimal management according to Public Health Observatory analyses.
Choice of solution An evidence-based framework for implementation was developed. This informed the design of an improvement collaborative to work with a sample of 30 general practices.
Implementation A two-phase collaborative was implemented between September 2009 and March 2012. Key elements of the intervention included learning events, improvement targets, Plan-Do-Study-Act cycles, benchmarking of audit data, facilitator support and staff time reimbursement.
Evaluation Outcomes were evaluated against two indicators: number of patients with CKD on practice registers; percentage of patients achieving evidence-based blood pressure (BP) targets, as a marker for CKD care. In Phase 1, recorded prevalence of CKD in collaborative practices increased ∼2-fold more than that in comparator local practices; in Phase 2, this increased to 4-fold, indicating improved case identification. Management of BP according to guideline recommendations also improved.
Lessons learned An improvement collaborative with tailored facilitation support appears to promote the uptake of evidence-based guidance on the identification and management of CKD in primary care. A controlled evaluation study is needed to rigorously evaluate the impact of this promising improvement intervention.
The gap between research and practice or policy is often described as a problem. To identify new barriers of and facilitators to the use of evidence by policymakers, and assess the state of research in this area, we updated a systematic review.
Methods
Systematic review. We searched online databases including Medline, Embase, SocSci Abstracts, CDS, DARE, Psychlit, Cochrane Library, NHSEED, HTA, PAIS, IBSS (Search dates: July 2000 - September 2012). Studies were included if they were primary research or systematic reviews about factors affecting the use of evidence in policy. Studies were coded to extract data on methods, topic, focus, results and population.
Results
145 new studies were identified, of which over half were published after 2010. Thirteen systematic reviews were included. Compared with the original review, a much wider range of policy topics was found. Although still primarily in the health field, studies were also drawn from criminal justice, traffic policy, drug policy, and partnership working. The most frequently reported barriers to evidence uptake were poor access to good quality relevant research, and lack of timely research output. The most frequently reported facilitators were collaboration between researchers and policymakers, and improved relationships and skills. There is an increasing amount of research into new models of knowledge transfer, and evaluations of interventions such as knowledge brokerage.
Conclusions
Timely access to good quality and relevant research evidence, collaborations with policymakers and relationship- and skills-building with policymakers are reported to be the most important factors in influencing the use of evidence. Although investigations into the use of evidence have spread beyond the health field and into more countries, the main barriers and facilitators remained the same as in the earlier review. Few studies provide clear definitions of policy, evidence or policymaker. Nor are empirical data about policy processes or implementation of policy widely available. It is therefore difficult to describe the role of evidence and other factors influencing policy. Future research and policy priorities should aim to illuminate these concepts and processes, target the factors identified in this review, and consider new methods of overcoming the barriers described.
Much attention has been paid to the role of individuals in evidence utilisation. Mainly this focuses on (a) how to increase uptake of research, (b) validating models of knowledge transfer (c) advocating for the use of knowledge brokers. Recently published research corroborates the importance of interpersonal relationships which these studies assert. The literature provides several frameworks to describe the activities associated with knowledge brokerage. However, little is known about the activities of actors who are influential in policy making. This study aims to describe the activities of powerful and influential actors and compare them with standard KB frameworks.
Methods
24 semi-structured interviews with national, regional and municipal policy makers were carried out. The data were transcribed and analysed using framework analysis to categorise and explain the strategies used by actors to influence policy.
Findings
Although influential actors did use knowledge brokerage as a strategy to influence policy, this was just one item in their toolkit of methods to influence policy. A range of knowledge brokerage roles were described by actors, which overlap with knowledge brokerage frameworks presented in the literature. However, these activities were just a subset of a larger spectrum of activities which fell into four categories; controlling policy organisations, controlling policy content, controlling policy makers, and using network structures
Discussion
While it is important to help policy makers to find and use evidence, the literature largely ignores the fact that there are already individuals out there playing knowledge brokerage roles. SNA can identify these individuals. These actors use a range of activities to influence policy. By categorising these activities, a deeper understanding of the policy process can be developed, and practical targeted messages developed by academics.
Statements
1. Introducing knowledge brokers from academia into the policy arena ignores the day-to-day activities of policy actors
2. To influence policy, researchers should consider adopting a more pro-active approach to make and exploit links with influential actors
OBJECTIVES:
To determine whether randomised controlled trials (RCTs) lead to the same effect size and variance as non-randomised studies (NRSs) of similar policy interventions, and whether these findings can be explained by other factors associated with the interventions or their evaluation.
DATA SOURCES:
Two RCTs were resampled to compare randomised and non-randomised arms. Comparable field trials were identified from a series of health promotion systematic reviews and a systematic review of transition for youths with disabilities. Previous methodological studies were sought from 14 electronic bibliographic databases (Applied Social Sciences Index and Abstracts, Australian Education Index, British Education Index, CareData, Dissertation Abstracts, EconLIT, Educational Resources Information Centre, International Bibliography of the Sociological Sciences, ISI Proceedings: Social Sciences and Humanities, PAIS International, PsycINFO, SIGLE, Social Science Citation Index, Sociological Abstracts) in June and July 2004. These were supplemented by citation searching for key authors, contacting review authors and searching key internet sites.
REVIEW METHODS:
Analyses of previous resampling studies, replication studies, comparable field studies and meta-epidemiology investigated the relationship between randomisation and effect size of policy interventions. New resampling studies and new analyses of comparable field studies and meta-epidemiology were strengthened by testing pre-specified associations supported by carefully argued hypotheses.
RESULTS:
Resampling studies offer no evidence that the absence of randomisation directly influences the effect size of policy interventions in a systematic way. Prior methodological reviews and meta-analyses of existing reviews comparing effects from RCTs and non-randomised controlled trials (nRCTs) suggested that effect sizes from RCTs and nRCTs may indeed differ in some circumstances and that these differences may well be associated with factors confounded with design. No consistent explanations were found for randomisation being associated with changes in effect sizes of policy interventions in field trials.
CONCLUSIONS:
From the resampling studies we have no evidence that the absence of randomisation directly influences the effect size of policy interventions in a systematic way. At the level of individual studies, non-randomised trials may lead to different effect sizes, but this is unpredictable. Many of the examples reviewed and the new analyses in the current study reveal that randomisation is indeed associated with changes in effect sizes of policy interventions in field trials. Despite extensive analysis, we have identified no consistent explanations for these differences. Researchers mounting new evaluations need to avoid, wherever possible, allocation bias. New policy evaluations should adopt randomised designs wherever possible.
Methods: Semi-structured interviews (n = 23) were carried out with a purposive sample of individuals involved in public health policy (gathering or analysing information, developing or implementing policy) in a large city in the UK. Individuals were recruited as part of a network study (Oliver 2013), and were approached to achieve representativeness in terms of professional group and centrality. Data were collected between January – September 2010, and were analysed using a framework approach.
Principle findings: KB activities were used by all professional groups, although most frequently by NHS managers and council officers. However, these formed a small part of a wide spectrum of activities which were used to influence policy, which ranged from design and management of key organisations, to influencing individuals directly. Examination of the discourse around these activities showed that influential individuals were able to exercise autonomy throughout the policy process, and used all strategies at appropriate times.
Conclusion: Policy is influenced by exceptional individuals who use a range of strategies to initiate, manage and control policy, including KB strategies. All professional groups were shown to use these strategies, with the most influential individuals being health and council managers. This paper presents a novel approach to understanding the relative importance of KB activities in influencing policy. Our analysis sheds new light on the policy process and suggests researchers are unlikely to influence policy using traditional KB approaches.
Although there are differences, research outputs can be seen as any other product entering a competitive market place. This means comparisons can be drawn between their producers (corporations and researchers respectively) and the techniques used to market or disseminate the product in question. Clearly, the motivations of corporate business and research groups are different. Corporations may be driven by profit, innovation, sustainability, longevity, and workforce pressures. To many, this will sound wildly divorced from the reality of conducting research – and will question the apparent difference in motivation. Businesses are driven to maximise profit, and researchers (in applied health sciences) are motivated to produce high-quality research which can inform practice and policy. Few researchers – perhaps increasingly in the new impact-driven world – would disagree that they would like their research to have more effect on the real world. . Even though researchers are not financially motivated, however, most would be keen to maximise the impact of our research; quite apart from the fact that we are often funded by public money, and are thus also motivated by public duty to increase our impact, are increasingly measured by our impact.
Businesses use a range of methods to increase their knowledge of the market place and help predict the likely reach of their product. For example, when Apple brings out a new iPhone model, they know they want to achieve a certain percentage market share, and so they develop a strategy to aggressively market the new product. Do researchers go through the same process of estimating the market share, developing a target percentage or a strategy to reach it?
In fact, many research outputs can be considered in the same way as any other product. For example, health services researchers may be interested in developing a needs assessment tool for a patient group. Potential and actual users of the tool could be identified (Clinical Commissioning groups, charities, community services), and contacted to assess usage, and thus calculate the market share. On the back of this information, decisions about targeted dissemination could be made, the development of new products could be informed, and so forth.
The concepts of ‘marketing’ and market share have been used in health services research before, although primarily to understand the population share of hospitals (Garnich 1987) or to market particular services (Banaszak-Holl 1996). This paper discusses some features of the business world which, if applied to production and dissemination of research outputs may enhance the ability of researchers to influence policy and practice.
Methods: This paper analyses this literature in detail to try and explain the persistence of barriers and facilitators. We critically describe the literature in terms of its theoretical underpinnings, definitions of ‘evidence’, methods, and underlying assumptions of the research. We aim to illuminate the EBP discourse by comparing the EBP literature with approaches from other fields.
Findings: Much of the research in this area is theoretically naive, focusing primarily on the uptake of research evidence and privileging academics’ research priorities over policy makers’ priorities. Little empirical data analysing the processes or impact of evidence use in policy is available to inform researchers or decision-makers. EBP research often assumes that policy makers do not use evidence; and that more evidence – meaning research evidence - use would benefit policy makers and populations.
Conclusions: We find that most EBP research is based on unsupported assumptions. The agenda of ‘getting evidence into policy’ has sidelined the empirical description and analysis of how research and policy actually interact in vivo. Rather than asking how research evidence can be made more influential, academics should aim to understand what influences and constitutes policy, and produce broader, more critically and theoretically informed studies of decision-making. We question the main assumptions made by EBP researchers, explore the implications, and propose new directions for EBP research.
Keywords: Evidence-based policy; critical analysis, knowledge utilization, science and technology studies
Research suggests that policy makers often use personal contacts to find information and advice. However, little is known who or what are the main sources of information for public health policy makers. This study aims to describe policy makers’ sources of information and compare categorical with network data.
Methods
A questionnaire survey of public health policy makers across Greater Manchester was carried out (response rate 48%). All policy actors involved in public health policy (finding, analyzing or producing information, producing or implementing policy) in Greater Manchester were included in the sampling frame. Respondents were provided with a list of sources of information and asked which they used; if they used any other sources of information, and finally to name specific individuals who acted as sources of information. Data were analysed using frequencies and network analysis.
Results
The most frequently chosen sources of information from the categorical data were NICE, government websites and Directors of Public Health. However, the network data showed that the main sources of information in the network were actually mid-level managers in the NHS, who had no direct expertise in public health. Academics and researchers did not feature in the network.
Conclusion
Both survey and network analyses provide useful insights into how policy makers access information. Network analysis offers practical and theoretical contributions to the EBP (spell out) debate. Identifying individuals who act as key users and producers of evidence allows academics to target actors likely to use and disseminate their work.
Methods
An electronic survey with telephone follow-up was carried out amongst public health policy actors and evidence producers working in a large UK city. Respondents were asked which types of evidence they used regularly, found most useful, and what were their main sources of information. Semi-structured interviews added qualitative depth to the categorical responses generated by the survey.
Results
Policy actors use a much greater range of evidence and information than is often indicated in the literature on evidence-based policy. Local data were by far the most used and most valued type of information, followed by practice guidelines. The main source of information were Government websites, followed by information obtained through personal contacts, including public health professionals, council officers and politicians. Academics were rarely consulted and research evidence was rarely seen as directly relevant.
Conclusions
Policy actors use a wider range of evidence types than previously discussed in the literature. Although local data were most valued by policy actors, results suggest that these were accessed through personal contacts, rather than specialised organisations. Systems to provide local high-quality evidence for public health policy should be supported.
"
In this study, social network analysis is used to study the relationships between perceived power, influence, and sources of evidence. Network and qualitative data about were gathered from a policy community in a large urban area in the UK. Powerful and influential people were initially identified using centrality scores. Their characteristics were compared with themes from the qualitative interview, including a narrative about insiders and outsiders in the policy process. Hubs and Authorities analyses were used to test these idea in the network data. Authorities tended to correspond to formal hierarchy, but the Hubs were mainly mid-level managers in the NHS and local councils.
These findings were then compared with a evidence network to see (1) where powerful and influential people got their information and (2) whether there was a relationship between being a source of information and being influential. The results showed that evidence-producers are not influential or powerful in this policy network. Powerful and influential actors are in some cases also considered source of evidence; more frequently these individuals were described in terms of roles (e.g. gatekeeping, brokerage) or personal characteristics. An additional analysis of roles (Gould-Fernandez roles) was therefore carried out to explore what roles were played by different actors in the networks with respect to evidence-brokerage and the policy process.
The results show that occupying advantageous network positions is dependent on both structural circumstance and individual characteristics; in other words, having the right person in the right place. To be effective, these individuals are able to identify important relationships, create and maintain them. Their ability to influence depends on having personal skills explicitly manage network structure. Although not a new conclusion for the business or organisational theory fields, it is new in the field of evidence-based policy.
Methods: Network data were gathered from a public health policy community in a large urban area in the UK (n = 152, response rate 80%), collecting relational data on perceived power, influence, and sources of evidence about public health policy. Hubs and Authorities analyses were used to identify powerful and influential actors, to test whether powerful and influential actors were also sources of information; and betweenness and Gould-Fernandez brokerage were used to explore the importance of structural position in policy networks. These data were analysed in conjunction with qualitative data from semi-structured interviews (n = 24) carried out with a purposive subsample of network actors. Characteristics of powerful and influential actors, the use of evidence in the policy process, and roles and strategies used to influence policy were analysed using a framework approach, and combined with network data.
Results: The most influential actors were mid-level managers in the NHS and local authorities, and to a lesser extent, public health professionals. These actors occupied advantageous positions within the networks, and used strategies (ranging from providing policy content, to finding evidence, to presenting policy options to decision-makers) to influence the policy process. Powerful actors were also sources of information for one another, but providing information did not predict power. Experts, academics and professionals in public health were represented in the networks, but were usually more peripheral and played fewer roles in the policy process. This study presents empirical evidence to support the suggestion that recognition of network structure assists individuals to be influential, and proposes a framework to categorise their activities.
Conclusions: In order to influence policy, actors need good relationships with other influential actors, and the skills to exploit these relationships. The relational approach is useful for both identifying powerful and influential people (potential evidence-users) and for exploring how evidence and information reaches them. Identifying powerful and influential actors and describing their strategies for influencing policy provides a new focus for researchers in evidence-based policy, and for those wishing to influence policy. For academics and researchers, this study demonstrates the importance of directly creating ties with decision-makers.""