Scientists have long since become accustomed to explaining the future value of their work. Nowada... more Scientists have long since become accustomed to explaining the future value of their work. Nowadays token statements are no longer sufficient. Societal impact must be embedded in the organisation of research. The call for societal impact is most explicitly expressed in and actively shaped by trans-disciplinary research programmes. We have examined two questions related to compliance in the principal-agent relation between a programme and its projects. The first question concerns the risk of moral hazard: is societal actor involvement a token activity or a substantial component of the research process? The second question relates to possible adverse selection: does societal actor involvement produce the expected benefits and, if so, under which conditions? We surveyed members and project leaders of 178 projects in two transdisciplinary climate research programmes in The Netherlands. There is no reason to suspect large-scale moral hazard. Projects formally labelled as transdisciplinary have characteristics typically associated with transdisciplinarity but academic projects share those characteristics. Neither is there reason to suspect adverse selection. The archetypical properties of transdisciplinary research are associated with the expected societal benefits. An important finding is that there are different types of benefit, each of which requires its own approach. Benefit is achieved through informal involvement and a diversity of outputs, and much less by giving societal actors a prominent role or influence in the research process. Based on our conclusions we recommend customizing the design of climate research programmes and projects towards the needs of the specific societal benefits they aim to generate and reconsidering the emphasis on formal involvement of societal actors in funding procedures.
In dit artikel reageren we op de bijdrage van Jelena Dobbels et al in Thema 1 van 2015. Dobbels e... more In dit artikel reageren we op de bijdrage van Jelena Dobbels et al in Thema 1 van 2015. Dobbels et al beschrijven valorisatie vanuit Belgisch perspectief. Wij vullen dit aan vanuit Nederlands perspectief. Net als in België worden wetenschappers ook in Nederland geacht bij te dragen aan maatschappelijke ontwikkeling en vooruitgang. Anders dan in België is het beleid van de Nederlandse overheid en wetenschapskoepels niet alleen gericht op economische, maar ook op maatschappelijke valorisatie. Desondanks zijn de ervaringen van onderzoekers in beide landen vergelijkbaar. De invulling van valorisatie hangt af van persoonlijke interesse en onderzoekers ervaren gebrek aan steun en erkenning.
Many countries have amended legislation and introduced policies to stimulate universities to tran... more Many countries have amended legislation and introduced policies to stimulate universities to transfer their knowledge to society. The effects of these policies on scientists are relatively unexplored. We employ principal–agent theory to increase our understanding of the relationship between impact policies and scientific practice. Our methodology includes the analysis of policy documents and of data gathered in focus groups. We conclude that there is a gap between policy on the one hand and how scientists perceive it on the other. Policy documents put forward a broad notion of impact, but scientists perceive them as focusing too narrowly on commercial impacts. Scientists are further puzzled by how societal impact is evaluated and organised, and their perceptions frame their behaviour. Our policy recommendations focus on improving the interaction between intermediaries, such as universities and research councils, and scientists so as to include the latter’s perspective in policy-making.
Bij kenniscoproductie zijn wetenschappers en vertegenwoordigers van overheden, NGOs en bedrijven ... more Bij kenniscoproductie zijn wetenschappers en vertegenwoordigers van overheden, NGOs en bedrijven betrokken. Dat maakt de evaluatie van dit soort onderzoek een uitdaging; de betrokken organisaties hebben eigen doelen en eigen monitoringsinstrumenten. In dit hoofdstuk gaan we op basis van onderzoek dat de afgelopen jaren onder andere door het Rathenau Instituut is uitgevoerd na hoe kenniscoproductie onderzoek het beste kan worden geevalueerd. We trekken daaruit de volgende lessen: - gebruik extended peer review om een compleet beeld te krijgen - focus op proces in plaats van product - gebruik nieuwe indicatoren - ga van ex post-evaluatie van wetenschappelijke en maatschappelijke kwaliteit naar integrale evaluatie
The Netherlands, after the UK, has the longest tradition of quality assurance in academic researc... more The Netherlands, after the UK, has the longest tradition of quality assurance in academic research. This edition of Facts & Figures looks at the development of the Dutch quality assurance system and at the results of research evaluation. What has twenty years of quality assurance in academic research achieved? This edition of Facts & Figures considers this question. For the first time since 1993, it provides a review both of the development of the system and of the results of evaluation, providing an insight into how universities and research institutes assure the quality of their research.
Universities are increasingly expected to fulfill a third mission in addition to those of researc... more Universities are increasingly expected to fulfill a third mission in addition to those of research and education. Universities must demonstrate engagement with society through the application and exploitation of knowledge. As societal impact of research is uncertain, long term and always dependent on other factors, we argue here that evaluation should focus on the conditions under which societal impact is generated rather than on the impact itself. Here we focus on a specific set of those conditions: the interactions between academic researchers and societal actors. Instead of speculating about potential impacts of research, we argue that current productive interactions of researchers with societal stakeholders improve the probability that future societal impact will occur. This article supports this idea by examining in detail several, mainly retrospective examples. As productive interactions are field specific, we restrict ourselves to ‘professional adhocracy fields’, especially to information and communication technologies (ICT) research. We address the patterns of productive interactions that result in societal impact within this field and we discuss whether differences are observed in contrast to other fields, such as social sciences and humanities (fragmented adhocracies). We end by discussing the implications that these patterns have for societal impact assessment. Shifting the focus to interactions allows assessment of short-term knowledge transfer and other collaborative efforts with stakeholders that contribute to long-term societal impact.
Strategic research consortia as policy instruments for research coordination have been on the ris... more Strategic research consortia as policy instruments for research coordination have been on the rise for more than a decade. Despite their rising popularity as coordination structures, there has been little comparative analysis of the actual coordination approaches such consortia develop. In order to enhance our understanding of consortia as coordination structures, this paper makes a systematic and in-depth comparison of the coordination approaches of two Dutch consortia. The analysis shows that research consortia coordinate their activities in very different ways. A consortium’s coordination approach turns out to be strongly influenced by its internal characteristics. The observed influence of internal consortium characteristics implies that the eventual coordination approach of consortia will not always match the rationale behind a policy measure to support these consortia. We recommend policy-makers to foster strategic research consortia with a heterogeneous composition that have organised sufficient flexibility for reacting to unforeseen developments.
In this chapter we describe and explain how the relationship between biology and society has cha... more In this chapter we describe and explain how the relationship between biology and society has changed in the Netherlands over the past 30 years, and how this has provided pressures and incentives for multidisciplinarity in two fields. In paleo-ecology we observed a rise in multidisciplinary research collaborations as a response to outside pressures to do more 'relevant' research, particularly in the area of climate change. In toxicology we observed the opposite: multidisciplinarity is used to strengthen the fundamental research on physiological mechanisms of toxicity. We conclude that multidisciplinary research collaborations can serve various ends, ranging from enhancing the practical value of a fundamental field to strengthening the fuindamental basis of an applied field
Globally, the call for impact of science on society is louder than ever. The Netherlands is no ex... more Globally, the call for impact of science on society is louder than ever. The Netherlands is no exception. In 2004, valorisation was introduced as a core element of Dutch science policy, aiming to increase the societal benefits of academic research. In scientific practice, the introduction of valorisation meant scientists got a new task, in addition to teaching and education.
This thesis studies the valorisation policy from a principal-agent perspective. It aims to answer two questions. 1) How has the valorisation policy of Dutch government been translated to academic practice in the Netherlands between 2004 and 2014? And 2) How can societal benefits of academic research be evaluated? Scientists from a multitude of disciplines, as well as societal actors, have been interviewed and surveyed and policy documents have been studied.
The first key result of the study is that the Dutch science system is in a transition. Scientists are motivated to engage with society and do so in many different ways. However, they have a limited understanding of valorisation policies and feel poorly equipped for the task. The second key result is that valorisation should be evaluated as a process. This facilitates learning among scientists and as such allows for improving valorisation practices.
The study ends with policy recommendations, addressing government, intermediaries, such as research council NWO and universities, and the academic community.
Scientists have long since become accustomed to explaining the future value of their work. Nowada... more Scientists have long since become accustomed to explaining the future value of their work. Nowadays token statements are no longer sufficient. Societal impact must be embedded in the organisation of research. The call for societal impact is most explicitly expressed in and actively shaped by trans-disciplinary research programmes. We have examined two questions related to compliance in the principal-agent relation between a programme and its projects. The first question concerns the risk of moral hazard: is societal actor involvement a token activity or a substantial component of the research process? The second question relates to possible adverse selection: does societal actor involvement produce the expected benefits and, if so, under which conditions? We surveyed members and project leaders of 178 projects in two transdisciplinary climate research programmes in The Netherlands. There is no reason to suspect large-scale moral hazard. Projects formally labelled as transdisciplinary have characteristics typically associated with transdisciplinarity but academic projects share those characteristics. Neither is there reason to suspect adverse selection. The archetypical properties of transdisciplinary research are associated with the expected societal benefits. An important finding is that there are different types of benefit, each of which requires its own approach. Benefit is achieved through informal involvement and a diversity of outputs, and much less by giving societal actors a prominent role or influence in the research process. Based on our conclusions we recommend customizing the design of climate research programmes and projects towards the needs of the specific societal benefits they aim to generate and reconsidering the emphasis on formal involvement of societal actors in funding procedures.
In dit artikel reageren we op de bijdrage van Jelena Dobbels et al in Thema 1 van 2015. Dobbels e... more In dit artikel reageren we op de bijdrage van Jelena Dobbels et al in Thema 1 van 2015. Dobbels et al beschrijven valorisatie vanuit Belgisch perspectief. Wij vullen dit aan vanuit Nederlands perspectief. Net als in België worden wetenschappers ook in Nederland geacht bij te dragen aan maatschappelijke ontwikkeling en vooruitgang. Anders dan in België is het beleid van de Nederlandse overheid en wetenschapskoepels niet alleen gericht op economische, maar ook op maatschappelijke valorisatie. Desondanks zijn de ervaringen van onderzoekers in beide landen vergelijkbaar. De invulling van valorisatie hangt af van persoonlijke interesse en onderzoekers ervaren gebrek aan steun en erkenning.
Many countries have amended legislation and introduced policies to stimulate universities to tran... more Many countries have amended legislation and introduced policies to stimulate universities to transfer their knowledge to society. The effects of these policies on scientists are relatively unexplored. We employ principal–agent theory to increase our understanding of the relationship between impact policies and scientific practice. Our methodology includes the analysis of policy documents and of data gathered in focus groups. We conclude that there is a gap between policy on the one hand and how scientists perceive it on the other. Policy documents put forward a broad notion of impact, but scientists perceive them as focusing too narrowly on commercial impacts. Scientists are further puzzled by how societal impact is evaluated and organised, and their perceptions frame their behaviour. Our policy recommendations focus on improving the interaction between intermediaries, such as universities and research councils, and scientists so as to include the latter’s perspective in policy-making.
Bij kenniscoproductie zijn wetenschappers en vertegenwoordigers van overheden, NGOs en bedrijven ... more Bij kenniscoproductie zijn wetenschappers en vertegenwoordigers van overheden, NGOs en bedrijven betrokken. Dat maakt de evaluatie van dit soort onderzoek een uitdaging; de betrokken organisaties hebben eigen doelen en eigen monitoringsinstrumenten. In dit hoofdstuk gaan we op basis van onderzoek dat de afgelopen jaren onder andere door het Rathenau Instituut is uitgevoerd na hoe kenniscoproductie onderzoek het beste kan worden geevalueerd. We trekken daaruit de volgende lessen: - gebruik extended peer review om een compleet beeld te krijgen - focus op proces in plaats van product - gebruik nieuwe indicatoren - ga van ex post-evaluatie van wetenschappelijke en maatschappelijke kwaliteit naar integrale evaluatie
The Netherlands, after the UK, has the longest tradition of quality assurance in academic researc... more The Netherlands, after the UK, has the longest tradition of quality assurance in academic research. This edition of Facts & Figures looks at the development of the Dutch quality assurance system and at the results of research evaluation. What has twenty years of quality assurance in academic research achieved? This edition of Facts & Figures considers this question. For the first time since 1993, it provides a review both of the development of the system and of the results of evaluation, providing an insight into how universities and research institutes assure the quality of their research.
Universities are increasingly expected to fulfill a third mission in addition to those of researc... more Universities are increasingly expected to fulfill a third mission in addition to those of research and education. Universities must demonstrate engagement with society through the application and exploitation of knowledge. As societal impact of research is uncertain, long term and always dependent on other factors, we argue here that evaluation should focus on the conditions under which societal impact is generated rather than on the impact itself. Here we focus on a specific set of those conditions: the interactions between academic researchers and societal actors. Instead of speculating about potential impacts of research, we argue that current productive interactions of researchers with societal stakeholders improve the probability that future societal impact will occur. This article supports this idea by examining in detail several, mainly retrospective examples. As productive interactions are field specific, we restrict ourselves to ‘professional adhocracy fields’, especially to information and communication technologies (ICT) research. We address the patterns of productive interactions that result in societal impact within this field and we discuss whether differences are observed in contrast to other fields, such as social sciences and humanities (fragmented adhocracies). We end by discussing the implications that these patterns have for societal impact assessment. Shifting the focus to interactions allows assessment of short-term knowledge transfer and other collaborative efforts with stakeholders that contribute to long-term societal impact.
Strategic research consortia as policy instruments for research coordination have been on the ris... more Strategic research consortia as policy instruments for research coordination have been on the rise for more than a decade. Despite their rising popularity as coordination structures, there has been little comparative analysis of the actual coordination approaches such consortia develop. In order to enhance our understanding of consortia as coordination structures, this paper makes a systematic and in-depth comparison of the coordination approaches of two Dutch consortia. The analysis shows that research consortia coordinate their activities in very different ways. A consortium’s coordination approach turns out to be strongly influenced by its internal characteristics. The observed influence of internal consortium characteristics implies that the eventual coordination approach of consortia will not always match the rationale behind a policy measure to support these consortia. We recommend policy-makers to foster strategic research consortia with a heterogeneous composition that have organised sufficient flexibility for reacting to unforeseen developments.
In this chapter we describe and explain how the relationship between biology and society has cha... more In this chapter we describe and explain how the relationship between biology and society has changed in the Netherlands over the past 30 years, and how this has provided pressures and incentives for multidisciplinarity in two fields. In paleo-ecology we observed a rise in multidisciplinary research collaborations as a response to outside pressures to do more 'relevant' research, particularly in the area of climate change. In toxicology we observed the opposite: multidisciplinarity is used to strengthen the fundamental research on physiological mechanisms of toxicity. We conclude that multidisciplinary research collaborations can serve various ends, ranging from enhancing the practical value of a fundamental field to strengthening the fuindamental basis of an applied field
Globally, the call for impact of science on society is louder than ever. The Netherlands is no ex... more Globally, the call for impact of science on society is louder than ever. The Netherlands is no exception. In 2004, valorisation was introduced as a core element of Dutch science policy, aiming to increase the societal benefits of academic research. In scientific practice, the introduction of valorisation meant scientists got a new task, in addition to teaching and education.
This thesis studies the valorisation policy from a principal-agent perspective. It aims to answer two questions. 1) How has the valorisation policy of Dutch government been translated to academic practice in the Netherlands between 2004 and 2014? And 2) How can societal benefits of academic research be evaluated? Scientists from a multitude of disciplines, as well as societal actors, have been interviewed and surveyed and policy documents have been studied.
The first key result of the study is that the Dutch science system is in a transition. Scientists are motivated to engage with society and do so in many different ways. However, they have a limited understanding of valorisation policies and feel poorly equipped for the task. The second key result is that valorisation should be evaluated as a process. This facilitates learning among scientists and as such allows for improving valorisation practices.
The study ends with policy recommendations, addressing government, intermediaries, such as research council NWO and universities, and the academic community.
Uploads
Papers by Stefan de Jong
landen vergelijkbaar. De invulling van valorisatie hangt af van persoonlijke interesse en onderzoekers ervaren gebrek aan steun en erkenning.
In dit hoofdstuk gaan we op basis van onderzoek dat de afgelopen jaren onder andere door het Rathenau Instituut is uitgevoerd na hoe kenniscoproductie onderzoek het beste kan worden geevalueerd.
We trekken daaruit de volgende lessen:
- gebruik extended peer review om een compleet beeld te krijgen
- focus op proces in plaats van product
- gebruik nieuwe indicatoren
- ga van ex post-evaluatie van wetenschappelijke en maatschappelijke kwaliteit
naar integrale evaluatie
What has twenty years of quality assurance in academic research achieved? This edition of Facts & Figures considers this question. For the first time since 1993, it provides a review both of the development of the system and of the results of evaluation, providing an insight into how universities and research institutes assure the quality of their research.
Books by Stefan de Jong
valorisation meant scientists got a new task, in addition to teaching and education.
This thesis studies the valorisation policy from a principal-agent perspective. It aims to answer two questions. 1) How has the valorisation policy of Dutch government been translated to
academic practice in the Netherlands between 2004 and 2014? And 2) How can societal benefits of academic research be evaluated? Scientists from a multitude of disciplines, as well as societal
actors, have been interviewed and surveyed and policy documents have been studied.
The first key result of the study is that the Dutch science system is in a transition. Scientists are motivated to engage with society and do so in many different ways. However, they have a limited understanding of valorisation policies and feel poorly equipped for the task. The second key result is that valorisation should be evaluated as a process. This facilitates learning among scientists and as such allows for improving valorisation practices.
The study ends with policy recommendations, addressing government, intermediaries, such as research council NWO and universities, and the academic community.
landen vergelijkbaar. De invulling van valorisatie hangt af van persoonlijke interesse en onderzoekers ervaren gebrek aan steun en erkenning.
In dit hoofdstuk gaan we op basis van onderzoek dat de afgelopen jaren onder andere door het Rathenau Instituut is uitgevoerd na hoe kenniscoproductie onderzoek het beste kan worden geevalueerd.
We trekken daaruit de volgende lessen:
- gebruik extended peer review om een compleet beeld te krijgen
- focus op proces in plaats van product
- gebruik nieuwe indicatoren
- ga van ex post-evaluatie van wetenschappelijke en maatschappelijke kwaliteit
naar integrale evaluatie
What has twenty years of quality assurance in academic research achieved? This edition of Facts & Figures considers this question. For the first time since 1993, it provides a review both of the development of the system and of the results of evaluation, providing an insight into how universities and research institutes assure the quality of their research.
valorisation meant scientists got a new task, in addition to teaching and education.
This thesis studies the valorisation policy from a principal-agent perspective. It aims to answer two questions. 1) How has the valorisation policy of Dutch government been translated to
academic practice in the Netherlands between 2004 and 2014? And 2) How can societal benefits of academic research be evaluated? Scientists from a multitude of disciplines, as well as societal
actors, have been interviewed and surveyed and policy documents have been studied.
The first key result of the study is that the Dutch science system is in a transition. Scientists are motivated to engage with society and do so in many different ways. However, they have a limited understanding of valorisation policies and feel poorly equipped for the task. The second key result is that valorisation should be evaluated as a process. This facilitates learning among scientists and as such allows for improving valorisation practices.
The study ends with policy recommendations, addressing government, intermediaries, such as research council NWO and universities, and the academic community.