The Conceptualization of RRI: An Iterative Approach

Chapter 5 The Conceptualization of RRI: An Iterative Approach Pim Klaassen, Frank Kupper, Sara Vermeulen, Michelle Rijnen, Eugen Popa, and Jacqueline Broerse Abstract To stimulate research and innovation (R&I), to contribute to the solution of societal challenges and to align R&I with societal values, the European Commission has launched the governance framework of Responsible Research and Innovation (RRI). RRI igures in many high-level EU policies as a means to promote smart growth, and a growing community of R&I practitioners from both the public and private sectors appears committed to it. Although debates on what RRI precisely entails have not reached closure yet, RRI provides an interesting avenue to explore ways of making R&I more societally germane. While recognizing the usefulness of keeping critical relection on RRI’s meaning alive, we suggest that to make the step from theorizing to implementation, RRI could beneit from a clearer conceptualization. This chapter presents the iterative trajectory in conceptualizing RRI followed as part of RRI Tools, one of a number of EC-funded research projects and support acts aimed at leshing out what RRI can and should be, and the conceptualization of RRI that this led to. It suggests that RRI is best captured if in R&I governance attention is paid to the ive p’s of Purpose, Products, Processes, Preconditions and People, and that further elaborations on the meaning of RRI should happen in dialogue with attempts at practicing RRI. Frank Kupper, Sara Vermeulen and Michelle Rijnen contributed equally to this work. P. Klaassen (*) • F. Kupper • E. Popa • J. Broerse Athena Institute, Vrije Universiteit Amsterdam, Amsterdam, North Holland, The Netherlands e-mail: [email protected] S. Vermeulen Athena Institute, Vrije Universiteit Amsterdam, Amsterdam, North Holland, The Netherlands Radboud Universiteit Nijmegen, Nijmegen, The Netherlands M. Rijnen Athena Institute, Vrije Universiteit Amsterdam, Amsterdam, North Holland, The Netherlands Dutch Cancer Society, Amsterdam, The Netherlands © Springer International Publishing AG 2017 L. Asveld et al. (eds.), Responsible Innovation 3, DOI 10.1007/978-3-319-64834-7_5 69 70 5.1 P. Klaassen et al. Introduction We are faced with global crises in the spheres of climate, inance and food and with trends including ageing populations, environmental degradation and rising disparities in income and wealth (World Economic Forum 2016). All of these pose a challenge to the resilience of the organizational and governance arrangements of our societies and economies. Efforts are being undertaken to deal with these crises and work is being done in response to today’s risks to our planet and its inhabitants. This is for instance illustrated by the recent UN agreement on sustainable development goals signed in September 2015 and the Paris Agreement under the United Nations Framework Convention on Climate Change that went into effect on 4 November 2016. Arguably, all such challenges can only be tackled through concerted action by actors at societal levels from business to policy and from civil society to research and innovation (R&I). In this chapter, we will focus on how R&I can contribute to solving today’s complex problems and respond to today’s risks. One of the issues this brings us to, is that although R&I’s role with regard to, for instance, the ight against infectious diseases, malnutrition or climate change, might be crystal clear to some, it is also debated. Thus, R&I might for instance contribute to better and more affordable healthcare, to more eficient and different resource use, to the transition to a bio-based economy and so on, but R&I also sparks controversies—for instance over UMTS, carbon capture and storage, use of genetically modiied organisms for fuels or foodstuffs, or geoengineering. In this light, it is of no small importance that the European Commission has identiied seven Societal Challenges to be dealt with in its research funding programme Horizon 2020.1 Moreover, in the EU we have seen that during the last 6 years both at the EC-level as well as through actions by research funding organizations and academic researchers, efforts have been put into developing and implementing a governance framework aimed at directing R&I efforts to more responsible ways of working: Responsible Research and Innovation (RRI). RRI has been proposed as a unifying framework that aspires to integrate ethical relection, stakeholder engagement and responsive change into research and innovation (R&I) practices (Stilgoe et al. 2013). In parallel with this narrative that presents RRI in relation to complexities of the world we inhabit today, RRI’s emergence can also be explained with reference to (not-independent) developments in philosophical and sociological studies of R&I, R&I policy, Technology Assessment in all its well-known versions, and so on (Owen et al. 2012). Overall, what the past two decades in these ields of study show, is an increasing focus on all possible forms of interaction between R&I and society. The articulation of this theme has been recorded and discussed, inter alia, by (Nowotny et al. 2001; Etzkowitz and Leydesdorff 2000; Regeer and Bunders 2009; Callon 1 These seven can all be found here: https://ec.europa.eu/programmes/horizon2020/en/h2020-section/societal-challenges 5 The Conceptualization of RRI: An Iterative Approach 71 et al. 2009). What these views have in common, despite all sorts of differences in emphasis, is the recognition that R&I processes are not assessed solely internally (by scientists themselves) and disciplinarily (by using domain-speciic criteria) but also externally (by society) and inter-, multi- and trans-disciplinarily. In addition, they acknowledge that the purpose for which knowledge is produced goes beyond the mere quenching of the scientiic thirst for knowledge, so as to include solving real-life problems. Although RRI gains popularity, closure has not yet been reached with regard to the concept’s meaning. For instance, Oftedal notes that “the more speciic content of RRI is largely left open” (Oftedal 2014, p. 1) while Zwart et al. describe RRI as a buzzword whose conceptualization is “open-ended” (Zwart et al. 2014, p. 3) and the source of “confusion”. Wickson and Carew also subscribe to the idea that “without concrete elaboration and conceptual development, the interpretive lexibility of RRI will be so broad as to render the concept meaningless” (2014, p. 256). And even scholars whose names almost immediately pop up when RRI is discussed, have expressed concerns regarding the vagueness surrounding the very idea of responsibility in research and innovation. Thus, Owen, Macnaghten and Stilgoe note that the notion suffers from “ambiguity as to motivation, theoretical conceptualisation and translation into practice” (Owen et al. 2012, p. 751). This brings us before a quandary. On the one hand, the lexibility in the notion of RRI is expedient since it provides a conceptual space for assimilating and comparing diverse approaches that have been developed in the past before the notion of RRI entered the scene. Those who had already been working on speciic aspects of responsibility in research and innovation (e.g., making science and innovation responsive to societal needs) will ind in RRI a useful mainstay and an opportunity for relection. If RRI is to work as a guiding concept (De Jong et al. 2016), RRI must allow for at least some interpretation and thus variation. On the other hand, the lexibility of the notion can also be detrimental to its application. We must not lose sight of the fact that the scholarship on the notion of RRI is also an instance of research and innovation. Thus, being true to form, we should appraise it based on the same standards that we use to observe others in their research and innovation practices. In short, if RRI is to be more than a sweet-sounding buzzword, it should eventually be crystalized into a policy instrument that achieves what it claims to achieve. In this chapter, we want to present our way out of this quandary. We will show that, despite what common sense might suggest, an increase in analytical clarity does not necessarily imply a decrease in interpretive lexibility. Quite the contrary, if an abstract concept such as RRI is ever to become a sustainable force in shaping R&I practices, then we should not shy away from rejecting the old distinction between ideals (dreams) and practices (reality). Moreover, we will not only present what we found at the end of our road, but also that road itself. With respect to this we can say that we must seek conceptualization methods that make the most of both our ability to dream the ideal-thus-unspeciic and our ability to observe and learn from the concrete-thus-speciic. On the whole, the route we took led us to a better understanding of RRI, an understanding we are now ready to lesh out and relect upon. What we have found 72 P. Klaassen et al. is not an unyielding answer to the question ‘What is RRI?’. Rather, we have reached what we see as a sensible approximation of this solution, one that is capable of reconciling the need for abundant dreaming and concrete governance actions—and perhaps, even one that inspires both such dreaming and such actions. 5.2 Laying the Path While Walking It: Outline of Our Iterative Exploration of What RRI Means The ideas presented in this chapter are largely developed in the context of EC-funded FP7 support action RRI Tools. The project’s aim was to foster RRI through the development of a toolkit tailored to the use in implementation of RRI by users from different R&I stakeholder groups and through training and advocacy activities. A multidisciplinary consortium consisting of 26 partners operating in 30 European countries collaborated on this.2 One of our roles in this project was the conceptualization of RRI that would be central to the different project tasks. What we share here, however, is not the academic version of an oficial project deliverable, but rather an essay that provides insight in the process of informal iterative concept development that we have engaged in throughout the project, and into the preliminary conclusions regarding RRI that based on that process we have managed to draw. Some such conclusions can in a different, abbreviated form be found in deliverables that are available on the RRI Tools website (Klaassen et al. 2014). One reason for that is while formally the conceptualization of RRI was a task that belonged to Work package 1 and that was inished in 2015, our process of constantly re-imaging RRI continued with all the different (other) tasks we engaged in in the context of this project. What we present is in fact something like a rational reconstruction of our iterative conceptualization process throughout the project in light of what these have led us to conclude as regards the concept of RRI. Six different project activities contributed to our understanding of the RRI concept: (1) literature review, (2) expert consultation, (3) stakeholder workshops, (4) identiication and classiication of promising practices, (5) speciication and reinement, and (6) case-studies. Each of these contributed in a speciic way to the resulting image of RRI. Vice versa, each of these six processes were informed by a certain (‘raw’) image of RRI, the image that we had at that speciic moment when the concept was still in the making. This two-way relationship between the model and the six steps in gathering data and information is represented in Fig. 5.5. Although these steps will now be discussed in the indicated order, it is important to note that most of the six overlapped in time and were thus informed by one another. In this 2 We feel indebted to all our colleagues in the RRI Tools project and would like to express our gratitude to them. Amongst other things, the partners included research funding foundations, universities, science centres and museums. For a complete list, see here: http://www.rri-tools.eu/ who-we-are 5 The Conceptualization of RRI: An Iterative Approach 73 way, we managed to compare, early on during the conceptualization process, the various conceptions of RRI that arose from each source. This lead to what we see as a very fruitful blend between what RRI is to various stakeholders and what RRI should be according to the same stakeholders. The literature research included both academic and policy literature. The ground covered mainly concerned literature explicitly addressing RRI, but we also built on the plethora of conceptual, theoretical and empirical resources that fuel RRI—from constructive Technology Assessment to public engagement, from Gender Studies to research ethics, from STS to science communication. Early 2014 a very irst working deinition of RRI was developed, for use in the irst stages of the RRI Tools project. According to this deinition, RRI is a dynamic, iterative process by which all stakeholders involved in the R&I practice become mutually responsive and share responsibility regarding both the outcomes and process requirements. During the expert consultations, the irst ideas on the delineation and operationalization of RRI were elaborately discussed with experts from a wide range of ields pertinent to RRI. To wit, we discussed our preliminary conceptualization of RRI with the Advisory Board members of the RRI Tools project as well as with other experts within the RRI Tools group. The Advisory Board members were selected based on their expertise on the different RRI “keys” as identiied by the EC: Ethics, Gender, Equality, Governance, Open Access, Public Engagement and Science Education; each key being represented by two experts.3 The feedback we received found its way into the project’s irst deliverable, a Policy Brief on RRI (Klaassen et al. 2014). Gradually, a highly specialized community of experts has arisen, as a result of the RRI Tool project’s aim and effort to build an RRI community of practice. Scholars from ields like Science and Technology Studies, philosophy of science, science communication, Technology Assessment, research ethics and research policy studies have interacted with one another, emerging as experts on RRI. However, these experts agreed that RRI should not be an idea that can only be grasped by a small intellectual elite. All actors that have an interest in research and innovation should translate this central idea within their own domains and this translation should lead straightforwardly to implementation. With this in mind, stakeholder workshops were organized during the fall and winter of 2014. A total of 27 stakeholder consultation workshops were organized with stakeholders representing the following ive domains: research, policy, business/industry, civil society and education. During the workshops, stakeholders were acquainted with the concept of RRI, invited to discuss RRI and to help the RRI consortium of RRI Tools to identify the opportunities, obstacles and needs they experience as regards putting RRI into practice. Workshops were held in 22 different countries, and 411 participants took part in them. The workshops provided us with valuable insights regarding the opportunities, obstacles and needs experienced by various groups whose work can be improved by a new research and innovation framework. Since these groups are driven by different social, economic and moral interests, the consultation workshops were also employed as an opportunity for the 3 The experts of the Advisory Board are listed here: http://www.rri-tools.eu/en_GB/who-we-are 74 P. Klaassen et al. Fig. 5.1 Selected practices – 31 in total 4 2 Tool 6 Project Program Organization 19 stakeholders to hear each other’s viewpoints on research and innovation. The discussions, which often revealed surprising differences and equally surprising similarities in worldview, were an eye-opening moment during the conceptualization process. The workshops constituted a point of departure for developing a catalogue of good practices, in addition supplying a much-needed input regarding opportunities, obstacles and needs (Kupper et al. 2015b). All 411 participants in the stakeholder consultation workshops were invited to share one or more examples of research innovation practices that instantiate RRI to a greater or lesser extent. These cases could be research and innovation projects, but also funding programs and organisations related to research and innovation (see Fig. 5.1). The assumption underlying the request to workshop participants to bring examples of RRI practices, is that concepts – as sets – can best be described by combining an intentional deinition in which the criteria for set-membership are spelled out in general terms (viz., the working deinition) with an extensional deinition in which members of the set are enumerated (viz., the catalogue of RRI practices). Having collected these practices, a irst selection of so-called ‘promising practices’ was made, leaving those out that did not meet any of the process requirements and/ or outcomes of the RRI working deinition. Hereafter, a database of additional promising practices was developed by making use of an online questionnaire. Together with the irst selection, the body of good practices was now studied and assessed. From all these suggestions 31 practices ended up in an RRI catalogue of good practices. Some descriptive statistics concerning these practices can be found in Figs. 5.1, 5.2, 5.3 and 5.4. As these igures indicate, the practices included in the catalogue all dealt with one or more of the so-called policy agendas of Public Engagement, Science Education, Governance, Ethics, Open Access or Gender (Fig. 5.2), were all rather inclusive in terms of the amount and types of stakeholders they managed to assemble together (Fig. 5.3), and all contributed to one or more of the EU-deined Grand Challenges (Fig. 5.4). Through an examination of the good practices and by revisiting the literature reviewed in the irst step, we formulated a set of criteria and indicators for RRI (Kupper et al. 2015a). In various stages of this development, we applied the formu- 5 75 The Conceptualization of RRI: An Iterative Approach Public Engagement 87 Science Education 55 Governance 48 Addressed Ethics 48 Open Access Not addressed 39 Gender 19 0% 20% 40% 60% 80% 100% Fig. 5.2 policy agendas addressed in percentages (out of 31 practices) Research 97 CSOs 81 Policy-makers 81 Involved Not involved Business&Industry 74 Education 71 0% 20% 40% 60% 80% 100% Fig. 5.3 stakeholders involved in practices in percentages lated criteria to the selected promising practices mentioned, changing the inal formulation so as to encompass as many of these practices as possible. At the same time, we kept an eye on the systematicity of the resulting set of criteria and indicators. We organized, merged and split some of these indicators in order to obtain an analytical instrument that is at the same time expedient (minimal overlap) and thorough (maximal applicability). In this way, i.e., by going back and forth between theoretical formulation and empirical application, we have sought to maintain the lexibility of the concept of RRI while increasing its clarity. 76 P. Klaassen et al. 1. Health, demographic change and wellbeing 2. Food security, sustainable agriculture and forestry, marine and maritime and inland water research, and the Bioeconomy 3. Secure, clean and efficient energy 4. Smart, green and integrated transport 5. Climate action, environment, resource efficiency and raw materials 6. Europe in a changing world - inclusive, innovative and reflective societies 7. Secure societies - protecting freedom and security of Europe and its citizens 0 2 4 6 8 10 12 14 16 Fig. 5.4 Grand challenges addressed by the 31 selected practices While the previous steps were successful in (abstractly) clarifying the nature of RRI, little in-depth suggestions were provided regarding the factual implementation of RRI. This is why we continued by selecting eight showcases and analysing them thoroughly. The analysis was based on semi-structured interviews with experts on the particular cases (mostly project or programme managers). This resulted in a series of eight elaborate narratives regarding responsibility in research and innovation, each delivering important lessons to be learned about the contemporary constraints and opportunities for applying RRI.4 Table 5.1 briely describes all eight and presents one distinctively illustrative lesson learned from each showcase.5 From the spring of 2016 onwards, these showcases have been used in training events on RRI throughout Europe, along with an abundance of other materials, facilitated by RRI Tools consortium members and afiliates. During such training sessions, again, feedback on the proper conceptualization of RRI was collected (Fig. 5.5). 5.3 Five Components of RRI These iterative processes have brought us in a better position to tell a more reined story of RRI. Although in what follows we will tell this story with the conviction that it is the right story to tell, the one that most naturally follows from our iterative approach, we do not wish to suggest that the version here presented is the deinitive one. Presenting work in progress might perhaps be at odds with current academic conventions. For conventionally, publishing and defending one’s conclusions is something that takes place after the discovery has taken place. We cannot but reject such linearity. In our case, the process of discovering RRI through a continuous and multifarious interaction with various stakeholders was the process of building a case 4 5 These can be found here: www.rri-tools.eu/training/resources See https://www.epsrc.ac.uk/research/framework/area/ Framework for responsible innovation by the engineering and physical sciences research council (UK) Stakeholder engagement in Fishery benchmarking research at The Portuguese Sea and Atmosphere Institute (Portugal) Brief description By funding research and innovation in consortia of partners that come from different societal organizations, cross-disciplinary, cross-sectorial and challenge-oriented research and innovation are promoted. The focus is on (i) Future healthcare, (ii) Sustainable attractive cities, (iii) Competitive industries, and (iv) Information society. The programme features a three-stage process, allowing large numbers of projects access to funding in the stage of idea development and testing, with smaller numbers of projects moving to subsequent stages in the research and innovation trajectory. The AREA “code of conduct” of anticipate, relect and engage is at the heart of the EPSRC’s framework for responsible innovation. It requires of researchers that they not only have good ideas, but also consider what potential consequences their research might have. This Framework aspires to convey that the two are not separate matters, but rather are part of the same package deal. Industrial, policy, research and societal stakeholders, all with a role in marine conservation, interacted in all stages of this research project designed to address a critical aspect in the sustainable use of marine resources—from agenda setting to follow-up. Lessons learned Successfully implementing a challenge-driven research programme requires strong leadership, the courage to change ingrained structures and working methods, commitment of agency staff and an openness to organizational learning through processes of trial and error. To implement RRI in (academic) research, rules, regulations or speciic grant conditions might not be the most pertinent facilitators. Rather, for the research community to embrace RRI, a framework such as this, with a proven track-record in helping researchers deal with societally challenging issues such as geo-engineering, appears more promising. The active engagement of all types of stakeholders contributes to the pertinence of the research, helps its products become accessible and facilitates implementing actions. However, different types of stakeholders — policy makers, researchers, representatives of industries and of Civil Society Organizations — all have their own speciic roles in facilitating and building co-creation partnerships, and sometimes dificult conlicts can emerge. The Conceptualization of RRI: An Iterative Approach Showcase name (country) Vinnova’s funding programme Challenge Driven Innovation (Sweden) 5 Table 5.1 RRI showcases other iterative steps can of course complement the six represented above. It is not our intention to suggest that these six categories are in some sense suficient or carved out in stone—they are simply the key steps we took on our journey. Keeping in mind the preliminary character of any result reached through this type of iterative conceptual modelling, it is both useful and personally rewarding to pause the modelling process and take a look at the results. In the next section, we will give a brief description of what we have so far learned about RRI (continued) 77 78 Table 5.1 (continued) Showcase name (country) Hao2 (UK) Knowledge for climate (The Netherlands) Novo Nordisk’s blueprint for change (Denmark) Social innovation factory (Belgium) Lessons learned Embracing gender and disability issues as integral part of one’s business activities can lead to opportunities and growth. Openness, diversity and inclusion can be drivers of success rather than obstacles, as they can help companies become responsive and adapt to changing needs. Boundary workers with the right knowledge and skills and suficient time to promote mutual trust and project continuity are vital to successfully engage in co-creative research that involves researchers, policy makers and industry professionals. By taking decisions in ways that are inancially, socially as well as environmentally responsible, the private sector can be a valuable partner in solving societal issues. Doing so requires that investments are made towards long term partnerships that cross sectoral borders. Building networks requires a skill-set of its own, and is an important requirement towards realizing creative social innovations. Implementing RRI in STEAM education enriches students’ perspective on science and innovation and contributes to their empowerment with respect to inding solutions for societal challenges. P. Klaassen et al. Xplore health (Spain) Brief description “Social company” Hao2 develops and sells 3D virtual environments, with the speciic aim not only to make money, but also to increase opportunities for people with autism and other complex needs. Its own workforce consists of some 80% of people with disabilities like autism. To transform the Netherlands’ vulnerability to climate change into opportunities, this programme aimed to increase knowledge about climate adaptation and improve the Dutch export position in climate and delta technology. It did so through co-creative projects, in which research, solutions and results resulted from dialogues between practicing professionals, policy-makers and scientists. Under the name of Blueprint for Change, pharmaceutical company Novo Nordisk developed a series of business cases aimed at identifying drivers of shared value creation, the measurement of societal and company beneits, and the sharing of information with stakeholders. Collaborations with research and local and national societal partners have been undertaken, in order for projects to serve both societal, environmental and economic success. A networking organization that searches for possible partners who can help strengthen concepts for social innovation, and that promotes, guides and supports businesses and their stakeholders in doing so. An educational programme aspiring to bridge the gap between research and secondary Science, Technology, Engineering and Mathematics (STEAM) education, with an innovative educational approach that includes acquainting students with decision-making on science and innovation and incorporating insight on real-life challenges therein, as well as ethical, legal and social issues. 5 The Conceptualization of RRI: An Iterative Approach 79 Fig. 5.5 The six activities involved in Iterative concept development for a certain version of RRI. Vice versa, the process of concretizing all those hundreds of hours of data analysis into a model we defend as ‘correct’ is as much defending a stance as it is discovering it. The uncanny feeling of deciding to follow a certain ideal (RRI) while looking for that ideal can best be compared with the uncanny feeling one typically has while looking at a mise en abîme (which in The Netherlands we know as ‘the Droste effect’). The uncanniness of it stems from the fact that an idea is employed in a discussion in which that very idea is at issue. In what follows we want to distinguish between ive components of the concept of RRI. We suspect that this ive-fold structure is typical of concepts representing ideals we pursuit in other settings, ideals such as ‘justice’ and ‘reasonableness’, yet for the present purposes we will assume it to be an expedient way of crystalizing the idea of RRI. The ive components are: Purpose, Product, Process, Preconditions and People. We refer to these informally as ‘the 5P structure’. Each of these ive components represents a speciic vantage point for understanding RRI story. Each is thus essential for obtaining a full-ledged image RRI but also for distinguishing the kind of research that is further needed for giving this image more depth and perspective. We will discuss these ive components in the order given above, as this order represents what we have found to be a natural way of asking questions about RRI. The irst question that comes to mind is: “What is the purpose of changing 80 P. Klaassen et al. current R&I environments – in any direction, not just towards an ideal of shared responsibility?” Having established a certain purpose, one can turn to questions regarding the more concrete outcomes that together would realize the designated purpose. The second question is thus: “What kind of products need to be obtained in order to eventually realize the designated purpose?” We assume there is always more than one way to obtain these products, so the third question concerns the manner in which the needed products are obtained: “Through what processes will these products be obtained?” Answering this third question will bring researchers in a better position to specify the kind of institutional setting in which these processes are to take place. To draw a parallel: deciding what music (process) you want to play on a certain instrument, say a guitar, will bring you in a better position to specify the needed characteristics of that instrument – whether you need an electric guitar or an acoustic one, a jazz guitar or a classical one. Our fourth question will thus concern the conditions under which the desired processes are to be created: “What institutional preconditions are necessary for hosting the development of the desired processes?” One might perhaps stop the questioning process at this fourth component, given that the entire setting is speciied, top-down from an abstract description of purpose to the concrete settings in which this purpose is to be pursued. Yet the more we interacted with colleagues on the theme of RRI, the more we acknowledged the importance of the individuals’ psychological predispositions and competences. The ifth and inal question is thus: “What kind of individuals function well and eficiently in the designated institutional preconditions?” It holds for all stakeholder groups that fostering RRI from the perspective of that group is a very speciic mission and that this mission requires a speciic set of competences. The burgeoning ield of RRI can be seen as the systematic attempt to ind an answer to these ive questions at the present time all these ive questions have been addressed in some form or another. However, some have inevitably received more attention than others. In what follows we will offer a brief overview of these ive components in the way they result from our iterative conceptual modelling. 5.3.1 Purpose The European Commission has identiied seven societal challenges with which the European (and possibly international) society is nowadays confronted. These challenges, also known as the “grand challenges” are broad, long-term purposes that have been set through a simultaneous look at the past (European Environment Agency 2002, 2013) and at the future (Boden et al. 2010). The seven grand challenges range from health and wellbeing to sustainable energy and secure societies.6 These seven challenges demand a contribution of research and innovation. At the same time, however, research and innovation themselves are contested in the pub6 For more details regarding each challenge, see https://ec.europa.eu/programmes/horizon2020/en/ h2020-section/societal-challenges 5 The Conceptualization of RRI: An Iterative Approach 81 lic sphere. Issues that are raised for example relate to a lack of sensitivity to societal needs and concerns, the distribution of (new and unforeseen) risks and beneits, and emerging ethical controversies. These ethical and societal aspects have long been described in the technology assessment and ethics literature (Rip et al. 1995; Schot and Rip 1997; Schomberg 2007), and were expressed in the consultation workshops in this project too. RRI aspires to deal with both issues at the same time. One of the major shifts in the RRI framework therefore is a primary focus on the question of purpose: what is the contribution that research and innovation can make? Rather than the effort to ‘do things right’, i.e. carefully investigate in order to mitigate potential negative impacts alongside the development of science and technology, RRI aspires to ‘do the right things’. In the words of Owen et al. (2012), RRI seeks to move beyond what we don’t want R&I to do towards what we do want R&I to do. To establish this shift, and create a productive environment to ask the question of purpose, RRI aspires to democratically open up research and innovation to processes of inclusive deliberation involving a variety of actors, tightly coupled to action and policy-making aimed to steer research and innovation towards desirable and acceptable ends. The different dimensions of these processes are discussed in 3.3. 5.3.2 Products The grand challenges formulated by the European Commission constitute longterm purposes for research and innovation. Realizing such purposes will not be the result of any speciic research and innovation process. Even large-scale research and innovation projects cannot, in one stroke, solve such complex issues as the sustainability of our economic processes and the security of our society. Furthermore, it would be highly unrealistic to hope that stakeholders involved in research and innovation would reshape their worldview overnight and reorganizing their professions around these seven grand challenges. It is thus necessary to distinguish between the purposes that make up the raison d’être of RRI and the short-term products that bring us closer to achieving these purposes. Focusing therefore on the short-term adjustments, we have found that, in both literature and stakeholders’ views, there is a natural inclination to make a distinction between two kinds of products resulting from research and innovation. On the one hand, there are products that constitute (proposed) solutions to research and innovation questions. We refer to these as ‘R&I products’. On the other hand, there are products that, while not directly solving any research and innovation problem, create the proper social and cultural environment in which the research and innovation can take place. We refer to these as ‘learning products’. Following the deinition suggested by Von Schomberg (2011), we started our conceptualization process from the assumption that responsible R&I products are ethically acceptable, sustainable and societally desirable. One of the main questions here is: when and how are processes and products ethically acceptable? Part of the 82 P. Klaassen et al. answer can be derived from accounts such as the Treaty on European Union (art. 2) that lists the values (supposed to be) shared in European societies like respect for human dignity, freedom, and equality. Other examples of shared values are wellbeing privacy, autonomy and security (Van den Hoven 2013). However, as we live in a pluralistic society, the interpretation of these moral values may differ between different cultural regions in Europe, but also between different people and groups. We argue that deining ethical acceptability in light of RRI implies an exploration of presumably common values and principles (to understand their situated meaning) and ethical assessments that go beyond protecting the rights, interests and desires of moral subjects (in line with Keulartz et al. 2004). Which values and norms contribute to a speciic case of responsible research and innovation should be discovered through a process of relective inquiry and deliberation between the stakeholders involved. With respect to sustainability, approximately the same argument can be made. Sustainable development is explained as meeting the needs of present generations without jeopardizing the ability of generations to come to meet their own needs (The Council of the European Union, 2006). In speciic research and innovation contexts, however, the contribution to sustainability has to be a matter of inquiry and deliberation amongst the actors involved. With respect to societal desirability, an important observation is that science and society are continuously evolving together, subject to the same evolutionary trends. Boundaries are increasingly transgressed and new collaborative modes of knowledge production emerge (Gibbons et al. 1994; Nowotny et al. 2001). Solutions are found in opening up science through continuous meaningful deliberation with societal actors (Broerse et al. 2009). By incorporating such activities in the R&I process, science as a whole is thought to become more responsive to real-felt social needs, concerns, ambitions and interests (Haywood and Besley 2014). If public concerns and needs are understood, the likelihood of R&I processes and new technologies being successful increases, i.e. innovations and the design thereof will be consistent with needs of society. It is by now evident that RRI involves a shift in thinking from product to process. An important characteristic of this process is mutual learning of the actors involved. It is therefore good to also distinguish the learning products of RRI processes. Learning products contribute to RRI because they create a kind of purposeful change in which responsibility is more easily, more often and more naturally achieved. Responsible research and innovation processes are fore example meant to lead to a wide range of empowered, responsible and relexive stakeholders (researchers, policymakers, NGOs, educators, businesses etc.). It follows that research and innovation should not only lead to a certain form of specialized knowledge, whether incorporated in a technological product or expressed explicitly in written works, it should also lead to a closer relationship between science and society. One important aspect of that relationship is engagement of the general public, not only to develop a robust understanding of scientiic work but also understanding socio-scientiic issues and to become involved in deliberation and decision-making processes. Although such learning products were seen by many stakeholders as ‘by-products’ of science, these products’ importance in fostering responsibility has been widely 5 The Conceptualization of RRI: An Iterative Approach 83 recognized. Further, as some stakeholders have noticed during the consultation workshops, undertaking to deliver both R&I products and learning products might change the actors’ propensities and interests, leading them towards research questions and puzzles that are conducive to contributing to the solution of, amongst other things, the seven grand challenges. 5.3.3 Processes The irst two components provide a reference point for designing the processes through which the aforementioned long-term aims (purposes) and short-term aims (products) are to be achieved. We now want to focus on the path towards these aims. In doing so we distinguish RRI processes as the ones satisfying the following criteria (or ‘process requirements’): (1) Diversity and Inclusion, (2) Anticipation and Relection, (3) Openness and Transparency and (4) Responsiveness and Adaptive Change. Diversity and inclusion refers to the early involvement of a wide range of stakeholders and publics in the deliberation and decision-making episodes that occur within research and innovation processes. This is accomplished through the timely and on-going involvement of a wide range of stakeholders and publics in deliberation and decision-making processes (Owen et al. 2012).7 In different scholarly traditions, such as public engagement (Irwin et al. 2012; Wilsdon et al. 2005) and technology assessment (Palm and Hansson 2006), a participatory-deliberative turn has been argued for because of both normative democratic, instrumental and substantial reasons (Abelson et al. 2003; Wilsdon and Willis 2004). A central issue in these and other studies concerns the right timing for engaging stakeholders. It is notoriously dificult to say with precision what ‘the right timing’ is. Indeed, the dilemma coined by Collingridge (1980) points precisely to the tension between the advantages brought by acting early (the ability to steer the research and innovation process in the desired direction) and the equally important advantages of acting late (knowledge regarding opportunities and limitations of the chosen direction). Still, to strive towards socially desirable (ethically acceptable, sustainable, and marketable) outcomes and to prevent misjudgements regarding each other’s interests, it is vital to have stakeholders articulate their standpoints relatively early in the research and innovation process (Schot and Rip 1997). Subsequently, discussions about ideas and values should be carried out continuously as a groups values and interests may change during the R&I process (Abma and Broerse 2010). In addition to the question of timing, the question arises what it means, in practice, to engage stakeholders within the research and innovation process. A genuine 7 Diversity is understood here in relation with demographic variables such as age, gender and education level as well as cultural variables such as values, interests, religion and worldview (Von Schomberg 2011; Wilsdon and Willis 2004). 84 P. Klaassen et al. dialogue fosters mutual learning processes, in which actors in the dialogue listen to each other, learn about and understand each other’s perspectives, and add new experiences to their repertoire (Stilgoe et al. 2013). Active participation of actors in dialogue, the willingness to share power, being respectful and open to others, and the ability to change one’s own perspective, are all important conditions for constructing a genuine dialogue (Abma and Broerse 2010; Abma and Widdershoven 2006; Chilvers 2012). How these conditions are facilitated ultimately depends on the situation at hand and the relevant actors involved. The wishes and needs of actors vary between practices and need to be taken into account not only concerning the topic at hand, but also in constructing the dialogue itself. Anticipation and relection refers to understanding how the dynamics of R&I shape the future; envisioning the impacts of dominant and alternative R&I futures; relecting on (alternative) problem deinitions, preferred solutions and underlying values, assumption and beliefs. Research and innovation are unequivocally futureoriented activities, with the power of shaping and transforming our future immensely (Borup et al. 2006; Owen et al. 2012; Grinbaum and Groves 2013). This requires anticipation: looking forward in time by imagining the variety of possible impacts of research and innovation practices and relecting on our values and roles in these practices (Schomberg 2011; Owen et al. 2012; Stilgoe et al. 2013). Anticipation can be explained as ‘describing and analysing those intended and potentially unintended impacts that might arise, be these economic, social, environmental or otherwise’, which is not the same as predicting the future (Owen et al. 2012, p. 38). In the past many anticipatory methodologies for science and technology have been developed, such as scenario development (Fisher et al. 2008), vision assessment (Grin and Grunwald 2000), ethical technology assessment (Swierstra 1997), constructive technology assessment (Schot and Rip 1997), and anticipatory governance approaches (Guston and Sarewitz 2002). These methods not only support actors in articulating their expectations, but provide means to explore alternative outcomes and implications that would otherwise be forgotten and help avoid reinforcing certain visions and making them into preordained roadmaps or trajectories (Owen et al. 2012). For anticipation to make sense we should be aware of how present dynamics and values inluence the progression of science and innovation. This means that we should not only anticipate uncertain products of science and think about plausible, intended and unintended consequences, but that we need to relect on underlying purposes, motivations, and actor roles as well (Owen et al. 2012). Acknowledging that irresponsibility in science and innovation is a manifestation of the innovation ecosystem, implies that not only relection on value systems of individual actors or institutions should take place, but that these actors and institutions also help build the collective relexive capacity within the practice of science and innovation. A collective and institutional relexive capacity lies at the heart of any learning process, and for research and innovation to progress – both in process as in outcomes – learning is a prerequisite. Relexivity, or rather relexive learning, requires both ‘insight into the assumptions which tacitly shape our own understandings and interactions’ by which the value of other sources of knowledge and perspectives will increase (Chilvers 2012). 5 The Conceptualization of RRI: An Iterative Approach 85 Room for these relective processes should be built into RRI practices to accomplish learning at different levels: irst-, second- and third-order learning. The description of different levels of learning is found in the work of different scholars across management science, learning science and philosophy, with amongst the most inluential the authors Argyris and Schon (1974). First-order learning refers to learning on the level of problem deinition, possible desired solutions and routines. Convincingly argued that in case of new and complex issues, second-order learning is required, i.e. learning at the level of values and assumptions of actors involved, which means holding a mirror up to one’s own activities, commitments and assumptions, being aware of the limits of knowledge and being mindful that a particular framing of an issue may not be universally held. We speak of third-order learning when a practice of research and innovation starts to transform itself and the way it is organized, connecting the process requirement of relexivity to the dimension of change. Openness and transparency refers to the honest and clear representation of research and innovation processes in society. By this we refer not only to correct and equal access to the products of research and innovation but also to a certain willingness to being open for and listen to input of people other than those directly involved in the research and innovation process. This willingness is a condition for responsiveness and adaptive change and should lead to a mutual understanding and trust. Transparency implies being open and clear about decision-making processes, for instance on issues such as who is included when, what is done with inputs (materials) and results in research and innovation processes (Abma and Broerse 2010; Rowe and Frewer 2004). By communicating decisions made in science and innovation policy, these processes become legitimate and both institutions and individuals can be held accountable. Moreover, open discussions about roles and responsibilities of stakeholders are indispensable, because through the evolvement of the concept of RRI new responsibilities emerge or responsibilities change and shift (Owen et al. 2012). Such discussions create awareness of roles and responsibilities and create clarity about ownership, which will ultimately lead to increased agency. Open access to research information is argued to advance science, as it will promote and accelerate the constructive generation of new knowledge and prevent unnecessary duplication of research. Open access not only improves the quality of scientiic work, but also beneit industry and government. For the wider community, it is argued that open access can beneit the ‘informed citizen’ or ‘informed consumer’, thereby improving knowledge and use of services (Houghton and Sheehan 2006; European Environment Agency 2013). Being open does not necessarily mean that raw data should be published and data sets become available without being edited. Openness should be meaningful; it needs to be understandable and usable for potential stakeholders and publics involved (Chilvers 2012). In practice, this might imply that the amount and level of openness depends on the context, situation and topic of the speciic research or innovation practices. Responsiveness and adaptive change refers to the development of a capacity to change existing routines of thought and behaviour, as well as overarching organizational structures and systems in response to changing circumstances, emerging 86 P. Klaassen et al. knowledge and value perspectives, views and concerns. This fourth cluster of process requirements is vital to RRI insofar as the capacity for change ultimately determines whether the effects of the previously described process requirements can manifest themselves. RRI requires that the direction people, organizations and practices take changes in response to (possibly changing) circumstances, values, ideas and needs of both stakeholders and the public to give true meaning to the requirements of inclusion and diversity. Second, openness and transparency are valuable from a democratic point of view, but become more signiicant through this fourth cluster of process requirements. It requires practices to respond to emerging knowledge, even if it is generated elsewhere, so a collective learning process can be build and R&I can be brought to a higher level. Something similar applies to anticipation and relection. One can anticipate possible futures and relect on one’s role and actions in R&I, but without responding to changing understandings or newly emerging insights, R&I outcomes in the form of learning or desirable futures will most probably not arise. Our systems of science and innovation should thus be open to and enable transformative change by way of responsiveness. Several approaches have already been developed for increasing responsiveness in R&I processes. These include constructive technology assessment (Rip et al. 1995), real time technology assessment (Guston and Sarewitz 2002), midstream modulation (Fisher et al. 2006) and anticipatory governance (Barben et al. 2008). Responsiveness should however not be limited to a capacity for change at the level of individual researchers and or project groups, as actions of individuals are often steered by the rigidity of the systems of which they are part (Cavallo 2000). Responsiveness of R&I processes should extend beyond the responsiveness of individual researchers, and institutionally embed the capacity to adapt to changing circumstances and newly emerging knowledge in such a way that inclusive deliberation is tightly coupled to policymaking, action and change (Owen et al. 2012). 5.3.4 Preconditions Now that RRI has been analysed in terms of purposes, products and processes, one might ask, what role do the key dimensions to RRI as identiied by the European Commission play: Gender, Ethics, Open Access, Public Engagement and Science Education? In our conceptualisation, the interaction between processes, products and purposes is what makes an R&I practice RRI. However, the keys as formulated by the EC give us something like a normative baseline, a way of stating preconditions that have to be met on a systemic level, an organizational level and a project level in order for R&I to be able to take the shape of RRI. To elaborate on this, we can say that for R&I to become truly RRI it is requisite that it takes place in the right environment. For this, governance repertoires need to be installed on all distinguished levels so the proper preconditions for making R&I responsible are created—and here is our fourth P. Focusing on the core processes distinctive of R&I projects, these can be said to be responsible if they entail open 5 The Conceptualization of RRI: An Iterative Approach 87 and transparent cycles of inclusion, anticipation, relection and responsiveness that lead to the variety of outcomes and impacts pictured above: engaged publics and stakeholder learning, responsible institutions, ethically acceptable, socially desirable and sustainable R&I outcomes, targeting the Grand Challenges. The variety of aforementioned agendas, on this view, form a subset of a number of conducive preconditions for such cycles to take place—preconditions that, in true RRI, are themselves open to change in response to the variety of types of outcomes RRI aims at. We picture RRI to blossom optimally in organizational and systemic environments that are governed with an eye to the variety of preconditions that are conducive to RRI, ranging from the promotion of research integrity to banning exclusionary practices in both human resource management as well as research agenda setting. This means, for instance, that for research projects to become responsible, involved research institutes should have policies in place or develop them along the way of research projects taking off concerning everything from gender equality and gender in research, communicating and disseminating research results, engaging stakeholders in agenda-setting and decision-making, research integrity, open access, Intellectual Property issues, and risks and safety. On a systemic level, such preconditions include for instance incentives for academic researchers that do not exclusively promote publishing in peer-reviewed journals, but at least as much steer towards contributing to the solution of complex societal issues. For commercial R&D this would for instance require that existing guidelines and regulations for Corporate Social Responsibility (CSR) explicate what CSR means for the design and execution of R&D trajectories. The main reason for introducing this multi-layered conceptualization of RRI, in which for instance issues relating to diversity and ethical relection emerge both as aspects of responsible R&I cycles as well as in the form of conducive conditions, is that these conditions are not suficient for R&I to be conceived of as responsible R&I, even if they might be necessary for putting RRI into practice. This can be illustrated with reference to ethics, for instance. Thus, for research in the health and life sciences, for example, it is vital to have directives in place concerning the use of laboratory animals—the three Rs of Replacement, Reduction and Reinement come to mind (Festing and Wilkinson 2007). Important as this may be, this in itself does not take one a long way on the inclusive, anticipatory, relective and responsive path of RRI. Rather, the variety of governance arrangements hinted at here “must [collectively] aim for [the effective transformation of] present day practices of R&I towards ‘responsibilisation’, i.e. a process by which the involved actors internalise the issues of concern” (Kuhlman et al. 2016, p. 10). 5.3.5 People More as a rule than as exception, putting RRI into practice will imply changing both what one does and how one does it. Put in the terminology of organizational management, RRI entices research organizations to amend their missions and visions 88 P. Klaassen et al. such that research is no longer a goal in itself but rather a means to accomplish independently identiiable goals best articulated through reference to societal needs and values. And this in turn requires that how R&I trajectories are shaped changes— along the lines sketched above. As studies of change management (Worren et al. 2016) and sustainability transitions (Voß et al. 2009) have convincingly shown, the types of changes required by such soft-governance approach as RRI is—relying on dispersed actors taking responsibility rather than on a framework of rules and regulations directing actions—never come cheap. They take time and require cultural, attitudinal and behavioural changes by many on multiple levels—from governmental or nongovernmental funding agencies to academic researcher institutions, innovative businesses and industries and civil society actors such as CSOs and citizens. To group together this plethora of changes, we introduce our ifth and inal P: The P for People, as those who travel through and connect all the different levels at which changes are requisite. People, moreover, who best pull of the transition that RRI aspires to contribute to, if they have an open mind and are responsive to change—as described under P number three. And people who, to achieve this, in many cases have to get attuned to new operational logistics, given that for instance including anticipation, relection and responsiveness in work practices requires not only additional training that allows them to develop new knowledge and skills distinctive of all those process dimensions involved in practicing RRI—which in turn depends on preconditions being met such as time and a commitment by management. To briely illustrate the latter, we can refer to the multitude of instances in which during our stakeholder consultation workshops we heard people say that soft skills requisite to successful engagement activities, time for undertaking these, and commitment from managerial layers to change (research) processes to become more inclusive were often lacking, both in research organizations, businesses, policy institutions and CSOs. Arguably, then, the People we refer to are the obligatory point of passage (Callon 1984) that simultaneously cannot be avoided when trying to give meaning to RRI and to implement it and that remains almost invisible as target of action in itself, as so much of our attention is easily drawn to leshing out any of the other conventionally referred to elements of RRI. Thus, we direct attention explicitly and speciically to People in a similar spirit as that in which, in the context of discussions revolving around the emerging technology of synthetic biology, human practices has become a term of reference (Rabinow and Bennett 2007). For any stakeholder in research and innovation to thoroughly grasp what it takes to make research and innovation more responsible, requires not only that they interact with people from diverse backgrounds and with different (societal) roles to play, but also that they ind ways to truly learn from and about each other, their work and their commitments. 5 The Conceptualization of RRI: An Iterative Approach 5.4 89 Looking Forward As part of the RRI Tools project we developed a conceptualization of RRI that indicates how the processes of R&I should anticipate on and interact with its foreseen products, enabling the people involved to strive for alignment of the purposes of R&I with the values and needs of society. In the meantime, RRI has steadily continued inding its way into the science policy discourse and attempts are made to implement it in practice. Nevertheless, various ambiguities and differences in interpretation can still be found in the ways experts and stakeholders make sense of RRI principles, actions and results. Is this problematic? Although we recognize the risk of RRI becoming an empty buzzword, we doubt that only more theoretical work will necessarily lead to the desired changes in R&I practices. The meaning and implications of making RRI work should emerge from the interactions between various actors involved and organized around particular issues in speciic contexts. In general, we can say that it is important to involve a relevant variety of stakeholders from start to inish in R&I trajectories, but what that means in terms of which stakeholders are engaged and what role they play ultimately depends on the context of application, the timeframe and the perspectives of the actors involved. For instance, in the context of commercial R&D, where issues involving intellectual property rights are at play, the engagement of stakeholders is likely to take different shapes than in the context of applied medical science, which again will be different from basic, curiosity driven science. In our contribution to the collective attempt at iguring out what RRI can be, we have aimed for a middle road between leaving the criteria that distinguish RRI from R&I open to the context of application and making them speciic and clear. Moreover, with our elaboration on RRI’s ive Ps we assume to carry a message that for a diversity of R&I stakeholders speaks to their motivations and interests and relates to their level of policy inluence. And while pulling off a balancing act of presenting a conceptualization of RRI that could arguably be described as partly normative, partly descriptive, partly a critical analysis and partly an instance of public relations, we have also aimed at presenting a narrative that, in different ways, resonates with various audiences. Thus, we trust that the Purposes of RRI are suficiently tightly embraced by a suficiently large number of R&I stakeholders from both commercial and public research institutes as well as R&I policy makers for RRI to really catch on; we assume that the deliverance of true RRI Products will not only contribute to reaching those Purposes, but accordingly will help strengthen RRI’s reputation among researchers as valuable R&I enterprise, and work as a binding force that helps connect researchers’ interests to those of policy makers, civil society organizations, citizens and society at large; and while recognizing that hurdles are on the way to realizing them, we see empirical evidence accumulating that suggests that the process requirements outlined here can be developed into productive guidelines to cocreate RRI practices; furthermore, we urge R&I stakeholders to recognize that meeting the Preconditions for RRI requires a concerted effort on various levels of 90 P. Klaassen et al. R&I governance, and that although this does not come cheap, it will be worth its while; and we cheer for all the People who have so far contributed to realizing RRI, be it either from a policy perspective, the perspective of R&I practitioners, or that of R&I- or RRI-policy researchers, and we invite the latter to further investigate this important aspect of implementing RRI and the policy makers to acknowledge it, and treat it accordingly. More than anything, however, we stress the importance of continuing the conceptual analysis mainly in connection with practical experiments in RRI. RRI is about a transformation of the research and innovation system. This involves new ways of thinking, doing and organizing research and innovation. Following the seminal work of Argyris and Schon (1974), we believe that researchers, innovators and their organisations learn from experience, gradually adjusting their assumptions and trying out new behaviour. This applies to their learning of RRI as much as it applies to anything. Offering more basic theory will not help them much in acquiring new repertoires for action. How to open up R&I processes to the ideas and concerns of a wider range of involved actors, how to respond adaptively to conversations, controversies, challenges and opportunities that arise, how to anticipate technological futures and relect on their underlying values and our implicit or explicit concerns: if it is to contribute to the embedment and institutionalization of RRI in various contexts of research and innovation, this should all be acquired through experimenting and relecting in practice. In line with Wickson and Carew (2014), we encourage researchers, innovators, funders, societal stakeholders and others to engage in analytic-deliberative processes to experiment with existing RRI frameworks like the one put forward here, but also develop their own evaluative criteria and standards to bring about the changes in their practice that they desire. If these experiences will be shared and used to build new experiments, RRI may indeed become the collective experiment in democracy that it can be. References Abelson, Julia, Pierre-Gerlier Forest, John Eyles, Patricia Smith, Elisabeth Martin, and FrancoisPierre Gauvin. 2003. Deliberations about deliberative methods: Issues in the design and evaluation of public participation processes. Social Science and Medicine 57 (2): 239–251. Abma, Tineke, and Jacqueline Broerse. 2010. Patient participation as dialogue: setting research agendas. Health Expectations 13 (2): 160–173. Abma, Tineke, and Guy Widdershoven. 2006. Moral deliberation in psychiatric nursing practice. Nursing Ethics 13 (5): 546–557. Argyris, Chris, and Donald Schon. 1974. Theory in practice: Increasing professional effectiveness. Oxford: Jossey-Bass. Barben, Daniel, Erik Fisher, Cynthia Selin, and David Guston. 2008. Anticipatory governance of nanotechnology: Foresight, engagement, and integration. In The handbook of science and technology studies, ed. Edward Hackett, Olga Amsterdamska, Michael Lynch, and Judy Wajcman, 979–1000. Cambridge, MA: The MIT Press. Boden, Mark, Christiano Cagnin, Vicente Caribias, Totti Könnölla, and Karel Haegemann. 2010. Facing the future: time for the EU to meet global challenges. Available at: http://ftp.jrc.es/ EURdoc/JRC55981.pdf. 5 The Conceptualization of RRI: An Iterative Approach 91 Borup, Mads, Nik Brown, Kornelia Konrad, and Harro van Lente. 2006. The sociology of expectations in science and technology. Technology Analysis & Strategic Management 18 (3–4): 285–298. Broerse, Jacqueline E.W., Tjard de Cock Buning, Anneloes Roelofsen, and Joske F.G. Bunders. 2009. Evaluating interactive policy making on biotechnology: The case of the Dutch ministry of health, welfare and sport. Bulletin of Science, Technology & Society 29 (6): 447–463. Callon, M. 1984. Some elements of a sociology of translation: domestication of the scallops and the ishermen of St Brieuc Bay. The Sociological Review 32 (S1): 196–233. Callon, Michel, Pierre Lascoumes, and Yannick Barthe. 2009. Acting in an uncertain world: An essay on technical democracy. Cambridge, MA: The MIT. Cavallo, D. 2000. Emergent design and learning environments: Building on indigenous knowledge. IBM Systems Journal 39(3.4): 768–781. Chilvers, Jason. 2012. Relexive engagement? Actors, learning, and relexivity in public dialogue on science and technology. Science Communication 35 (3): 283–310. Collingridge, David. 1980. The social control of technology. London: Francis Pinter. Etzkowitz, Henry, and Loet Leydesdorff. 2000. The dynamics of innovation: from National Systems and “Mode 2” to a Triple Helix of university–industry–government relations. Research Policy 29 (2): 109–123. European Environment Agency. 2002. Late lessons from early warnings: The precautionary principle 1896–2000. www.eea.europa.eu/publications/environmental_issue_report_2001_22. Accessed 26 Jan 2017. ———. 2013. Late lessons from early warnings: Science, precaution, innovation. www.eea. europa.eu/publications/late-lessons-2. Accessed 26 Jan 2017. Festing, Simon, and Robin Wilkinson. 2007. The ethics of animal research. EMBO Reports 8 (6): 517–610. Fisher, Erik, Roop Mahajan, and Carl Mitcham. 2006. Midstream modulation of technology: Governance from within. Bulletin of Science, Technology & Society 26 (6): 485–496. Fisher, Erik, Cynthia Selin, and Jameson Wetmore, eds. 2008. Presenting futures: The yearbook of nanotechnology in society. Dordrecht: Springer Science. Gibbons, Michael, Camille Limoges, Helga Nowotny, Simon Schwartzman, Peter Scott, and Martin Tow. 1994. The new production of knowledge: The dynamics of science and research in contemporary societies. London: SAGE. Grin, John, and Armin Grunwald, eds. 2000. Vision assessment: shaping technology in 21st century society: Towards a repertoire for technology assesment. Berlin: Springer Verlag. Grinbaum, Alexei, and Christopher Groves. 2013. What Is “Responsible” about responsible innovation? Understanding the ethical issues. In Responsible Innovation, ed. Richard Owen and John Bessant, 119–142. Chichester: Wiley. Guston, David, and Daniel Sarewitz. 2002. Real-time technology assessment. Technology in Society 24 (1–2): 93–109. Haywood, Benjamin K., and John C. Besley. 2014. Education, outreach, and inclusive engagement: Towards integrated indicators of successful program outcomes in participatory science. Public Understanding of Science 23(1): 92–106. Houghton, John, and Peter, Sheehan. 2006. The economic impact of enhanced access to research indings. vuir.vu.edu.au/472/. Accessed 26 Jan 2017. Irwin, Alan, Torben Jensen, and Kevin Jones. 2012. The good, the bad and the perfect: Criticizing engagement practice. Social Studies of Science 43 (1): 118–135. Jong, De, Frank Kupper Marije, Marlous Arentshorst, and Jacqueline Broerse. 2016. Responsible reporting: Neuroimaging news in the age of responsible research and innovation. Science and Engineering Ethics 22 (4): 1107–1130. Klaassen, Pim, Frank Kupper, Michelle Rijnen, Sara Vermeulen, and Jacqueline Broerse. 2014. D1.1: Policy brief. RRI tools project. Amsterdam: Athena Institute, VU University Amsterdam. Kuhlman, Stefan, Ralf Lindner, Sally Randles, Bjørn Bested, Guido Gorgoni, Erich Griessler, Allison Loconto, and Niels Mejlgaard. 2016. Navigating towards shared responsibility. doc. utwente.nl/102432/1/RES_AGorA_ebook.pdf. Accessed 26 Jan 2017. 92 P. Klaassen et al. ———. 2015a. D1.3: Report on the quality criteria of Good Practice Standards in RRI. RRI tools project. Amsterdam: Athena Institute, VU University Amsterdam. Kupper, Frank, Pim Klaassen, Michelle Rijnen, Sara Vermeulen, Remco Woertman, and Jacqueline Broerse. 2015b. D1.4: A catalogue of good RRI practices. RRI tools project. Amsterdam: Athena Institute, VU University Amsterdam. Nowotny, Helga, Peter Scott, and Michael Gibbons. 2001. Re-thinking science: Knowledge and the public in an age of uncertainty. Cambridge, MA: Polity. Oftedal, Gry. 2014. The role of philosophy of science in Responsible Research and Innovation (RRI): The case of nanomedicine. Life Sciences, Society and Policy 10 (5). Owen, Richard, Phil Macnaghten, and Jack Stilgoe. 2012. Responsible research and innovation: From science in society to science for society, with society. Science and Public Policy 39 (6): 751–760. Palm, Elin, and Sven Ove Hansson. 2006. The case for ethical technology assessment (eTA). Technological Forecasting and Social Change 73(5): 543–558. Rabinow, P., and G. Bennett. 2007. From bioethics to human practices, or assembling contemporary equipment. In Tactical biopolitics art, activism, and technoscience. Cambridge, MA: MIT Press. Regeer, Barbara, and Joske Bunders. 2009. Knowledge co-creation: Interaction between science and society. Den Haag: DeltaHage. Rip, Arie, Thomas Misa, and Johan Schot. 1995. Managing technology in society: The approach of constructive technology assessment. London: Pinter. Rowe, Gene, and Lynn Frewer. 2004. Evaluating public-participation exercises: A research agenda. Science, Technology & Human Values 29 (4): 512–556. Schomberg, von René. 2007. From the ethics of technology towards an ethics of knowledge policy & knowledge assessment. https://ssrn.com/abstract=2436380. Accessed 26 Jan 2017. ———. 2011. Prospects for technology assessment in a framework of responsible research and innovation. In Technikfolgen abschätzen lehren: Bildungspotenziale transdisziplinärer Methoden, ed. M. Dusseldorp and R. Beecroft. VS Verlag: Wiesbaden. Schot, Johan, and Arie Rip. 1997. The past and future of constructive technology assessment. Technological Forecasting and Social Change 268: 251–268. Stilgoe, Jack, Richard Owen, and Phil Macnaghten. 2013. Developing a framework for responsible innovation. Research Policy 42 (9): 1568–1580. Swierstra, Tsjalling. 1997. From critique to responsibility; The ethical turn in the technology debate. Techné: Research in Philosophy and Technology 3 (1): 45–48. Van den Hoven, Jeroen (ed.). 2013. Options for strengthening responsible research and innovation. http://ec.europa.eu/research/science-society/document_library/pdf_06/options-forstrengthening_en.pdf. Accessed 26 Jan 2017. Voß, Jan-Peter, Adrian Smith, and John Grin. 2009. Designing long-term policy: Rethinking transition management. Policy Sciences 42 (4): 275–302. Wickson, Fern, and Anna Carew. 2014. Quality criteria and indicators for responsible research & innovation: Learning from transdisciplinarity. Journal of Responsible Innovation 1 (3): 254–273. Wilsdon, J., and R. Willis. 2004. See-through science: Why public engagement needs to move upstream, 1–69. London: Demos. Wilsdon, J., B. Wynne, and J. Stilgoe. 2005. The public value of science: Or how to ensure that science really matters. London: Demos. World Economic Forum. 2016. The global risks report 2016. https://www.weforum.org/reports/ the-global-risks-report-2016. Accessed 26 Jan 2017. Worren, Nicolay, Keith Ruddle, and Karl Moore. 2016. From organizational development to change management: The emergence of a new profession. The Journal of Applied Behavioral Science 35 (3): 273–286. Zwart, Hub, Laurens Landeweerd, and Arjan van Rooij. 2014. Adapt or perish? Assessing the recent shift in the European research funding arena from “ELSA” to “RRI.”. Life Sciences, Society and Policy 10 (1): 1–19.