This is the first of two blog posts about the SUPER MoRRI annual event. This blog post focuses on the presentations given by the SUPER MoRRI consortium and general discussions on monitoring and RRI. The second blog post will discuss the various citizen science projects that were presented and how they relate to RRI.
Responsible Research and Innovation (RRI) as a policy device seeks to make a more just, inclusive, reflective, open, and responsive research and innovation (R&I) system. On the 29th of January, 2020, SUPER MoRRI (Scientific Understanding and Provision of an Enhanced and Robust Monitoring system for RRI), a project with an acronym as vast as its ambitions, brought together a group of nearly 90 academics, policy-makers, civil society actors to discuss citizen science and RRI. Discussions of power in citizen science projects, the function and role of monitoring for responsibility, 13 ‘Science with and for Society’ project pitches, and tensions between the demands of the academy and the needs of society were all debated. This blog focuses on the presentations of the SUPER MoRRI project team, the SwafS projects that were pitched, and the closing panel discussion by Wendy Reijmerink and Paul Wouters.
RRI, SUPER MoRRI, and other introductions
The day began with a set of presentations from the SUPER MoRRI team. Ingeborg Meijer and Ralf Lindner gave a brief description of the SUPER MoRRI project. The focus of this brief introduction highlights both streams of work being conducted within the project, which can be seen below:
It was emphasized that there must be a reflective and responsive approach to the monitoring framework that the project develops. This requires a continued reflection upon the indicators and their appropriateness alongside continuous attention on the ways in which RRI is being institutionalized. The SUPER MoRRI project is deeply committed to ensuring that the monitoring framework developed throughout the project is in line with recent recommendations and insights from research into science, technology, and innovation indicators.
Strategic and Implementation plans
The past year has been one rife with reflection and contention within the SUPER MoRRI project. Any assumption of the meaning, purpose, and makeup of indicators for the development of RRI has been opened, thoroughly inspected, and made explicit through hours of discussion and debate. These questions will remain an open question throughout the project, however some strands have been knotted and we have developed coherent strategic and implementation plans, which were presented by Richard Woolley and Niels Mejlgaard at the annual event.
The SUPER MoRRI Strategic Plan
When presenting the strategic plan of SUPER MoRRI, Richard Woolley highlights that monitoring will take the form of a framework, with the goal to:
- Support R&I actors engaged with various problems and challenges;
- Ensure contextually relevant information for users’ monitoring purposes and needs;
- Include consolidated, exploratory, and even experimental information streams and tools.
Core to the strategic plan of SUPER MoRRI are following the tenets of responsible quantification. Responsible quantification requires continuous reflexive conceptual development, staying attuned to the most recent findings in studies on the use of indicators, but also, and perhaps most challenging, humility. Responsible quantification requires acknowledging that what is deemed responsible or irresponsible is actor, context, and relation dependent. This requires coming to terms with the need for an open dialogue, especially with those being monitored, on what responsible quantification and monitoring means for them. Closely attached to this notion is that of credible contextualization. Credible contextualization requires acknowledging that concepts as contentious as RRI cannot be decontextualized and require accompanying information for users to properly contextualize their creation, use, and meaning. Not only is there a requirement to accompany indicators with information for contextualization, there must also be a shift towards their design and use ‘in the wild’. Ismael Rafols defines this shift as research that is ‘conducted with the participation of stakeholders in non-purified and non-isolated environments, with awareness of the values and contexts of use, and thus infused with reflexivity on the consequences of the knowledge created in the macrocosmos’. Furthermore, this also requires involving those who are historically in positions lacking the (social, economic, cultural) capital to intervene in monitoring practices, so as to democratize the science system and make it more responsible.
The SUPER MoRRI Implementation Plan
Niels Mejlgaard presented the implementation plan after the strategic plan. This presentation highlighted the diversity of data collection vehicles that exist throughout the project. These include international studies of RRI in the context of research funding organizations (RFOs), research performing organizations (RPOs), secondary data sources, the Eurobarometer, and a series of complementary case studies.
Following a keynote presentation by Alan Irwin (which will be discussed in an upcoming blog post), the SUPER MoRRI annual event featured a ‘pitch session’ which included ‘3 minute rapid fire’ presentations 13 different projects and organizations. The project websites can be found below:
For additional information on each project, and pictures from the pitch session during the event, please feel free to check out our twitter account: https://twitter.com/MorriSuper.
Closing the annual event – a panel with Wendy Reijmerink and Paul Wouters
The closing of the SUPER MoRRI event featured two concluding presentations by Wendy Reijmerink, a senior strategist at the Dutch medical research council (ZonMW) and Paul Wouters, the dean of the faculty of social and behavioural sciences at Leiden University. After the brief presentations, the two speakers participated on a panel discussion with the SUPER MoRRI team’s Roger Strand as the moderator.
Wendy Reijmerink presented the existing strategies and challenges in how to better govern the R&I system from a funder’s perspective. The strategies presented included ‘setting justifiable research priorities; robust research design, conduct, and analysis; ensuring that research regulation and management are proportionate to risks; and that information on research methods and findings from studies is accessible and usable’. Additionally, when reflecting on the function of Citizen Science within the priority setting approaches in the funder’s perspective, Wendy Reijmerink reflects that “Citizen Science strategies should adaptively address the core question [of] what do we want to achieve (outcomes), why and when (context), [and] how and with whom (engagement)?
Paul Wouters presented insights from his experience on the expert group for indicators of open science (https://ec.europa.eu/research/openscience/index.cfm?pg=altmetrics_eg). He highlighted in his presentation that there exist challenges when providing indicators for practices as complicated and diverse as those under the umbrella term of open science. No doubt the practices that fall under the notion of RRI are similarly complicated and diverse – if not more so. With the task of providing an indicator framework for a swathe of practices as complex and diverse as those within RRI, the SUPER MoRRI project should be informed by the advice given and lessons learned within the project of developing indicators for open science. Two of the most important lessons to consider when creating the monitoring framework for RRI that the SUPER MoRRI project is tasked to create are the following: First, one must consider the following dimensions in evaluation informed by indicator frameworks (I have modified these dimensions from the indicators of the open science report):
1. The Goal of the evaluation exercise;
2. What is the mission of the practices that are being evaluated;
3. What Scientific field and methodological approaches
4. Who are the potential stakeholders, audiences, and beneficiaries;
5. Research environment;
6. What human and technical resources exist that allow for the evaluated practices to develop;
7. What capabilities and infrastructure need to be present for the evaluated practices to occur?
These recommendations require the active reflection upon what the intended purpose of the evaluation is and how best to ensure that the desired practices are nourished by those indicators which are chosen to be included within the evaluation exercise. The second lesson involves asking the question: who should be able to determine the above considerations that go into the subsequent evaluation exercise? This requires humility and inclusion in determining the role for the evaluators and the evaluated. When the intention of an evaluation exercise is to change the practices of a community, that community should have the opportunity and power to decide which indicators would best inform the changes that they seek to take. This requires being tolerant of a conception of indicators and evaluation for learning, rather than solely monitoring or comparison.
The SUPER MoRRI annual event held on the 29th of January in Leiden contained a wealth of information, with a focus on how citizen science is changing the R&I system. If indeed the SUPER MoRRI project has the aim of supporting practices to make a more responsible R&I system, it will continuously benefit from engaging in an equitable dialogue with those communities who are already making these changes.