Abstract
The Horizon 2020 funded RI-PATHS project aims at providing policy makers, funders and Research Infrastructure (RI) managers the tools to assess the impact on the RI on the economy and their contribution to resolving societal challenges, etc. The goal is to improve the understanding of the long-term impact pathways of the various types of RIs operating in Europe, and, indeed, internationally. In the following we briefly summarise the work carried out by the RI-PATHS consortium partners to mid-2019 and derive useful lessons for future work.
Contribution to the ECONOMICS OF SCIENCE—Workshop, Brussels, 25 June 2019.
You have full access to this open access chapter, Download chapter PDF
Similar content being viewed by others
1 Introduction
The European Commission (EC)Footnote 1 defines research infrastructures (RIs) as facilities, resources and services that are used by the research communities to conduct research and foster innovation in their fields [1]. They include:
-
Major scientific equipment or sets of instruments;
-
Knowledge-based resources such as collections, archives or scientific data;
-
E-infrastructures, such as data and computing systems and communication networks;
-
Any other infrastructure of a unique nature essential to achieving excellence in research and innovation.
This definition is now internationally accepted (e.g. by the OECD’s GSF,Footnote 2 ESFRIFootnote 3 and the ERIC Council RegulationFootnote 4). Moreover, RIs can be:
-
1.
Single-sited facilities (unified single body of equipment at one single physical location);
-
2.
Distributed facilities (a network of distributed resources: instrumentation, collections, archives, and scientific libraries);
-
3.
Virtual facilities (e.g. ICT based system for scientific research, including high-capacity communication networks, and computing facilities providing services electronically, also known as e-infrastructuresFootnote 5), or
-
4.
Mobile facilities (vehicles designed for scientific research).
2 The RI-PATHS Project
The growing importance placed on the role of RIs, both in Europe and globally, as a basis for conducting research, but also for research-based education, is evident from the increasing public funding allocated to the design, development, operation and upgrading/renewal of RIs. Under the EC’s Horizon 2020 research and technological development framework programme (RTD FP), a key activity line is focused on research infrastructures and aims to “endow Europe with world-class research infrastructures that are accessible to all researchers in Europe and fully exploiting their potential for scientific advancement and innovation. The total budget for this activity in Horizon 2020 is €2.488 million.Footnote 6 In addition, during the period 2014–2020, the European Structural and Investment Funds (ESIF) are investing €6.931 million (out of a total planned expenditure by the European Union (EU) Member States of €10.131 million)Footnote 7 in public investments in favour of research and innovation infrastructures through the European Regional Development Fund (ERDF).
The overall scale of funding is thus significant and concerns investments in both European level RIs (ESFRI projects such as ELI, ESS, etc.) and national and regional scale RIs that serve a broad-based community of researchers and users (including business users, etc.). As the OECD [2] underline, the level and range of types of investments in RIs leads to a diverse group of stakeholders’ (ranging from national and regional public authorities, funders including charitable foundations, RI managers, scientific and industrial users and special interest groups and ultimately civil society) with (differing) interests in the topic and outcome of RI impact assessments.
In this context, the RI-PATHS projectFootnote 8 aims to provide policy makers, funders and RI managers the tools to assess RI impact on the economy and their contribution to resolving societal challenges, etc. The goal is to improve the understanding of the long-term impact pathways [3] of the various types of RIs operating in Europe, and, indeed, internationally. The project is being implemented over a 30-month period beginning in January 2018 and ending in June 2020. The activities include a review of the state of the art, consultation on the needs and practices of the RI community concerning IA, a phase of co-design with RI managers and stakeholders of an IA framework and the piloting of the IA framework by, at least, the four partner RIs (CERN, DESY, ALBA and ELIXIR). The project builds on and has ensured interaction with all relevant past and on-going initiatives including those by the OECD GSF, various ESFRI working groups, the ACCELERATEFootnote 9 project, etc.
Most approaches to date have sought to undertake specific type of impact assessments often at specific points in the lifecycle of an RI, including, e.g., cost-benefit analysis at the pre-investment stage or before an expansion or upgrading, impact on scientific excellence or the attraction of funding (from competitive programmes or industrial partners) during the operational phase, impact on suppliers through procurement, impact on human resources through training, doctoral research, etc. The RI-PATHS team acknowledge that a single ‘one-fits-all-model’ to assessing socio-economic impact is unlikely to meet the needs of the RI stakeholders. Rather the effort is focused on developing an IA framework adopting a modular approach with a generic core model and more detailed sub-models of IA. Moreover, the IA framework is designed to cover the whole life-cycle of RI development, operations and ultimately decommissioning.
The OECD [2] argued that impact assessment should be connected to the strategic objectives and mission of each RI. They proposed six standard objectives that they considered cover all the main dimensions of impact:
-
Be a national or world leading scientific RI and an enabling facility to support science.
-
Be an enabling facility to support innovation.
-
Become integrated in a regional cluster/in regional strategies/Be a hub to facilitate regional collaborations.
-
Promote education outreach and knowledge transfer.
-
Provide scientific support to public policies.
-
Provide high quality scientific data and associated services.
-
Assume social responsibility towards society.
3 A taxonomy for RIs
Taking account of the need to frame impact assessment in the context of the overall mission an RI has set itself (or in many cases has had set for it in the context of national, European or international funding), the RI-PATHS project developed [4] a taxonomy of types of RIs based on their mission and type of research conducted at the RI or research services provided to users (see Table 1).
The taxonomy was tested during the consultation phase of the project with RI stakeholders and managers. Overall, it was considered to be a useful framework as it covers a wide range of organisations with different objectives (60% of respondents of a survey of RI managers said it fits their case well). At the same, it was recognised that it may be difficult for many RIs to place themselves into just one category. Hence, the typology served as a framework (during the participatory workshops run by the project) to test the hypothesis that there are distinct pathways to impact depending on the level of interaction with users. To this end, a broader distinction can be made between:
-
(a)
RIs that offer services and perform own research activities;
-
(b)
RIs that are exclusively or primarily user/services oriented.
In line with OECD [5], we define impact as any long-term effect, whether intended, unintended, positive, negative, direct or indirect produced by an intervention. They are the ultimate changes produced in the society by means of a given action or investment decision. Having an overarching list of impact categories can allow a RI to identify where it is not having impacts, as well as where it is having some. This is an important issue for accountability and advocacy, since it allows RIs to investigate more closely why, for instance, it is ‘failing’ to achieve desired results under certain types of impacts.
A distinction can be made between impacts caused by the RI pursuing its core science mission and impacts caused by the RI being a socio-economic actor. In broad terms, these are both socio-economic impacts resulting from the operation of an RI. The former can be e.g. contribution to short-term and/or long-term problem solving, qualification of scientists, impacts on innovation and productivity in the economy, opening up of new perspectives in the policy discourse and outreach and popularisation of knowledge in society. Examples of the latter include employment effects, wages paid and multipliers, qualified procurement effects with impact on innovation, procurement of standardised, off-the-shelf goods and qualification effects for technical staff.
Demonstrating the causal pathway between investment in RIs and the wider socio-economic impacts can be an intricate, complex and labour-intensive task. This suggests the need for a framework that classifies the diverse impact pathways. An IA framework should provide the opportunity to visualise the potential pathways to impacts or the forms of impact that exist. Visualisation can also help understand the system by which impacts are produced and thus improve it.
Accordingly, a core element of the RI-PATHs project has been on co-creating with the RI community a logical model that links RI activities to outputs, outcomes, longer-term results and finally impacts [6]. As described in Fig. 1, the IA framework has been developed in two stages via a series of participatory co-design workshops, followed by a validation workshop and piloting in the four partner RIs before the launch of a final impact assessment framework.
4 Discussing Preliminary Results
The findings of the first round of workshops underlined that there are three main lines of inquiry relevant for all types of RIs (even if each of them may be more or less strongly associated with a specific type).
-
Quantifiable impacts that can be captured through quantitative metrics available e.g. in economic analysis: In some areas, impacts can be determined reliably based on economic modelling. Areas to be thus covered include the transmission of socio-economic effects through employment, (qualified) procurement, some dimensions of training as well as RIs ‘gravitational effect’ on visitors. This approach takes and justifies assumptions but does not further reflect on specific pathways in the process. It has the clear advantage to produce aggregate figures in monetary terms and easy-to-process results for the political discourse.
-
Non-quantifiable impacts the assessment of which can, however, be supported by quantitative means (e.g. social network analysis, dedicated surveys): Some impacts can be characterised through quantities but not suitably translated to monetary terms and aggregated. In these areas actor-based analysis can chart patterns of interaction between RIs and their socioeconomic environment, most prominently their users. It can quantify these interactions and to the extent possible attribute specific effects to them. Thus, it can help establish not only the socio-economic, but also the societal contribution of RIs based on the properties of the network that they have built around them and the known impact pathways that they can be demonstrated to trigger.
-
Complex network effects captured through exploratory approaches which require qualitative approaches such as narrative-based case studies: The analysis of network effects has been highlighted as an area of particular relevance for distributed RIs. It has so far not been intensively explored and will require more work to establish methodological foundations. Ideally, it should be possible to outline a core set of quantitative indicators for this area, e.g. in the area of human capital. In many other areas, this type of analysis will look at complex issues that quantitative approaches fail to satisfactorily capture.
During the first round of RI-PATHS participatory workshops four categories of impacts were used to frame the discussions: (1) Direct economic impact, (2) Human resources impact, (3) Social/societal impact and (4) Policy impact (see Fig. 2). During the workshops, these four broad categories were used to identify twelve stylised impact pathways that help to measure the multi-faceted impact of research infrastructures [7].
The goal of the second round of workshops was to engage the RI community in the identification of suitable indicators and metrics that can, in practice, serve to measure the various impacts identified in the first round. As a result of the discussions, the RI-PATHS team decided to re-phrase the initial set of pathways as follows:
Impacts as a result of RIs Enabling Science (i.e. performing their primary mission)
-
Employment & standardised procurement
-
Publication-citation-recognition (incidental spillovers)
-
Technology transfer & licensing (supported spillovers)
-
Learning- and training-through-specialised procurement
-
Generic learning- and training-through-usage
Impacts as a result of RIs Supporting Problem Solution (by providing services to users)
-
Interactive industrial problem solution
-
Interactive public sector problem solution
-
Provision of specifically curated/edited data to industry
-
Provision of specifically curated/edited data to public sector
-
Other interactive societal problem solution
Impacts through shaping future trends in Science and Society
-
Changing fundamentals of research practice
-
Contribution to formal standards in science
-
Creating and shaping scientific networks and communities
-
Creating and shaping networks between science and society
-
Communication, outreach and engagement.
5 Conclusion
The proposed conceptual framework (see Fig. 3) fulfils two central functions:
-
First, it defines impact domains where RI may play a role. While political expectations may shift, findings from the participatory workshops and surveys have confirmed that IA should no longer be solely motivated from a purely compliance-oriented perspective. Increasingly, many RIs are articulating their mission to contribute to broader societal goals.
-
Second, the model helps to define the logic of impact causation, or impact pathways, through which different types of impact materialise. Understanding these pathways can assist decision making by RI managers in terms of the launch of or intensification of specific internal activities or through the collaboration and engagement with external partners.
Given this overall framework, the second round of workshops were also used to assess and refine a long-list of indicators down to a workable set that can be used to track the various types of impacts and the inter-relations between the impact pathways [8]. In the final step of the project, this approach will be piloted, then refined into an operational guidance that will provide RI managers and stakeholders with an easy-to-use framework that helps them identify the sort of questions an IA exercise should answer and what data needs to be collected.
Moreover, the EC has stated that the RI-PATHS framework will guide the future IA of ESFRI projects and the project team expect that the guidance (to be licenced under Creative Commons) will be further developed by practitioners in the coming years.
Notes
- 1.
Article 2 (6) of Regulation (EU) No 1291/2013 of 11 December 2013: ‘Establishing Horizon 2020 – the Framework Programme for Research and Innovation (2014–2020)’.
- 2.
Science Global Forum OECD (2014; 2016) and OECD (2014).
- 3.
ESFRI, the European Strategy Forum on Research Infrastructures (2010).
- 4.
Community Legal Framework for a European Research Infrastructure Consortium (ERIC) (2009).
- 5.
See the catalogue of e-infrastructure services developed by the eInfraCentral project available at https://www.einfracentral.eu/home
- 6.
- 7.
Source data: https://cohesiondata.ec.europa.eu/d/s499-6d7x
- 8.
See: www.ri-paths.eu. All deliverables of the project can be downloaded here.
- 9.
References
EU (2013) Consolidated text: Regulation (EU) No 1291/2013 of the European Parliament and of the Council of 11 December 2013 establishing Horizon 2020 - the Framework Programme for Research and Innovation (2014-2020) and repealing Decision No 1982/2006/EC (Text with EEA relevance). https://eur-lex.europa.eu/legalcontent/EN/ALL/?uri=CELEX:02013R1291-20150704
OECD (2019), “Reference framework for assessing the scientific and socio-economic impact of research infrastructures”, OECD Science, Technology and Industry Policy Papers, No. 65, OECD Publishing, Paris, https://doi.org/10.1787/3ffee43b-en.
Griniece, E., Reid, A. and Angelis, J. (2015). Guide for Evaluating and Monitoring the Socio-Economic Impact of Investment in Research Infrastructure. DOI: https://doi.org/10.13140/RG.2.1.2406.3525/1
Giffoni, F, Vignetti, S., Kroll, H., Zenker, A., Schubert, T., DeYoung Becker, E., Ipolyi, I., Griniece, E., Angelis, J., (2018). Working note on RI typology. RI-PATHS Project (Brussels) DOI: https://doi.org/10.13140/RG.2.2.29020.23684
OECD (2002) Glossary of Key Terms in Evaluations and Results Based Management. Re-printed in 2010
Vignetti, S., Griniece, E., Cvijanovic, V., Reid, A., Helman, A., Kroll, H. (2019), Report on stocktaking results and initial IA model. RI-PATHS project (Brussels). DOI: https://doi.org/10.13140/RG.2.2.34844.95360
Griniece, E., Kroll, H., Cvijanovic, V., Zenker, A., Reid, A. (2019) Concept note of the modular impact assessment framework. RI-PATHS project (Brussels) DOI: https://doi.org/10.13140/RG.2.2.28102.42566
Kroll, H., Zenker, A., Hansmeier, H., Griniece, E., Helman, A., Angelis, J, Vignetti, S., Reid, A. (2019) Consolidated report on the participatory workshop results. RI-PATHS Project (Brussels). DOI: https://doi.org/10.13140/RG.2.2.34813.31201
Full Presentation
Acknowledgements
This short paper summarises the work carried out by the RI-PATHS consortium partners to mid-2019 and draws on the key deliverables coordinated, notably, by Eline Griniece (EFIS Centre), Henning Kroll (Fraunhofer ISI) and Silvia Vignetti (CSIL).
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Open Access This chapter is licensed under the terms of the Creative Commons Attribution 4.0 International License (http://creativecommons.org/licenses/by/4.0/), which permits use, sharing, adaptation, distribution and reproduction in any medium or format, as long as you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons license and indicate if changes were made.
The images or other third party material in this chapter are included in the chapter's Creative Commons license, unless indicated otherwise in a credit line to the material. If material is not included in the chapter's Creative Commons license and your intended use is not permitted by statutory regulation or exceeds the permitted use, you will need to obtain permission directly from the copyright holder.
Copyright information
© 2021 The Author(s)
About this chapter
Cite this chapter
Reid, A. (2021). Designing a Socio-Economic Impact Framework for Research Infrastructures: Preliminary Lessons from the RI-PATHS Project. In: Beck, H.P., Charitos, P. (eds) The Economics of Big Science. Science Policy Reports. Springer, Cham. https://doi.org/10.1007/978-3-030-52391-6_9
Download citation
DOI: https://doi.org/10.1007/978-3-030-52391-6_9
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-030-52390-9
Online ISBN: 978-3-030-52391-6
eBook Packages: Physics and AstronomyPhysics and Astronomy (R0)