1 Motivation

Assessing the socio-economic impact of a Research Infrastructure (RI) requires an evaluation framework and a data collection plan that should be put in place as early as possible. This essay maintains the idea that if data collection for impact assessment is not episodic, motivated by external requests from stakeholders and funding agencies, but rather becomes a routine activity, it could maximize its potential to inform the RIs management and serve the strategic planning function of the infrastructure.

2 An Increasing Demand of Socio-Economic Impact Assessment

There is an increasing demand of demonstrating the expected socio-economic impacts triggered by research infrastructures already during the preparatory phase. Traditionally, the case for funding a new or upgraded RI has been usually advocated by scientific collaborations often supported by peer reviews exercise on the basis of a scientific case [1]. More recently, ex-ante impact assessment is becoming a routine analysis complementing the scientific and business cases accompanying the request for public funding (see for example the ESFRI Roadmap Guide 2021 and the Proposal Submission Questionnaire) [2, 3].

The fact that RIs can demonstrate their contribution to socio-economic development beyond the primary objective of excellent science is becoming crucial in the increasing international competition of hosting countries [4, 5]. The importance of promoting science facilities as a hub of knowledge and innovation within a wider ecosystem with a relevant territorial dimension is fully embraced by the EU Cohesion Policy. Investments funded by the European Structural and Investment funds for RIs should be planned however in the context of wider national or regional innovation strategies (according to the smart specialisation approach).

Against this background, evidence from a recent survey collecting almost 200 questionnaires from RI managers in the context of the Ri-Paths project,Footnote 1 indicates that 40% of respondents have some experience with socio-economic impact assessment [6]. However, only half of them carry out impact assessment on a regular basis, while the remaining ones only episodically and upon request. The main motivation is the need to comply with formal procedures for funding applications. Internal initiatives include mostly the need for strategic evaluation and reporting rather than ensuring public accountability. In addition, in the majority of cases, the socio-economic impact assessment has been carried out internally with own resources and only in few cases externally contracted to professional experts.

More widespread is the practice of performance monitoring as a routine reporting tool to funding agencies for accountability purposes. When looking at existing practices for data collection and monitoring based on respondents’ experience, the survey found that 41% collect data for management and compliance purposes based on a list of key performance indicators (KPIs), while only 9% collect impact indicators for the purpose of impact assessment exercises.

Impact assessment is linked but not identical to monitoring: performance monitoring is rather a continuous process generating data to track the progress of an action while impact assessment is a structured process that takes place at a given point in time allowing to assess the implications (past, future or both), of proposed actions. Monitoring indicators can be useful input to impact assessment exercises but shall not be confused with impact indicators.

Many RI managers reported that a systematic procedure to ensure efficient data collection and monitoring is not done in the majority of cases and among the main barriers to perform such activities there is the lack of internal resources and expertise. As a matter of fact, especially for large and multi-purpose facilities, there is the need for deep understanding and collaboration of different managers in the same RI to engage in a systematic data collection activity.

3 A Plan for Impact Assessment

Setting up a plan for impact assessment means deciding the roles (who does what), the timing (when: ex-ante, in itinere, ex-post, routinely every a number of years, etc.), the types of impact that are assessed (one or a sub-set in particular, all of them), which methods and techniques are used (with the related data collection methods and tools) and what is the expected use of the results in terms of feedback on strategy and management.

Currently there is no unique methodological framework for socio-economic assessment of RI but a variety of methods are used depending on the scope of the analysis, the type of impacts and the target users. Methods range from macroeconomic modelling to cost-benefit analysis to more qualitative narratives and case studies [7].

Despite different methodological approaches there is some consensus about the main impact areas (see [8,9,10,11,12]):

  • Scientific impact

  • Education impact

  • Technological spillover and innovation

  • Cultural and outreach

  • Science as a public good

For each of these items a short list of common indicators and tools for data collection can be identified (see Table 1). The list can be adjusted and expanded/restricted according to the specific need and remit of the assessment as well as the nature and type of activities of each RI.

Table 1 Examples of common impact indicators and tools for data collection by main impact areas
Full size table

As can be seen from the table below systematic tracking of scientific publications and citations, procurement contracts, patents and other innovation output, visitors and doctoral students, social media output and other dissemination products are the basis for a solid assessment and the condition for evaluability. In addition, regular surveys to former students, supplier companies and other users help to grasp useful insights on the way impacts materialise and how to maximise them. They require data collection strategies and evaluation plans. If a dedicated data collection plan is not in place, the ex-post reconstruction of past data can be challenging if not totally unfeasible. For example, getting in contact with students and researchers who spent time at a RI some years before might be impossible. Data protection issues are relevant in a number of cases, calling for a proper assessment and a rigorous management plan to ensure that there is no infringement to existing legislation.

For these reasons, setting up and implementing a plan for impact assessment from an early phase can make data collection more efficient and effective, despite some initial design costs.

4 Conclusion

Socio-economic impact assessment of research infrastructure is a challenging and time-consuming activity. Yet, RI managers are increasingly faced by requests by funding agencies and policy makers to show the socio-economic impacts beyond the purely scientific one. For this reason, the collection of a well identified list of impact indicators shall be planned and implemented early in advance to avoid problems of unavailable or too expensive data to be produced on past actions. Including impact data collection within the routine activities for strategic management, not only will make the assessment exercise more efficient and effective, but would also increase its solidity and thus credibility as a strategic management tool.