Keywords

1 Policy Context

EU-level investment in Research and Innovation (R&I) focuses on excellence through EU-wide competition and cooperation. Successive EU Framework Programmes have aimed at supporting training and mobility for scientists, creating transnational, cross-sectoral and multidisciplinary collaborations, leveraging additional public and private investment, building the scientific evidence necessary for EU policies, and strengthening national research and innovation systems. Over the years, the political narrative has put more and more emphasis on ’ shaping the future’ through R&I policy and funding, thereby lending even more importance to the assessment of the funding programmes’ economic impact.

Horizon Europe is the European Commission’ s proposal for the 2021–2027 Framework Programme for EU R&I policy, succeeding the Horizon 2020 Programme (active between 2014–2020).Footnote 1 With a proposed budget of about 100 billion euros for the period 2021–2027, Horizon Europe is the most ambitious R&I funding programme ever. This Programme builds on lessons learnt from previous evaluations, as well as on feedback from experts and from other stakeholders.Footnote 2 It will be an evolution, not a revolution, focusing on a few design improvements to further increase openness and impact. These changes in design aim at making this Programme achieve even more impact than its predecessor (through, i.e., the European Innovation Council and mission-orientation) and more openness (through strengthened international cooperation, a reinforced Open Science policy, and a new policy approach to European Partnerships).

2 Macroeconomic Modelling, EU R&I Framework Programmes and the EU Policy Cycle

Assessing the impact of the Framework Programmes is crucial for policymakers in order to inform their strategic decisions. There is a general consensus (Hall et al., 2009; Di Comite & D’Artis, 2015; European Commission, 2017c) that R&I policies are decisive in fostering productivity growth. However, putting a precise figure on the expected benefits of large R&I programmes such as the EU Framework Programmes is a challenging task with a lot of uncertainties, especially in an ex-ante approach. This is rendered even more difficult by the long-term horizon that a proper analysis of these impacts requires.

In this context, macroeconomic modelling is an essential tool to support policymaking, since it attempts at quantifying the impact of the Programmes and assessing policy options. Depending on when the assessment takes place in the policy cycle (Figure 8.1), this can be done in an ex-post/interim (monitoring and evaluation of a programme) or ex-ante/design (impact assessment) fashion, with policy options examined during impact assessments only in order to feed the preparation phase of the Programmes.

Fig. 8.1
figure 1

(Source adapted from the EU better regulation guidelines (European Commission (2015)))

The EU policy cycle

Full size image

The first ever ex-ante impact assessment of any EU policy initiative in the field of research was the impact assessment of the 7th Framework Programme (FP7) (Muldur et al., 2006; Delanghe & Muldur, 2007). This exercise relied on historical data (e.g. publications and patents) and on simulations based on a macroeconomic model. The NEMESIS model was used for this impact assessment, and subsequently for the impact assessment and interim evaluation of Horizon 2020 (European Commission, 2012; European Commission, 2017a). Since FP7, macroeconomic models have evolved and lessons from previous impact assessments can help policymakers in using these models for current and future assessments.

The latest assessment of a EU R&I Framework Programme is the impact assessment of Horizon Europe (European Commission, 2018). A key novelty in the approach for this assessment is the use of three different macroeconomic models for assessing the continuation of the Programme (’baseline’ scenario), which are the models presented in the previous chapters: QUEST III, RHOMOLO and NEMESIS.

3 How much is the Continuation of Horizon 2020 Worth?

Quantifying the impact of R&I policies at a macroeconomic level requires modelling tools that appropriately capture how R&I translates into economic gains. By relying on three models, namely QUEST III, RHOMOLO and NEMESIS, the impact assessment of Horizon Europe (European Commission, 2018) was aimed at leveraging on their respective strengths while partly counterveiling some of their limitations.

The strengths of these models rely on their specificities, and differences between the models can help address specific needs of policymakers. Di Comite and D’Artis (2015) consider that NEMESIS is the richest model in terms of the types of innovation types captured and the number of policy-sensitive elasticities when compared to other standard macroeconomic models for R&D and the number of innovation policies. This means that policymakers can easily design and evaluate a wide range of policy options related to specific innovation types or innovation channels when using this model. On the other hand, the forward-looking, dynamic approach of QUEST makes the model most appropriate for assessing the impact of R&D and innovation policies over time. This is particularly important as the effects of initial investments are expected to bear fruit only after the period covered by the Programme, which calls for a model that can measure long-term impacts with precision. Finally, by modelling regional economies and their spatial interactions, RHOMOLO is the most suitable model to address questions related to the geographic concentration of innovative activities and spatial knowledge spillovers, which is also a crucial aspect for policymakers.

When using and interpreting the results produced by these models, it is also essential to acknowledge their main limitations. Any model allows only for a partial representation of reality, subject to the assumptions made. RHOMOLO balances its detailed spatial and regional dimensions by keeping optimisation problems static and, hence, not capturing the inter-temporal consequences of innovation decisions. These are binding constraints for ensuring the tractability of the model. In addition, it does not distinguish between private and public innovation or between different types of endogenous innovation. On the other hand, QUEST III, not being a multisector macroeconomic model, groups all R&D activities in a unique R&D sector without capturing the complexity and diversity of the type of R&D investments, such as private and public R&D activities, product and process innovation, non-R&D and disruptive innovations. These elements are also not present in RHOMOLO, albeit the latter features more extensive sectoral and geographical details. Lastly, NEMESIS is based on empirically observed relationships among variables as well as on adaptive expectations instead of forward-looking ones, allowing for more degrees of freedom in behaviour than in other models. This may generate inconsistencies with recent developments in macroeconomic theory. As opposed to the other two models, however, NEMESIS incorporates private and public R&D activities, product and process innovation, and non-R&D investments.

With these caveats in mind, Figure 8.2 shows the comparisons of the simulated impact of Horizon Europe on the GDP trajectory discussed in the previous chapters.

Fig. 8.2
figure 2

(Source European Commission (2018); Note EU+ indicates that NEMESIS uses higher performance and leverage for EU funding compared to national funding as a reflection of the EU added value of the Programme. QUEST *1 assumes that financing of the Programme relies on VAT increases. QUEST *2 assumes that financing relies on lowering public investment)

GDP impact of horizon 2020 continuation (% deviation from a baseline, no framework programme scenario)

Full size image

Overall, Figure 8.2 shows that NEMESIS, QUEST and RHOMOLO present consistent results in terms of the sign and temporal pattern of the GDP gain from the Framework Programme (compared to the discontinuation of the Programme) over 2021–2050. The three models show a strong increase in GDP especially after the period covered by the Programme, with highest impacts predicted between 2029 and 2034. The size of the GDP gain is highest for the simulations based on the NEMESIS model. This can be explained by the fact that the three models use different sets of innovation channels and elasticities.

Furthermore, the parameters and mechanisms in QUEST and RHOMOLO do not directly take into account the higher leverage and performance expected from EU funding of R&I compared to national funding, which are acknowledged in NEMESIS as an illustration of the EU added value of the Framework Programme. This can potentially explain a significant part of the difference between the results from NEMESIS and the other models. Several studies (Delanghe et al., 2011; Vullings et al., 2014; Rosemberg et al., 2016; ECDG & Elsevier, 2017; PPMI, 2017) provide empirical evidence that shows that EU funding could be expected to perform ’ intrinsically’ better at EU level compared to national level due to factors that are not directly captured by these models, such as multidisciplinary transnational collaborations or critical mass. However, the way this EU added value is translated in a model, i.e. the size of the effect, is not trivial and requires caution in its interpretation.

Another essential aspect for all models is the mode of financing of the Framework Programme. Money spent for the Framework Programme can come from different sources, and it is tempting but rather unrealistic and undesirable to not model how the funds are financed. In this regard, both RHOMOLO and NEMESIS assume that the financing of the Programme can be reflected by lower national expenditure. The results from QUEST highlight the difference between two funding scenarios: (i) raising additional VAT revenues across Member States and (ii) lowering national public investment. It is shown that VAT funding should be unambiguously more beneficial compared to the second scenario as it allows Member States to continue public investment in productive uses.

In short, the three models used for the impact assessment of Horizon Europe are based on different modelling strategies, assumptions and parameters specifications and values, which results in different quantitative estimates of the economic impact of Horizon Europe. Nevertheless, the comparison of results across different models is essential to ascertain the consistency of a policy intervention, in this case Horizon Europe. This comparison is also required to understand the different aspects and mechanisms at play within the models, which partially mirror those determining the actual impact of Framework Programmes.

4 Modelling for Policymaking

Overall, past experience demonstrates the growing importance of macroeconomic modelling in the evaluation and impact assessment of EU R&I policy. The need for state-of-the-art modelling approaches all along the policy cycle has never been as pressing today. However, the complexity of the modelling exercise can make it challenging for policy-makers and modelers to collaborate effectively. In this regard, modelers also have a role to play to help policymakers understand the key aspects and assumptions that they need to reflect upon when using and interpreting models and their results. For instance, while discontinuation versus continuation scenarios can be straightforward to interpret and can inform policymakers on the ‘cost of non-Europe’, it can be challenging to translate policy options regarding the design or implementation of a Programme into assumptions in the models if there is lack of collaboration or understanding from the different parties involved.