!!2. FP9 ‘s ambitious aims for societal impact call for a step change in interdisciplinarity and citizen engagement

The European Alliance for SSH welcomes the invitation of the Commission to
contribute to the development of the next Framework Programme. In response
EASSH has prepared two position papers, with a thematic and a cross-cutting
focus respectively. This paper addresses the cross cutting issue. Its starting point
is the mid-term review of Horizon 2020, in particular its finding that, although
the Societal Challenges multidisciplinary approach offered benefits in
comparison with previous funding programmes, there were two unintended
effects; the calls appeared to promote projects with little or no innovation and,
the evaluation process did not respond adequately to H2020’s ambitions for
\\ \\
The on-going discussions in relation to FP9, including various high-level
indications from the Commission, suggest that FP9 will continue to address
societal challenges, will identify a number of key missions, will pay increased
attention to social impacts and will include innovative approaches to promote
citizen engagement and participation.
\\ \\
EASSH greatly welcomes these ideas and trends: the European SSH community,
represented by EASSH, has been a strong champion of “real” interdisciplinarity
and of concrete and innovative approaches to citizen engagement at various
critical stages in the programme and project cycles.
\\ \\
Our concrete proposals are elaborated below, organized into three main groups
of actions for FP9- especially its collaborative societal challenge elements. While
we believe that each of our proposals will contribute to the above aims we are
also convinced that by combining them within an __overall strategic framework
for interdisciplinarity and engagement__ FP9 can make an unprecedented
contribution to the concerns and aspirations of citizens.
\\ \\
On the basis of our analysis and based on the review of papers submitted for the
consultation, EASSH would like to make recommendations in three main areas:
__radical reform of instruments to support research; programme evaluation
and key performance indicators; and dedicated evaluations of
interdisciplinary proposals__. The latter two sections also address the
assessment of social impact. This paper also stresses that fundamental research
in relation to the societal challenges has a central role, and should not be ignored
but rather nurtured and encouraged.
!1. Instruments to Support Multidisciplinary Research: Social Missions and Integrative Platforms

EASSH encourages __the expansion of the range of instruments to support
multidisciplinary research__ in the 9th framework programme.
First, we need to look beyond the short-termism of the current 3-year ‘project’
cycle. Many of the problems to be tackled through the missions will require
contributions from across research fields and involve a wide variety of
stakeholders. EASSH believes that such missions could be considered as being
appropriate for longer-term investment in the form of ‘__integrative platforms__’,
which bring together researchers and stakeholders in sustained collaborations.
__EASSH supports the introduction of ‘integrative research platforms’, which
would be supported over 6-8 years__, which will partners time to develop
approaches to working across disciplines, to learn from early research outcomes
and to respond to changing social dynamics in a context which encourages both
research and innovation. These platforms will engage with bottom-up issues
where there is emerging critical mass to enforce dynamic collaborations of
different disciplines and different agents. This is where the real __European
Added Value__ for Research lies in sustained investment in key issues.
\\ \\
EASSH reiterates its position that research endeavours should be guided by the
nature of the challenges being addressed and not by a limited number of predetermined
‘instruments’. EU-level collaborative research is not always best
implemented by large-scale projects with many partners. EASSH believes that
__supporting more, and smaller, social missions__ can provide more targeted
research insights for both local and European policy makers. Such missions
should emerge from research stakeholders and users - such as policy makers,
citizens and civil society organisations to provide momentum to address a
relevant and pressing issue.
\\ \\
More focused social missions, on the one hand, and integrative platforms, on the
other, are complementary approaches to address major research challenges.
EASSH hopes that this is the meaning behind the HLG’s recommendations calling
for __adaptability in choice and the design of funding instruments__. In fact,
defining instruments and appropriate responses to the social missions should be
primarily determined by those engaged in such missions, rather than decided in
a rigid, top-down way.
!2. Programme Assessment and Key Performance Indicators

EASSH members have been involved in different projects to review the concept
of impact assessments in different programmes. It has now been clearly
established that impact cannot be expressed as a linear process. In-depth
research has demonstrated that moving ‘from lab to market’ hardly ever occurred in such a simple progression and, more importantly, that it is not just a market ‘product’ that demonstrates achievement, but rather social uptake and
scalability which also need to be taken into account.
\\ \\
We propose that assessment of research __impact should be looked at on an
aggregate level__ and not in relation to single projects. Assessment should take
into account what the call design and the programme were originally intended to
achieve. Impact of research, in fact, is not just about ‘''fixing problems''’ or ‘''making
things''’. It is about generating new knowledge and evidence, and understanding
how these are used for developing pathways and measures towards addressing
an issue or a challenge, which may be relevant to European society now or in the
future. Crucially, this process should also address the assessment of calls and of
challenges, which will allow the __unintended effects of research developments
to be clearly identified__.
\\ \\
In a similar view, we need to identify the right __key performance indicators to
measure whether a programme delivers what it has been designed for__. We
should not use or re-use indicators where the aims and purpose of a programme
are new; we will need to identify appropriate new indicators.
In Horizon 2020, the approach of the Societal challenges was innovative, but it
continues to be evaluated on the basis of the same key performance indicators
used in previous frameworks. This suggests that some of the aims remain
consistent across FPs: namely, academic excellence and interplay between public
and private research collaboration to produce new knowledge pathways and
social and technical innovation. However, if FP9 is to introduce new approaches,
like value for society, then we must recognise that we cannot rigidly apply the
same sets of indicators. We must review and update these so that they can reflect
the multiple dimensions of what the new programme is meant to achieve.
For example, some of the current indicators remain far too remote in the attempt
to assess whether research funding provided by the EU has a real value added
for European citizens. A balanced evaluation must take into consideration
academic publications, as they remain important for the dissemination and
communication of science, but should be combined with tracking research
communications beyond traditional journals into other media channels with a
wider readership.
\\ \\
Patents and prototypes, as well as new products and process, are valuable
indicators of the relevance of some research, but can tell us little about usage or
market exploitation or social acceptance and benefits. __Indicators need to
address, among other dimensions, citizens’ involvement in the process of
acceptance of products, or the reception of social innovation initiatives
where entrepreneurs have been able to access freely the information__
generated within the project. Society does not just benefit from products, but
more importantly from a fundamental understanding of the social dynamics that contributes to their success. Finally, we must encourage __policy makers to play
their part in recognising the influence of research by citing the emerging
knowledge__ that has influenced a policy or an idea, directly or indirectly.
\\ \\
EASSH is working in collaboration with European civil society to facilitate more
open access to knowledge on the part of those who can most benefit from funded
research, with the expectation that they will then report back on the usefulness
of the work they have been able to access and read. __EASSH proposes a close
collaboration between researchers and users of research__ for the purpose of
generating a better social understanding of the role, importance and impact of
research. Programme evaluations need in turn to engage with the research users
to understand how projects have influenced a given sector or an area of specific
!3. Evaluation for multi and interdisciplinary projects

In some preliminary analysis conducted by EASSH, we have come to the
conclusion that it is questionable whether the current evaluation process for the
selection of projects to be funded in H2020 is fit for purpose; whether the
Commission has been able to create a pool of experts with the correct blend and
depth of expertise; whether the conditions have been created to identify the best
multi and interdisciplinary projects to deliver the overall aims of the Societal
Challenges in the Horizon 2020 programme. EASSH provides below a set of
technical recommendations for a more efficient evaluation of proposals.
\\ \\
For FP9 to provide a successful proposal evaluation process, __EASSH proposes
that the Advisory Groups that design the aims and purposes of the calls
within the projected challenges should also contribute to the
establishment of semi-permanent proposal evaluation panels__. These must
be populated with experts who have the full range of skills and expertise from all
sectors of society to assess whether the call intentions are being fulfilled by
proposed projects. We also call for greater stability in the membership of such
panels, which will learn to work together over time and understand how to
reward truly interdisciplinary projects in line with the aims of the calls. A certain
degree of coherence and consistency emerges in this process too as those
designing the calls can fully ensure that relevant expertise is brought to bear on
the evaluation. Self-nominated experts and selection of reviewers on the basis of
keywords are not efficient as we demonstrated in a previous paper on ''Evaluation
in H2020 Societal Challenges''.
\\ \\
Finally, proposal evaluation must reserve a space to assess - __ex-ante - the
potential for the impact of each project to be evaluated__. This should include
a declaration of objectives at the outset (i.e. relating to potential impact) on the
part of the research consortium, highlighting how methodologically robust
research is combined with a demonstration of relevance in society, via means of consistent engagement with the research subjects or beneficiaries. Alongside the
scientific evaluation of projects, teams will also be expected to show evidence of
their relevance in their direct or indirect influence over time of the social
environment in which they are to be realized.
\\ \\
EASSH is working in collaboration with European civil society to facilitate more
open access to knowledge on the part of those who can most benefit from funded
research, with the expectation that they will then report back on the usefulness
of the work they have been able to access and read. __EASSH proposes a close
collaboration between researchers and users of research__ for the purpose of
generating a better social understanding of the role, importance and impact of
research. Programme evaluations need in turn to capture from the research
users how projects have influenced the sector or the area of specific interest. At
the same time, the relevance of research must also be assessed according to the
extent to which research has provided a fertile environment for ideas to develop
and evolve, even through negative findings or the identification of unproductive
pathways for future work.
\\ \\
Download the [document|FP9_Technical issues_fnl.pdf]