FTA Seminar Theme 2 Anchor Paper Extended Abstract

FTA Seminar Theme 2 Anchor Paper Extended Abstract

FTA Evaluation, Impact and Learning

Remi Barré and Michael Keenan

FTA activities come in many shapes and sizes, with variations in contexts (e.g. policy and socio-cultural milieu, resources available for the exercise, etc.), rationales and objectives (see Box 1), intended audiences, time horizons, methodological approach (including the extent of participation), and the way knowledge is generated, transferred and mobilised during and after the exercise. A further ‘layer’ of variety comes in the form of competing interpretative frameworks, i.e. in the ways in which we understand and give meaning to FTA activities. These interpretative frameworks are apparent among FTA participants (particularly among those who are responsible for managing or sponsoring FTA, who will hold a set of assumptions as to the effectiveness of their interventions) and foresight analysts and evaluators.

Box 1: Some common objectives for FTA activities[1]

·  Support decision-making

·  Form new networks, provoke existing ones

·  Facilitate better understanding of potential disruptive change

·  Support the empowerment of system actors

·  Gain insights into complex interactions and emerging drivers of change

·  Facilitate thinking out of the box

·  Challenge mindsets

·  Detect and analyse weak signals to ‘foresee’ changes in the future

·  Produce future-oriented material for the system to use

·  Support system actors to create their own futures

·  Aid discussions of the future

·  Creating a shared vision

·  Build trust between system actors

·  Managing external pressures and challenges

·  Overcome path dependency and lock-ins

·  Improve policy implementation

·  Contribute towards development of identities

·  Provide anticipatory intelligence to system actors

·  …

A lot of claims have been made as to the benefits of using FTA in support of policy making and decision making more generally. Moreover, in recent years, further ‘softer’ benefits have been suggested, such as network building, changing mindsets, building trust among actors, and developing better preparedness for change. Many of these supposed benefits, both tangible and intangible, have been increasingly adopted and articulated by the managers and sponsors of FTA in the formulation of the objectives set out for their exercises (see Box 1).

At the same time, many problems have been regularly highlighted. For example, in the specific case of foresight exercises, some of the reported problems include:

·  the generation of visions that are too vague for follow-up action

·  disconnection of foresight exercises from real centres of power, making the likelihood of impacts less certain

·  a lack of vision that results in few surprises, which calls into question the added-value of foresight

·  tokenistic or limited participation in foresight exercises by system actors

·  insufficient learning across space and time resulting in much re-inventing of the wheel

·  at the same time, sometimes inappropriate emulation of foresight approaches and rationales from one setting to another

·  the challenge of managing unrealistic expectations of the nature and time-scale of impacts of foresight exercises. This is largely related to an immature understanding of how foresight might have effects in a given setting, i.e. ignorance of foresight’s theories of action.

Staying specifically with foresight, according to van der Meulen et al (2003),

“Foresight processes require quite substantive investments of direct, often public funding, and imply considerable costs in terms of time and expertise invested by the participants. If effects of such processes cannot be made clear, the willingness to invest resources will soon decrease, and as a result, the possibility for a good foresight process as well. (…) Further learning on foresight is necessary and can only be achieved by investing in a better understanding of the dynamics of foresight”

As things stand, not only is there an absence of a better understanding of the dynamics of FTA, there is also little systematic evidence to suggest that many of the claimed-for benefits regularly accumulate from FTA activities. There are several reasons for this:

·  There are so many different methodologies and settings for FTA that it is difficult to arrive at standardised evaluation approaches

·  The objectives set for FTA are often wide-ranging and vague, making them problematic starting points for evaluation

·  The intangible benefits that are said to accrue from FTA are difficult to assess in themselves

·  The complexity of cause–effect relationships which cannot be handled by the often overly simplistic models used when trying to understand and give meaning to FTA activities and their effects

·  The systemic and distributed nature of FTA means that benefits are likely to be dispersed across a landscape of actors and systems making attempts to account for effects very difficult

·  The costs associated with evaluating FTA activities

At the moment, policy makers and analysts are still trying to better define the expected outputs and outcomes of FTA, based largely upon an empiricist approach of learning from case experiences. Box 2 lists several reported benefits associated with several foresight and futures exercises that have been mapped by the European Foresight Monitoring Network (EFMN).

Box 2: Reported impacts of foresight exercises (www.efmn.info)

·  Better informed strategies in general

·  Making the case for increased investments in R&D

·  Using foresight results to evaluate and future-proof strategies

·  More informed STI priorities

·  Development of new ways of thinking

·  Creating a language and practice for thinking about the future

·  Highlighting the need for a systemic approach to both policymaking and innovation

·  Development of reference material for policymakers and other innovation actors

·  Better evidence-based policy

·  A source of inspiration for non-governmental actors

·  Creation of new networks and clusters

·  Establishment of communication structures between innovation actors

·  Collective learning through an open exchange of experiences

·  Enhanced reputational position and positive image of those regions running a foresight

·  Better visibilities of a region’s strengths and competencies

·  Interest from the general public

·  Achievement of long-term reform of the productive system through a raised emphasis on high technology

·  Accumulation of experience in using foresight tools and thinking actively about the future

·  Stimulation of others to conduct their own foresight exercises after being inspired

·  …

Whilst an empiricist approach is worthy and necessary, the argument in this paper is that it is insufficient on its own. To identify and elaborate ‘theories of action’, there is a need to submit FTA practices to interpretation of their significance by the relevant disciplines of the social sciences and humanities (SSH). These will provide a variety of interpretative lenses that offer the possibility to expand our conceptualisation of FTA, which will in turn improve the prospects for evaluating processes and outcomes. The following SSH disciplines could offer potentially useful lines of interpretative enquiry and will be explored more extensively in the full version of this paper:

·  From epistemology, the status of knowledge claims generated through the use of FTA methods; FTA knowledge production in the continuum from ‘scientifically-certified’ to ‘socially-robust’ to ‘politically-relevant’ knowledge

·  From political science, FTA as an instrument of deliberative democracy; the role of FTA in the evolving multi-level governance structures of knowledge societies; the legitimacy of FTA activities in relation to the mainstream political system; the construction and operation of FTA regimes for decision-making under conditions of uncertainty and lack of knowledge

·  From sociology, FTA as a process of co-production of stakeholder communities (social capital); FTA and the ecology of promises of science and technology; FTA as an instrument of transaction among social groups and stakeholders; symbolic and cultural uses of FTA

·  From economics, FTA as a coordination mechanism of the agents in a knowledge economy; FTA analysed with the concepts of knowledge economics, including characterisation of the flows of knowledge in FTA exercises; FTA analysed with the concepts of evolutionary economics, for example, the role of FTA in the shaping of behavioural and adaptive routines of agents

·  From management and organisation science, FTA as a collective learning mechanism through the sequential interplay between codified and tacit knowledge; development of absorptive capacity and readiness to utilise FTA processes and outputs; the organisation and location of FTA operations in relation to issues of strategy and governance; relation of FTA to the development of anticipatory, inclusive and adaptive capacity

[1] This list is far from exhaustive. With further work, the various objectives could be set out into a hierarchy.