TAFTIE Statement on ERA Instruments (Draft)

The European Network of Innovation Agencies
Proposal for a Task Force

on Benchmarking the Agencies’

approaches to demonstrate Effectivenessand Efficiency

1. Need for benchmarking

Especially in times of economic problems and governments budget cuts, there is a strong need for innovation agencies to demonstrate their added value to society. The added value can manifest itself on different levels: First there is the need to show that the programs and policy instruments that we execute have an impact on the economy. Secondly we have to show that the agencies´activities themselves “make a difference” (agency effectiveness) and that this is done in an efficient way (agency efficiency).

The TAFTIE TF on additionality / smart agencyallocated impact, effectiveness and efficiency along the Effect Dimension, the Process dimension and the administrative dimension (2007).

As potentials sources of value adding outcomes of agencies activities the TF on additionality / smart agency focused on typical processes of funding agencies:

In april 2012 an interesting Taftie Academy Workshop on Performance Indicators of Funding Agencies was hosted by TEKES. There were lively discussions on evaluation techniques, additionality, benchmarking systems, balanced score cards, etcetera. In particular, considering e.g. learning effects within the agencies was discussed. How do we capture competence building in the agencies? What is the need for and the effect of specific competencies of agency staff?

The main conclusion of the workshop was that there is a lot of interest in these subjects by TAFTIE-members and that we should enhance these discussions in a structured way, for instance in a TAFTIE Task Force, Ad Hoc Group or Network.

In the TAFTIE Board Meeting in June 2012 there was a discussion on allocating the budgetreserves. There was “… a general support of additional benchmarking and research into topics of interest to a majority of members. Members stressed however that this is best done in the context of a task force. Taftie should manage the process, rather than just buying a report. A task force also guarantees the input by members.”

In this paper a new taskforce on benchmarking the Agencies’ effectiviness and efficiency is being proposed.

1. Scope of the Task Force

The first objective of the taskforce is an inventory and comparison of different methods of measuring agencies’ performances. Secondly the task forcewill try to develop a benchmarking system by which the agencies can compare performances in terms of efficiency and effectiveness. The outcome of this wil not be a ranking (‘top 40’) of agencies from ‘best’ to ‘worst’ but a method to compare different aspects of agencies’ performances and learn from best practices on these different aspects.

The Task Force focusses on three main subjects:

Effectiveness: inventarisation and comparison of evaluation techniques (impacts of funding programmes and agency effectiveness)

Efficiency: inventarisation and comparison of performance indicators

Comparison of possible dashboard and balanced-score card techniques

The new task force will build on the findings of the TAFTIE TF on Additionality (“smart agency”, to avoid duplicating work that was already done.

1. Working packages

1. Scoping and definition

The subject the Task Force is working on highly complex. There are a lot of different indicators, definitions and evaluation techniques being used by the agencies. The first step therefore will be to get a common feeling on the subject.

1. Commisioning a consultant

Hiring a consultant to identify and inventory the different indicators that are being used by different agencies possibly is a good idea. This needs a clear definition of the task and mandate for a consultant.

1. Implementing recommendations and constructing benchmark system

1. Reporting to the Board

1. Time table

september: discussion in EWG

oct-nov: formation of Task Force

dec: decision on TF in Board

jan: 1st meeting (scope, definitions, working packages)

feb-march: preparing procurement

april: start inventarisation by consultant

may: 2nd meeting TF

june: status-report to Board

aug: 3rd meeting: consultant-report

….

1. Resources

10 participants: time for research and preparing meetings

-5 meetings of TF-members: travel expenses for each member

-contract for consultant: EUR 50.000

1. Participants

-

-.

-.