Evaluation of Exodus
(Action 2)

Final Report

by

Professor David J Parsons

and

Simon Bysshe

©HOST, 2008 February 2008

HOST Policy Research

PO Box 144, Horsham, West Sussex, RH12 1YS

Tel: 01403 211440 Fax: 01403 251866

Email:

Website:

HOST Policy Research

Established in 1986, HOST has built an international reputation for bringing a fresh approach to market intelligence and policy research in education, employment, training and related studies. HOST has also built up a strong track record across the public sector for practically focused evaluation studies.

Our goal is to ‘…support policy, good practice and innovation in employment, workforce development and related business issues through practical leading edge research and related activities’. Recent client organisations have included:

  • Government departments, including DIUS (formerly DfES), OME, DfBERR, (formerly DTI), DWP, Home Office, and various other directly managed national and regional agencies, including a number of RDAs.
  • The Quality Improvement Agency, Learning and Skills Council National Office, and a range of local LSCs. HOST has framework agreements for research and evaluation with the LSC National Office (in three areas: good practice, operational evaluation and strategic evaluation), and with QIA.
  • Sector Skills Councils (SSCs), and Standard Setting Bodies (SSBs) - HOST has a research framework agreement with the Energy and Utilities SSC.
  • The Training and Development Agency for Schools, QCA and SQA, and other education development and quality assurance bodies in the public sector.
  • The European Commission, and Cedefop.
  • Non-governmental and voluntary organisations.

Full details of HOST clients are available on HOST’s website. HOST’s work across these is widely recognised as practical as well as authoritative. Our work programme has included numerous sector and cross-sector studies to inform policy developments, and enhance participation and the delivery of education and employment programmes.

Contents

Page

Chapter 1: Introduction

1.1The evaluation 1

1.2Action 2 and the evaluation 1

1.3The objectives of the evaluation 2

1.4Approach and focus 3

1.5The report 5

Chapter 2: Partnership in Action 2

2.1Introduction 6

2.2Origins and evolution 6

2.3The infrastructure of Action 2 9

2.4The Exodus Development Partnerships 18

2.5The profile and positioning of Exodus 23

2.6Integration of the trans-national dimension 25

2.7Overview 26

Chapter 3: Implementation and Effectiveness

3.1Introduction 29

3.2Implementation of the development projects 29

3.3Backgrounds of beneficiaries 34

3.4Outputs and performance gaps against targets 38

3.5Implementation and equality of opportunity 41

3.6Empowerment and integrating the client group 44

3.7Contribution of the trans-national dimension 45

3.8Overview 46

Chapter 4: Innovation and Added Value

4.1Introduction 47

4.2Cross-regional innovation and added value 47

4.3The quality of innovation in Development Partnerships 50

4.4The added organisation value of activity in DPs 53

4.5The added value of activity in DPs to individuals 55

4.6The added value of the trans-national dimension 57

4.7Constraints to adding value and innovation 57

4.8Overview 64

Continued…/

Contents (continued)

Page

Chapter 5: Impact of Action 2

5.1Introduction 66

5.2Outcomes evidence in context 66

5.3Beneficiary outcomes: economic activity 67

5.4Beneficiary outcomes: qualifications 70

5.5Beneficiaries and intermediary outcomes 72

5.6Exodus impact on re-offending 74

5.7Overview 75

Chapter 6: Mainstreaming and Sustainability

6.1 Introduction 78

6.2 Potential for mainstreaming in Action 2 78

6.3 Review of progress in achieving mainstreaming 81

6.4 Mainstreaming success factors and constraints 84

6.5 Overall sustainability within Action 2 87

6.6 Overview 90

Chapter 7: Issues and Implications

7.1Introduction 91

7.2Achievements of Action 2 91

7.3Service delivery and implications for future practice 95

7.4The quality of innovation 100

7.5Sustainability and transferability 102

7.6Implications and lessons learnt 103

Annexes:

Annex A: The evaluation methodology

Annex B: Summary of local EPs – their partnerships and focus – and projects funded

Through extension

Annex C: Selected tabulations: Exodus delivery and beneficiary distribution

Annex D: Selected tabulations: Exodus outputs and outcomes

Annex E: Exodus in Context: A Comparative Study

Final ReportConfidential

Exodus - Evaluation of Action 2

Chapter 1: Introduction

1.1The evaluation

Exodus was conceived as an ambitious, broadly-based and bi-regional programme to help tackle the long-standing issue of persistently high levels of re-offending and recidivism amongst offenders. Funded through the European Social Fund (ESF EQUAL), it sought an independent evaluation to help shape its progress and effectiveness in delivery. In March 2006, the South East England Development Agency (SEEDA), as lead partner in Exodus, commissioned HOST Policy Research (HOST) to conduct the external evaluation of the programme. The focus has been on Action 2 of the ESF-EQUALfunded programme, with a broadly-based review of constituent activities across the two participating regions: London and the South East.

This report brings together the findings as the programme draws to a conclusion for the proposed funding period for Action 2. It follows three previous interim reports and takes account of the significant extension to these activities to cover the second half of 2007. The report is presented to the Exodus Research and Evaluation Working Group (R and E Working Group)[1]and precedes any wider distribution of these findings to other partners.

1.2Action 2 and the evaluation

The central goal of Exodus has been to promote social inclusion through reducing recidivism and by facilitating offender rehabilitation through employment. Its particular focus has been among short-term prisoners (STPs) as well as ‘persistent and prolific and other priority offenders’ (PPOs). Its focus and working model has been geared to the funding requirements of ESF-EQUAL[2] which sets out to help: ‘…people who are in difficulty in being integrated or re-integrated into a labour market that must be open to all’.

Action 2 of Exodus followed the early development and implementation of the Development Partnership Agreement (DPA) in Action 1.[3] It has operated through a unique and evolving multi-agency central Development Partnership brought together in the (then) Management Group (now Strategy Group), and linking both London and the South East through a series of sub-regional activities delivered through 13 initially funded individualLocal Development Partnerships (DPs). Two other ‘agreed’ DPs did not commence funded activity in the first year of Action 2 and were replaced with two smaller-scale development projects.

The cross-project Management Group/Strategy Group[4] was also anticipated to: ‘…manage and match fund projects’ and as such, the central partners were seen as at the heart of managing the DPA, and shaping and focusing Exodus. Action 2 was in turn central to the transfer of learning of these projects through Action 3.[5]

Exodus has consequently been a very complex programme, involving various tiers of activity, a range of mainly sub-regional innovations through various DPs, and developing (and changing) management processes across these tiers. The evaluation was expected to be a central tool for partners to review what had worked well (and what had not) and why and to consider the effectiveness ofthe transferability of this into mainstream actions.

1.3The objectives of the evaluation

The evaluation’s aims and objectives remain those as set out by SEEDA in the original Invitation to Tender (ITT). This required a multi-level and focused external evaluation which was expected to complement the arrangements put in place for internal evaluation and monitoring actions within the programme. The overall goal of the external evaluation was to provide an independent and cross-cutting assessment of the extent to which Exodus meets the established goals for the programme. More specific objectives were to assess:

i)The quality of innovation and added value of the work ofExodus.

ii)The potential for mainstreaming the work of Exodus to sustain effective actions.

iii)The achievement of mainstreaming the work of Exodus.

iv)The level of empowerment of beneficiaries served by Exodus.

v)The achievement of equality of opportunity by Exodus.

vi)The publicity and awareness raising achieved by Exodus.

These were cross-cutting objectives, affecting all tiers of Exodus activity: funded sub-regional development activity within the two participating regions; regional and sub-regional integration; and trans-national review.

The evaluation was also set against the regional and national policy context which has been evolving as Exodus has itself developed. This has proven to be shifting ground for Exodus. The early expectation was that Exodus would help to inform Custody Plus, and in particular the thinking and practice of the (then) newly developed National Offender Management Service (NOMS) to support offenders. Early in Action 2, this was affected by an initial delay in launching Custody Plus, as well as subsequent reviews and revisions to policy. At the same time, it has taken place against a background of important changes to the national and regional infrastructure for implementing any such programmes (and also the funding routes).

The evaluation consequently needed to combine quantitative and qualitative inputs across a diverse range of Exodus activity. At the same time, it needed to adopt methods of evidence gathering and review which would be responsive to the changing national policy context and picture.

1.4Approach and focus

The evaluation anticipated a multi-layered series of activities, staged assessment and progressive reporting to partners. Figure 1 summarises how the evaluation model planned to integrate evidence gathering across the different facets of Exodus activity.

Figure 1: Evaluation framework for Action 2

Six specific, but interrelated, stages of evaluation activity were planned. These involved:

  • Stage 1: Intensive project planning, inception and secondary research to shape and implement an Evaluation Action Plan(s), combined with staged feedback to take account of changes in Exodus activity over 2006-2007.
  • Stage 2: Design and development, with an initial phase of developing and refining evaluation tools, taking into account in-project monitoring data[6]and internal evaluation, and adjusting these to changing project circumstances.
  • Stage 3: A progressive and two-tier review of local DP working models, progress, processes and achievements against goals and programme expectations. This combined in-depth fieldwork on five of these projects (Blue Sky, Action Acton, St Giles- Holloway, Southampton - City Limits and Project Spring - HMP Springhill) with light touch assessment (and an end-of-Action 2 review) with all DPs. The in-depth reviews were selected from the early evaluation activity and also from a cross-DP review of those most relevant to future mainstreaming. This stage of the evaluation has also includedthree cross-DP review events, organised as stakeholder workshops to disseminate, feedback on, and critically review, evaluation findings.
  • Stage 4: An initial, interim and end-of-Action 2 review of the regional and regional-national dimensions of the programme, including partnership working, policy links and integration, and the quality of early mainstreaming.
  • Stage 5: A small-scale review of the trans-national dimension of the project, and in particular its impact at local level, specifically looking at local and sub-regional level engagement in these processes set against wider EQUAL goals.
  • Stage 6: Collation and reporting of the Stages 3 to 6 evidence into a series of staged reports.

The evaluation has been guided by an initially agreed Evaluation Action Plan (April 2006), subsequently remodelled to reflect changes to the programme (and evaluation) in summer 2006 (September 2006) and extended to ensure the evaluation took account of activity in the significant extension (July 2007 to December 2007).

The scope of the methodology within each of these stages has changed during the course of the evaluation. In particular, there was substantial remodelling of the evaluation inputs in summer 2006, and some delay to fieldwork. This was to provide for a more streamlined evaluation within a substantially reduced evaluation budget consequent on the consolidation of the programme in June-July 2006. The changes did not affect the overall objectives and proposed delivery of the evaluation, but did see a shift from a formative evaluation (with a summative output), to a summative evaluation with limited scope for sharing emerging findings directly with partners. Full details of the evaluation approach, changes, and an assessment of its effectiveness are set out at Annex A.

Delivery of the evaluation has proceeded very much to the revised schedule anticipated in the summer 2006 Action Plan. In addition to this draft final report, it has provided: a pre-interim report (May 2006); three further summaries of progress and findings (November 2006; March 2007 and November 2007); two staged interim reports of emerging findings and development issues for stakeholders, partners and practitioners in DPs (November 2006 and July 2007). A separate report of summary case studies has also been made available to SEEDA and partners.[7]

1.5The report

Exodus has been a highly ambitious programme rooted in the ESF-EQUAL spirit of searching for innovative solutions to very real issues of disadvantage. An important focus within the evaluation - crossing the more specific objectives - has been to provide an independent assessment of how much has been achieved against these stretching goals. This report draws that together.

The evaluation shows that Exodus has not been able to succeed in all these respects. While the report is able to identify and celebrate much of what has been achieved, parts of the evaluation nonetheless may make for difficult reading by some of those who have worked hard to see Exodus succeed. HOST hopes this constructively critical approach will help partners to develop a shared understanding of the wider messages to help future practice which may be better placed to build on these foundations. To do this, the draft final report is set out in seven chapters which, following this introduction (Chapter 1), look at:

  • Progress and Partnership in Action 2: Looking at the infrastructure of Action 2, its origins, the expected and achieved collaborations, and integration of the trans-national dimension (Chapter 2).
  • Implementation and effectiveness: Reviewing some of the quantitative achievements of the programme, as well asimplementation and effectiveness, performance gaps against ‘targets’, and also empowerment of the client group and the contribution of the trans-national dimension (Chapter 3).
  • Innovation and added value: Assessing for this key feature of Action 2 the extent and quality of innovation and added value, and constraints to innovation (Chapter 4).
  • Impact of Action 2: Drawing together the evidence that is available on Action 2 outcomes, and observations from the evaluation case studies, as well as for constraints to achieving impact (Chapter 5).
  • Mainstreaming and sustainability: Looking at both the potential, and achievement of, mainstreaming in Action 2, as well as mainstreaming success factors and constraints and overall sustainability (Chapter 6).
  • Issues and implications: Drawing together the findings and conclusions from HOST’s assessment of the findings from Chapters 2 to 6, to look at the lessons emerging, transferability and the added value of Action 2 (Chapter 7).

The main text is supported by five annexes which provide: the evaluation methodology, including a critique of approach (Annex A); a summary of the local DPs, their partnerships and focus (Annex B); selected statistical tabulations on Exodus delivery and beneficiary distribution (Annex C); selected statistical tabulations on Exodus outputs and outcomes (Annex D) and a comparative review looking at the outputs and outcomes of Exodus in the context of programmes with similar aims and target groups (Annex E).

A particular feature of this assessment is that it draws on feedback from staged and co-operative reviews of findings and issues arising throughout the evaluation. It is nonetheless an external review by HOST, informed by wider input, but which remains a completely independent interpretation and assessment.

Chapter 2: Partnership in Action 2

2.1Introduction

Action 2 anticipated high levels of collaboration across all tiers of Exodus activity. A starting point for the evaluation is an exploration of how these arrangements evolved, how they worked, and their overall effectiveness. To do this, the evaluation draws on: the baseline review, set out in the pre-interim report, HOST’s later review of the effectiveness of the restructured arrangements (ie after summer 2006), and the most recent evidence collected from stakeholders and most of the DPs,[8] at the end of Action 2. This chapter looks at:

  • The origins of Exodus, the evolution of the cross-regional partnership and quality of partner engagement.
  • The ‘central’ infrastructure put in place for shaping, managing and steering Action 2, the organisation of central roles and responsibilities, and theireffectiveness.
  • The sub-regional and delivery collaborations put in place for Action 2 across the ESF-funded (local) DPs.
  • The achieved project-level profile and positioning of Exodus, including publicity and awareness raising.
  • The origins of the sub-regional partnerships and how well their formation and collaboration has served Action 2.
  • The scope and integration of the trans-national dimension into Exodus.

The review sets the scene for the analysis of specific achievements across Action 2, the constituent working arrangements, innovation, impact and mainstreaming that follow in Chapters 3 to 7.

2.2Origins and evolution

The transition from Action 1 to Action 2 within Exodus seems to have been effective in establishing the necessary foundations. This included forgingdiverse links, and engaging many of the statutory (and other) agencies with policy, implementation or funding interest in offender resettlement in both regions. The available evidence suggests that this formative stage was timely and conducted intensively.

In parallel, the groundwork in Action 1 also provided for a fast start to inviting project bids, selecting projects, and subsequently to speedily kick-starting over a half of those. It is less clear if this could be seen as an open process, with some evidence of voluntary sector and other delivery agencies with established links with statutory agencies being better placed to engage in the streamlined bidding arrangements. Early in this process, partners were also able to attend a high profile launch conference to set out the goals and scope of Action 2. This included a shop window for Exodus activity with some profiles of the early activity then under way.

On this evidence,[9] by autumn 2005, Action 2 had secured the fast start and breadth of regional embedding for which it had aimed. Yet very shortly after this start-up phase, Exodus central delivery and direction was in difficulty. At the same time, latent, but very serious issues of matched funding distribution and levels were apparent. As a result, some ten months into Action 2, the scope and funding of the whole programme was in ‘crisis’.[10]