Prove it!
Measuring the effect of neighbourhood renewal
on local people
Written by staff at the New Economics Foundation:
Perry Walker, Julie Lewis, Sanjiv Lingayah and Florian Sommer
Designed by Iris Wilkes. Illustrated by Maxine Hamil
Printed by RAP on recycled paper
ISBN: 1 899407 28 6
© Groundwork, The New Economics Foundation and Barclays PLC, June 2000
Contents
Who is behind this handbook?
Contacts
Our thanks to…
Foreword
Prove it!
Measuring the effect of neighbourhood renewal
on local people
Contents
Foreword
Who is behind this handbook?
Contacts
Our thanks to…
Foreword
1: Introduction 1
What this handbook is and who it is for 1
The contents 2
A word about language 2
2: Measuring the impact of the Barclays SiteSavers programme in Groundwork 3
Introduction 3
Groundwork at work: the Gellideg Estate, Merthyr Tydfil 3
Barclays SiteSavers 4
Measuring the impact of Barclays SiteSavers 4
3: Whether to measure 5
4: Whether to involve local people 7
What is good evaluation? 7
Local people can improve measurement 7
The effect on the project 7
The pitfalls of local involvement 8
5: What to measure 9
What are the options for measurement? 9
Measuring outcomes 9
What sort of outcomes? 10
Social capital 11
Are there other approaches? 12
6: Getting started 15
Which projects to measure? 15
How much time does it take? 16
How to build measuring into a project 16
Who to involve - the stakeholders 17
Participation in different stages of the Groundwork pilots 18
Learning from the experience of others 18
7: Agreeing issues 19
The tension between the framework and freedom of 19
choice 19
Managing the tension 21
Other frameworks 24
8: Choosing indicators 26
What are indicators? 26
What do indicators do? 26
Making good indicators – some pointers 27
Generating the indicators 29
Trusting ourselves 32
Trusting each other 33
Trust and agencies 33
9: Gathering information 34
Introduction 34
Non survey indicators 34
Surveying people’s attitudes 35
The Barclays SiteSavers Survey 35
Designing the survey 36
Who to survey? 39
When to survey? 41
How to survey? 41
Guidance for interviewers 42
Doing the interviews 43
Analysing the surveys 43
Key pointers from the first Barclays SiteSavers survey 45
Checklist 46
10: Communicating progress 47
The why, when, what, who and how of communication 47
What’s the story? 48
Letting the outside world know 49
11: Taking action 51
Using the surveys to improve human and social capital 51
Celebrate! 55
Evaluation 56
Appendix 1: Bluff your way in social capital 57
Robert Putnam 57
Jane Jacobs 57
Albert Hirschmann 57
The components of social capital 58
The example of Roseto 60
What affects people’s connections? 60
Some health warnings 61
Social capital and quality of life - the example of health 63
Appendix 2: Indicators, definitions and sources 65
Other aspects of quality of life 71
The core Barclays SiteSavers Indicators 71
Appendix 3: the Barclays SiteSavers questionnaires 71
Guidance for interviewers 71
DRAFT INTERVIEWERS SCRIPT for PROJECT SURVEY 71
References 71
Who is behind this handbook?
Groundwork
Groundwork is a leading environmental regeneration charity putting the theory behind sustainable development into practice. We work with local people, local authorities and businesses to bring about economic and social regeneration by improving the local environment.
From small community projects to major national programmes, Groundwork believes in using the environment as a way of engaging and motivating local people to improve their quality of life. Our activities reach over a third of the population and we are increasingly working with the most deprived and disadvantaged groups within society. These activities join together the themes of people, places and prosperity and range from landscaping former coal tips to cross-community youth projects in Northern Ireland.
Groundwork is a federation of 43 local Trusts based across the UK from Tyneside to South Wales, Northern Ireland to Hackney.
Specific projects include:
· Reclaiming derelict and neglected land for new community uses
· Working with young people on projects designed to reduce nuisance behaviour in high-crime neighbourhoods
· Helping small businesses improve their environmental performance through environmental ‘health-checks’ and consultancy
· Providing training and jobs for long-term unemployed people as part of the government’s New Deal strategy
· Working with schoolchildren on a range of environmental initiatives such as energy efficiency schemes and practical landscaping exercises.
Groundwork involves 80,000 adults and over 160,000 schoolchildren in sustainable development projects each year. Over 1 million schoolchildren have benefited from Groundwork’s Schools for Sustainability programmes.
Groundwork’s activities are funded by 100 local authorities, the Department of the Environment, Transport and the Regions (DETR), the Welsh Assembly, Northern Ireland Office, private sector supporters and other agencies such as the European Union and the Millennium Commission.
The New Economics Foundation (NEF)
The New Economics Foundation (NEF) was founded in 1986 and has become one of the UK’s leading research, training and policy organisations. Its mission is to put people and the environment at the centre of economic thinking.
NEF works in the UK and internationally with a wide range of partners, and at all levels from the village hall to United Nations Plaza. It has 25 full-time staff based at its London office, and 2,000 community activists and supporters around the country.
NEF is committed to developing practical methodologies for indicators that work, and promotes the use of innovative and participative new indicators in the corporate, voluntary and public sectors. We have been working with community-based indicators for the last five years. Other areas of work include community finance, local money flows, participative democracy, corporate accountability and the global economy.
Barclays PLC
The overriding objective of the Barclays community programme is to achieve real and lasting mutual benefit both to the community and to Barclays. We aim to build successful partnerships with local communities and charitable groups which produce lasting and tangible benefits.
Barclays SiteSavers recognises that the way in which the process of transforming derelict or underused land is tackled is equally as important as the end result. Involving local communities in all stages Is vital not only for the success of the project, but also in acting as a spur to tackle other pressing issues.
Our other major sponsorship schemes include Barclays New Futures, which makes £1 million available each year in cash and resources for school/community partnership projects, and Barclays Stage Partners, which helps top quality productions to tour regional theatres nation-wide.
Contacts
For more copies of this handbook, please contact Groundwork or NEF.
Groundwork UK
85-87 Cornwall Street, Birmingham B3 3BY
Tel: 0121 236 8565
Fax: 0121 236 7356
E-mail:
Website: www.groundwork.org.uk
New Economics Foundation
6-8 Cole Street, London SE1 4YH
Tel: 020 7407 7447
Fax: 020 7407 6473
E-mail:
Website: www.neweconomics.org
Prove it!
Our thanks to…
Our partners:
Katie Martin, Dawn Peace and Chris Bestwick from Groundwork UK, Lucy Mason, Head of Community Affairs, Barclays PLC, and the Groundwork officers (past and present) for the pilot projects: Chloe Aspinall, Mandy Brown, Gaye Calvin, Clare Dykes, Dave Frame, Peter Herbelet, Joanne Hendren, Val Holden, Tony Lund, Ally McEvoy, Bernie McEwan, Fiona Mather, Mick Moreton, Nigel Potter, Ruth Pryor, Emma Ramet, Steve Roberts, Fiona Scapens, Lee Turner, Paul Turrell, David Urion and Carl Wilson
Particular thanks to the old friends above who are now Proving It as far afield as Kenya and Canada…..
All the community participants who have generously donated their time both to the make the pilot projects happen, and to measure their impact. In particular we thank those who contributed to the workshop that shaped the entire project, or who welcomed touring NEF members on their visits to far-flung Groundwork Trusts:
Carol and John Ashden, Wayne Blagdon, Sarah Cox, Alex Fairweather, Michael Helene, Anne Lyons, Cath Lister, Stuart Neale, Becky Russell, Karen Smith, Linda Southam and Karen Weisberger.
A host of people whose combined talents, from publicity to proof-reading and researching to reviewing have made Prove It! come to life:
7
Prove it!
Andrea Buckley, Mark Bunting, Graham Duxbury, Martin Fodor, Simon Forrester, Stuart Hashagen, Marcus Hellquist, Markus Larson, Lindsey Lavender, Bea Malleson, Peter North, John Salkeld, Tricia Stewart, Rachel Thackray, Lucinda Roberts, Alice McQuillan, Pete Wilde, and Mandy Wilson.
7
Prove it!
The Steering Group for advice:
Alan Brown, Ian Christie, Gabriel Chanan, Tony Hams, Nina Kowalska, John Seargeant, Marilyn Taylor and Ken Webster.
And all the people who will in the future go and melt in the sun, stand in the rain or clamber up yet another flight of stairs to talk to people about things that really matter.
7
Prove it!
Foreword
Groundwork aims to make a difference to the lives of thousands of people living in some of the UK’s poorest communities.
We have set ourselves ambitious targets. Being able to measure whether or not we hit these is crucial both for use, for the communities affected by our activities and for our partners and supporters.
However, it is no longer enough to simply concentrate on measuring those things that are more easily quantifiable, like the number of trees planted, new facilities created and hectares of land improved. Attention has been switched to finding ways to measure the effects such projects have on local people, for the projects’ long term success relies on their active participation and involvement.
But how can we quantify the impact we have had on someone’s life?
Like many organisations, Groundwork has struggled to accurately capture the true worth of its work with local people. Through support from Barclays PLC, and in partnership with the New Economics Foundation, the method detailed in this handbook is our contribution to this on-going debate.
With the launch of the Government’s national strategy for neighbourhood renewal, finding a solution to this problem has become increasingly important. The strategy offers two very clear lessons that have been learned from years of regeneration practice.
Firstly, it recognises that, if we are going to improve the prospects of deprived neighbourhoods, we have to begin by improving the prospects of the people that live there.
Secondly, it states that local people should be in the driving seat, making the decisions about their own neighbourhoods.
Attempts to improve deprived areas should be judged not just on whether houses are better kept, streets are cleaner and spaces greener, but on how people living in these areas feel and act. We need to be able to measure whether their attitude has changed, whether they have more or less confidence, greater or fewer skills and whether we have helped them achieve their hopes and aspirations.
We hope the methodology we have developed, and which we will continue to develop, will provide a useful starting point for anyone wanting, like us, to make a real and lasting difference to deprived communities across the UK.
Tony Hawkhead
Chief Executive, Groundwork UK
7
Prove it!
7
Prove it!
1: Introduction
What this handbook is and who it is for
‘I know that half my advertising works, but I don’t know which half’, complained the first Lord Leverhulme, the soap baron. Neighbourhood improvement projects are similar. We know that some of them work, but we don’t know which ones. Hence the cry, ‘Prove it!’
Proof needs measurement, which can come in the form of indicators – tools that measure, simplify and communicate important issues. The temptation is always to measure what is measurable, rather than what is important. So measures tend to cover two aspects of projects. First there are physical changes: the number of houses built or trees planted. Second there are activities: the number of training places provided or the number of jobs created.
Not measured are the effects on people who have taken part in these projects and on the community of which they are part. Yet this is what is really important. It matters because improving people’s quality of life is the aim of most projects. It also matters because unless people feel that they benefit, they will not support these and future projects.
This handbook describes a method for measuring the effect of community projects on local people, on the relationships between them and on their quality of life. The method involves local people in both choosing the indicators and collecting data. It has been tested in 16 areas of the country as part of Groundwork’s Barclays SiteSavers programme. This handbook is written by the New Economics Foundation (NEF), who acted as consultants for the project.
The interest the project has simulated so far leads us to believe that there will be great demand outside Groundwork for this approach. (It is because we are eager to meet this demand that we are producing this handbook before the Barclays SiteSavers measurement is complete). We had an enthusiastic response from some 400 regeneration practitioners from around England and Wales, to whom we presented
the method in June 1999. This is what they said they liked about it:
- Being participative increases the community’s sense that it owns the measures
- The indicators validate social as opposed to physical outcomes
- Participative measurement becomes part of the project, in that it contributes to the building of trust
- Flexibility and adaptability
- Useful in educating policymakers and funders
- Will help to stimulate action.
We also asked where they thought it might be useful. Their suggestions included:
· sectors such as health and transport
· initiatives such as Local Agenda 21, New Deal, Best Value.
Other comments were that it could be used to assess:
· Partnership working
· The effectiveness of a local council
· Commercial projects.
People also thought that it could provide evidence for the need to provide revenue support to develop community capacity for an initiative before ‘squandering’ capital monies.