WCVA Research Aims and Activities

BIG VolunteeringProject

Final Evaluation Report

B E Collis

© WCVA

July 2014

Contents

Introduction

Summary of Information used

Performance measures

Equality monitoring

Targets

Quarterly reports from staff

Evaluation and learning workshops

Case Studies

Interviews with staff

The project met a local need

Developing a relationship with the volunteer

Taking action to remove barriers to volunteering

Working with other agencies

Mainstreaming new practice

Communicating volunteering to a wider audience

Spreading the learning

Lessons learned from the BIG project

Learning products

Conclusions

Outcome 1:

Outcome 2:

Outcome 3:

Attrition

The benefits of volunteering

Unexpected results

Learning from the project

Appendix 1: Case Studies process and quality

Appendix 2: Quarterly reporting – narrative questions

Appendix 3: Workshop agendas

Appendix 4: Interviews with staff

Appendix 5: Learning products

Appendix 6: Resources relevant to the project

Introduction

This report is a summary of the evaluation activity undertaken and lessons learned during a two year BIG Lottery funded project ‘Volunteering in Wales’. This was funded from April 2012 to March 2014 and involved Wales Council for Voluntary Action and the Volunteer Centres in Wales.

The aims of the project were

‘The project will enhance the ability of WCVA, County Voluntary Councils with Volunteer Centres and independent Volunteer Centres to meet the rising demand for volunteer placements and to ensure that the diverse support needs of volunteers are met.

Each local delivery organisation has chosen its own activities to undertake in order to achieve the desired outcomes of the project and each has taken account of local needs and priorities. The learning from undertaking these activities will lead to lasting improvements in whole-organisation practice throughout Wales’.[1]

The outcomes of the project were planned to be

• More people take up volunteering activity than would be the case without this project, benefitting volunteers, organisations and the community. Volunteers will benefit in ways such as growth in confidence, development of skills and enhancing feelings of self worth. The target was for an additional 2,024 people to take up a volunteering opportunity.

• Greater support ensured for volunteers with higher support needs and/or who are not currently working but ultimately aiming to take on paid work. Volunteers will benefit in ways such as growth in confidence, development of skills and enhancing feelings of self worth. Of the 2,024 total number of people taking up a volunteering opportunity, 369 were expected to have higher support needs and 617 to be unemployed.

• Throughout Wales, improvements take place in whole-organisation practice, by deliverers, as a result of learning from the project. These improvements were to be documented and shared amongst the partner organisations and then used more widely.

The evaluation of the project centred on the following actions for WCVA:

• Collate the information provided by the local delivery organisations, checking its accuracy and working with the local delivery organisations to produce a range of case studies which provide evidence for learning for all organisations involved. Five case studies to be produced as digital stories (KPI 3.2)
  
• Conduct up to 10 interviews with project staff to develop an understanding of the learning in particular local delivery organisations (KPI 3.2 and 3.3).
  
• Lead discussions with the staff regarding the outcomes of the programme, the learning which has emerged and how it can be implemented (KPI 3.1 and 3.3)
  
• Disseminate through e-bulletin, WCVA networks, website, newsletters (Network Wales, WCVA e-briefing, Volunteering News) and meetings (eg WCVA/CVCs/VCs meetings, WCVA Volunteering conference, UK Volunteering Forum, Voluntary Sector Studies Network) any learning through evidence based case studies and experience gained (KPI 3.3)
  
• Report to BIG how projects met their outcomes and how they adapted to changes and learnt from the programme (KPI 3.3).

The approach of the whole project was based on participative action research, where a group is formed that has lived experience which can be reflected upon with the aim of bringing about improvements or solving problems. The staff in the Local Delivery Organisations were viewed as the group, and the evaluation and learning meetings were specifically aimed to facilitate the staff to share experiences, work together to seek solutions and develop a sense of group identity. The role of the evaluator in this is different from other situations. The evaluator is a critical friend, setting cultural norms and steering the group in a positive way through the action research cycle (Figure 1). The evaluator was also involved in developing the learning products, and facilitated at several events where the learning from the project was tested.

This makes the evaluator’s role more complex than is usually the case. This report is written from the perspective of one who has been involved with the project and is therefore reflective rather than summative.

As part of the project we also searched out advice and resources that complemented the learning that we were undertaking. These resources were included in some of the learning products and some reports are listed in Appendix 5.

The following sections detail the evidence sources that have been used, with a section for each information source, along with a critique regarding the quality of the evidence and whether the evidence contributed to the aims of the evaluation and the project as a whole.

Figure 1: The Action Research Cycle

Summary of Information used

The following evidence has been gathered during the project:

• Case studies of participants who have volunteered
  There were 287 case studies collected during the project. All were examined for learning about the core themes that emerged during the project. The guidelines and forms used are in Appendix 1.
  
• Quarterly reports from all volunteer centres
  These included the numerical data and reflective questions that sought to identify the learning that was occurring in the volunteer centres. The information was collected using an on line survey tool. The generic questions are in Appendix 2.
  
• Agendas, presentations and notes from evaluation and learning workshops
  There were six workshops which acted as a way to train staff in the core skills of action research, record and reflect on learning, identify the emerging themes, share good practice, discuss areas for further investigation, test out learning products and report back on performance and achievements. Information on these are in Appendix 3.
  
• Interviews with staff during January 2014
  A purposeful selection of staff employed on the project and their line managers was made that resulted in seven interviews. These were recorded, transcribed and analysed thematically. The questions used in the interviews are in Appendix 4

• Learning products
  There were three digital stories, three newsletters and nine information sheets, branded as ‘Top Tips’, produced. The digital stories and information sheets were tested out in the evaluation and learning workshops and at the WCVA Volunteering Conference in September 2013. The transcripts of the digital stories and the information leaflets are in Appendix 5
  

Performance measures

Equality monitoring

The equality statistics from the final report are presented below, alongside the statistics for Wales. Generally, the percentages are similar, showing the project was accessing the main areas of Welsh society. Particular exceptions are:

• A higher percentage of people reporting a disability than working age adults reporting a limiting long term illness in 2011. This is in line with the target group of the project.
• Fewer people aged 65+. This is to be expected where volunteering is seen as working for free.
• More people report no religion in the BIG project than in the Census – this may be due to filling out a form with another person.
• Fewer people reported that they spoke Welsh than in the Census (10.5% compared to 19%). This may be due to the perception that volunteering is a ‘English’ activity (Welsh people just help out in their communities) or may be due to answering the question with the volunteer advisor present (if you are unsure of your language ability and afraid that someone will start treating you like a native speaker, you may say no)

a) Gender%

Male 48%(49.1% all persons, 2011 Census)

Female52%(50.9)

TOTAL 100%

b) Disability%

Disabled23% (17% LLTI 2011 Census, working age pop)

Not disabled77%

TOTAL 100%

d) Ethnic background%

White94.5%(95.6% 2011 Census)

Mixed ethnic background1.5%(1.0%)

Asian/Asian UK1.5%(2.2%)

Black UK1.5%(0.6%)

Other ethnic group1.0%(0.5%)

TOTAL `100%

e)Age

0 – 24 years34.5%(30.4% 2011 Census)

25 – 64 years63.0%(51.3%)

65 + years2.5%(18.4%)

TOTAL100%

f)Religion or belief%

No religion60.5%(32.1% 2011 Census)

Christian28.8%(57.6%)

Buddhist0.82% (0.3%)

Hindu0.0%(0.3%)

Jewish0.1%(0.1%)

Muslim1.4%(1.5%)

Sikh0.0%(0.1%)

Other religion8.4%(0.4%)

TOTAL 100%

g)Sexual orientation%

Heterosexual74.3%97%(98.3% Stats for Wales Bulletin SB 104/2010R)

Gay men 0.6%0.8%(0.5%)

Gay woman0.7%0.9%(0.3%)

Bisexual0.7%0.9%(0.4%)

Other0.4%0.5%(0.4%)

Prefer not to say23.3%

TOTAL 100%

(Note these have been recalculated due to the large number of ‘prefer not to say’ responses to enable a comparison to be made with the national statistics)

h)Welsh language %

People who speak Welsh10.5%(19% 2011 Census)

People who do not speak Welsh89.5%(81%)

TOTAL 100%

Targets

The final grant report stated that the project had met its targets in that 2,048 people had taken up a volunteering opportunity (102% of the target). Of these 1,196 were reported as having additional support needs and 1,662 as unemployed. These two groups made up a higher proportion of the total number of volunteers placed than was expected.

There was a variation across Wales of volunteer centres meeting their targets in the local area. This ranges from 40% to 226%. There are a number of explanations for this:

• Some of this variation may be related to the targets that were set in some areas at the beginning of the project (some targets set were much higher in some areas than others with the same resource).
• Some may be due to the level of support required by volunteers with different needs. In some areas where the needs of volunteers who were referred to the project were complex, delivery partners were unable to work with the target number of volunteers as their capacity to support the target number of volunteers became limited due to the level of support required.
• There were also instances of difficulties in terms of lack of suitable opportunities in some areas. Many delivery partners worked hard to secure new opportunities with volunteer-involving organisations which was resource intensive.
• The variation did not appear to be due to lack of people being referred to the project or people accessing the project through usual means (contacting the volunteer centre directly of using the Volunteering Wales website).

The number of people interviewed during the project was also recorded. Overall 4,771 people were interviewed. This gave an overall conversion rate of 43 percent. The range in different areas was 23 percent to 85 percent, with most being between 30 and 60 percent. There is a problem with the conversion rate as calculated. It includes those who were interviewed in a period but not placed in that period. In general staff reported it taking 6-12 weeks to place a volunteer. Thus those interviewed in the last quarter of the project are unlikely to all have been able to be placed. This depresses the figure.

Two areas that had high conversion rates had different approaches. One used a group volunteering model, working with referral agencies to recruit a cohort of potential volunteers that then volunteered as a group with the referral organisation’s support. These groups went through a varied programme of volunteering and completed some training which resulted in an OCN qualification. The cohort was then moved onto other opportunities, some of which were created by the volunteer centre.. Another volunteer centre had associated services, such as a community cafe or a community support group who individuals could be referred to directly within the organisation. This led to a better brokerage service and the volunteering staff being more confident that the placement would succeed.

Looking at the variation in conversion rate compared to the number of potential volunteers being interviewed, it seems that the best conversion rates are achieved by areas that supported between 4 and 6 new people per month. If numbers interviewed are much higher, then the lack of capacity to provide the necessary support reduces the conversion rate. This is not just a matter of inefficiency on behalf of the project, as many of the potential volunteers will have added volunteering to the list of things they cannot do, and their self confidence and self esteem will be even lower.

Many potential volunteers were interviewed by volunteer centres but were assessed as not being yet ready for volunteering. Many individuals were referred to other support agencies e.g. drug and alcohol agencies, mental health services and domestic violence support services.

Several volunteers also returned to the volunteer centre when their placement had come to an end or they wanted to try different activities. This is not recorded in the monitoring and volunteers who are placed more than once with various agencies are not included in the numbers of volunteers placed. This reflects two aspects of the approach of the project:

• The relationship that is built up between the potential volunteer and the local staff is often far deeper than is normally possible for volunteer staff. This means that when things go wrong, the person is more likely to return to the volunteer centre for further help, rather than just give up.
• The risk of placing someone with additional support needs means that more placements are likely to break down and hence the support provided has to cope with this.
• This additional load varies between people and is not predictable. This makes planning based on targets and a finite resource much more difficult.

The profile of the interviews and placements over the whole project (Figure 2) shows that a quasi steady state was achieved with approximately 500-700 interviews taking place in a quarter with approximately 250-350 placements. The conversion rate reported for the project is shown to be about 10 percent lower than the steady state conversion rate of about 50 percent.

One volunteer centre provided information on the reasons for the loss of volunteers between interview and placement. This is summarised in Figure 3. This shows that 80 percent of contacts were interviewed and 44 percent of contacts were placed. What is of interest is that the reasons for some of the attrition are good with 6 percent having found paid employment and 7 percent still waiting for a placement.

Figure 2: Numbers of Interviews and Placements carried out in each quarter.

Figure 3: Attrition

106 / Contacted /
/ 21 / 16 / disengaged

/ 20% / 2 / found paid employment
1 / moved out of area
2 / personal or family illness
85 / Interviewed / 31 / 19 / disengaged

/ 29% / 4 / found paid employment
3 / moved out of area
2 / not ready to volunteer
3 / personal illness of substance misuse relapse
54 / Placed/waiting for placement / 7 / 7 / not placed yet

/ 7%
47 / Placed

Quarterly reports from staff

Quarterly reports were received from all local delivery organisations during the project and these contained comments on the approaches that had been tried and what had been learnt. These were reviewed by WCVA and used to inform the evaluation and learning meetings and the learning products.

Some example entries are shown below. They show the general type of comments that were captured. The project encouraged reflective practice so that staff were able to put learning into practice and try new things. For example:

I am discovering that the potential volunteers that I accompany for their initial visit to organisations are more likely to take up volunteering. The organisations tend to be more accommodating to the potential volunteer also.NPTCVS Q6

However, some have shown continued barriers in communication with volunteers:

Some volunteers are happy to go on the volunteering placement themselves, but some of them don’t get back to me after, or don’t reply when I phone them. This can be difficult.Mantell Gwynedd Q2

Although some volunteers are confident enough to go after the volunteering placement themselves, some of them don’t get back to me afterwards, and don’t respond when I phone them either.Mantell Gwynedd Q7

Texting is very useful to remind the volunteers of appointments and keep in regular touch with them. In this quarter have come across a volunteer who has no signal at home, it has been impossible to get an update on her situation in terms of volunteering.Mantell Gwynedd Q7

The reports were also used to give quite detailed background information by some staff, eg:

Have seen four volunteers with mental health problems

One person with problems leaving the house – she’s eager to improve and thought that volunteering would be beneficial to her. Have accompanied her to her volunteering placement and also helped her complete the organisation’s application form. Unfortunately have not been able to progress with the volunteering placement since her condition has deteriorated again.

Another volunteer has overcome depression and is dealing with his mental health condition, he sees volunteering as a way of getting out of the house and meeting new people. He’s eager to volunteer and his details have been passed on to an organisation that runs a befriending scheme.

Another volunteer I saw in mid December is hoping to be ready to volunteer following a period of depression and years of suffering domestic abuse. Her details have been passed on to an organisation before Christmas. Hopefully she’s still be as eager to volunteer in the new year.