[LIBSS1]
Evaluation of Round 5 of the Adult and Community Learning Fund
Sue McMeeking
Monica Taylor
Robat Powell
David Sims
Basic Skills Agency July 2002
CONTENTS
Acknowledgements
EXECUTIVE SUMMARY......
1.INTRODUCTION......
1.1Background......
1.2Evaluation Aims......
1.3Methodology......
1.4Structure of the Report......
2.THE ENGAGEMENT AND MOTIVATION OF LEARNERS......
2.1Marketing the Programmes......
2.2Recruitment and Retention......
2.3Programme Features......
2.4Support Mechanisms......
2.5Key Messages......
3.IMPACT ON LEARNERS......
3.1Introduction......
3.2Impact-related Issues......
3.3Personal, Social and Learning Development......
3.4Basic Skills Development......
3.5Barriers to Learning and Impact of the Programmes......
3.6Progression......
3.7Key Messages......
4.IMPACT ON ORGANISATIONAL CAPACITY......
4.1Introduction......
4.2Impact......
4.3Sustainability and Future Development......
4.4Key Messages......
5.CONCLUSIONS......
5.1Main Findings......
5.2Challenges in Evaluating Impact and Outcomes......
5.3Implications for Future Development......
ACKNOWLEDGEMENTS
The authors would like to express their thanks to the staff at the Basic Skills Agency for their advice and support throughout the project. In particular, Miriam Sampson and Annabel Hemdstet provided valuable guidance and assistance. We also acknowledge the contribution made by the other members of the project steering group: Jan Eldred of the National Institute of Adult Continuing Education, and Jonathon Webster and Andrew Macdonald of the Department for Education and Skills.
Special thanks are due to the ACLF programme staff and learners who gave so generously of their time to participate in research interviews, providing essential information and data for the project. We are grateful to the following organisations which agreed to participate in the case studies undertaken:
The Big Issue Foundation
Christchurch, Hackenthorpe
Friends of St Nicholas Fields – Urban NaturePark
Platform – Boscombe
Gwellheans Recovery
Look Ahead Housing and Care Ltd
Midland Vietnamese Community Association
The Society of the Blind – Dewsbury, Batley and District
The Social Partnership
We thank our colleague Emma Scott in the Statistics, Research and Analysis Group who helped with analysing the ACLF statistical reports. Finally, we acknowledge Maureen Greenaway, the project administrator, for the administrative assistance she provided.
1
EXECUTIVE SUMMARY
Introduction
The purpose of the Adult and Community Learning Fund (ACLF), which was launched by the government in 1998, was to explore innovate ways to expand the provision of, and increase adults’ access to, local community-based learning opportunities. The ACLF aims to build the capacity of community-based organisations to provide learning opportunities, to support the development of partnerships to deliver learning at the local level and to share good practice.
The ACLF aims to draw more people unaccustomed to education, some of whom are hard to reach and at risk of social exclusion, into learning activities in order that they can develop the self-confidence, knowledge and skills needed to realise their potential and become effective citizens. Furthermore, the ACLF aims to improve the basic skills (literacy and numeracy) of adults who have difficulties with them. It should make a contribution to Skills for Life, the national strategy for improving adults’ basic skills which has the target of reducing the number of adults who lack adequate functional literacy and numeracy skills by 750,000 by 2004.
The ACLF is managed by the Basic Skills Agency and the National Institute for Adult and Continuing Education. The Basic Skills Agency commissioned the National Foundation for Educational Research (NFER) to undertake an evaluation of the Round 5 ACLF programmes (59) which it managed. This summary presents the key findings from the evaluation which was carried out between October 2001 and May 2002.
Key Findings
The most successful method for marketing the programmes was word of mouth. The credibility of the marketing message was strengthened where learners joined programme staff in outreach work in the community.
The main strategies used to remove barriers to recruitment and retention included paying the cost of learners’ travel to the programmes, providing learning materials, and in some cases, making provision for childcare.
The learners considered that the most consistent and most dependable form of support was that provided by programme managers, coordinators and tutors. Learners on some programmes provided useful support for each other.
The greatest impact of the programmes was on learners’ personal development, including their self-confidence, self-esteem, motivation, and ability to think for themselves. Learners’ progress in developing and using new skills was dependent on their confidence and emerging identity as learners.
The ACLF programmes had helped to improve learners’ literacy and numeracy. Some had also acquired IT skills, practical skills and coping skills.
The ACLF had helped to increase the capacity of participating organisations to deliver basic skills provision by raising staff awareness and by providing resources for training, for buying teaching expertise, and for purchasing learning materials.
Background
In 1998 the government provided £20 million to fund the ACLF over three and a half years. Subsequently, the ACLF was extended to the end of March 2004 with an additional £5 million being allocated for 2002-2003. The Basic Skills Agency and the National Institute for Adult and Continuing Education allocate the funds to community-based organisations through a bidding system. Unlike earlier bidding rounds where organisations could apply to run programmes for two years or two and a half years, Round 5 funded programmes for one year between April 2001 and March 2002 when the ACLF was planned to end. The relatively limited timescale of Round 5 should be taken into account when interpreting the evidence from the evaluation of the 59 programmes managed by the Basic Skills Agency.
Evaluation Aims
The aims of the evaluation were to:
identify the key features of programmes that are successful in engaging and motivating learners to improve their basic skills
assess the programmes’ effectiveness in increasing participants’ basic skills and other personal and community-related gains
assess whether programmes increased their capacity to identify, promote and support basic skills and what features contributed to their success.
Methodology
The evaluation used a qualitative research methodology to examine in depth the working and outcomes of the programmes supported by the ACLF. NFER carried out case studies of nine programmes located in six regions which targeted learners from different groups, including homeless people and rough sleepers, young single mothers, people recovering from drug misuse, visually impaired people, refugees, and ethnic minority groups. Three of the programmes were visited twice to investigate developments over time.
Interviews were conducted with 13 programme managers and coordinators, 11 tutors and 37 learners. Six of the learners and five programme managers were interviewed twice.
The case-study data was supplemented by an analysis of end-of-programme reports: 15 Vital Statistics Reports, 16 Final Statistics Reports and 24 Final Narrative Reports.
The Engagement and Motivation of Learners
Client Groups
The case-study programmes’ client groups shared several characteristics. Their lack of competence in basic skills often formed part of much broader personal and social needs. These were related to their marginalisation in society and their attendant vulnerability. Clients were mainly unemployed, with uncertain health, housing and financial backgrounds. They were living fragile lives with varying degrees of support from family and friends. Few of the clients had achieved success in their previous educational experiences which had often been so negative as to discourage them from seeking further learning opportunities. Clients usually had few if any aspirations or ambitions for the future when they joined the projects.
Programme Marketing
Attracting hard-to-reach adults in the target groups to join the programmes and improve their literacy and numeracy was a major challenge. A range of methods were used to market the programmes, including placing leaflets, brochures and posters in community locations such as doctors’ surgeries, social services offices, jobcentres, and supermarkets. Open days and taster days were also used to market provision. The most effective method was personal contact through word of mouth, particularly through a past or present learner telling friends or family about the opportunities available.
An effective outreach strategy involved programme staff visiting groups in the community to talk about, and in some cases, provide visual displays of, the programme. This gave people the opportunity to meet providers and discuss the programme with them. The credibility of the marketing message was strengthened where learners joined staff in outreach work.
Recruitment and Retention
The evaluation found that, although staff marketed their programmes vigorously, they recruited fewer clients than they had planned. This is explained by the difficulty of making contact with, and gaining the involvement of, hard-to-reach adults who generally lack the confidence to take up learning activities.
Retaining learners was a problem for some programmes. Learners left programmes for several reasons including poor health, housing difficulties, family responsibilities, and unstable lifestyles.
Programmes took action to remove barriers to recruitment and retention. This included usually paying the cost of travelling to the programmes and providing the basic tools for learning such as books and writing materials. Some programmes also offered or arranged childcare facilities.
The evaluation identified three aspects which were critical to learner retention:
The quality of the learning experience and its relevance to learners’ needs.
The social atmosphere of programmes which enabled learners to make friends and to provide mutual support.
The personal commitment of tutors which was the basis of good staff-learner relationships.
Programme Features
The programmes visited for the case studies were provided by voluntary organisations which aimed to promote the social rehabilitation and inclusion of the clients groups through improving their basic skills. They aimed to offer a gateway to learning and other opportunities in the community such as employment and voluntary work.
Most of the organisations running the programmes were new to, and had little in-house expertise in, delivering basic skills. The main response was to buy in basic skills tutors from local colleges. Others decided to train their own volunteers.
The duration of the programmes was usually between 12 and 20 weeks. Most held one or two teaching sessions per week, usually between 45 minutes and two hours. Some tutors thought that this was not sufficient for learners to make marked or measurable progress in improving their literacy and numeracy. Tutors used assessment tasks (e.g. writing and reading), observation and in some cases Fast Track to screen learners for basic skills needs and based individual learning plans on this initial assessment.
The case-study programmes integrated basic skills tuition with other learning activities, including cooking, childcare, healthy eating, computer skills, motor bike maintenance and riding, photography, and recycling waste. This approach was considered appropriate for the client groups who would be deterred from learning literacy and numeracy skills as discrete activities. Staff delivered basic skills through individual, differentiated teaching within a small group of up to seven learners. One-to-one teaching was occasionally employed where learners, such as the visually impaired, had special needs. The use of IT in teaching basic skills varied considerably.
A common feature of programmes was the informal style of delivery. Learners valued this relaxed approach because it took pressure off them, let them learn at their own pace, and was unlike school of which many had bad memories.
Support
The learners interviewed in the case studies reported that the most consistent and most dependable form of support was that given by programme managers, coordinators, and tutors. Some organisations also provided volunteers, support workers or key workers with whom learners could discuss personal problems. Learners on some programmes provided useful support for each other. Other forms of support, particularly the payment of travel expenses and the provision of childcare, were also regarded as extremely important.
Impact on Learners
Personal and Social Development
The evaluation found that the greatest impact of the programmes was on learners’ personal development, including their self-confidence, self-esteem, motivation and ability to think for themselves. The experience of being in a group, with others ‘in the same boat’, which gave them social contact and support was important in this respect. For some learners it was the practical experiences which programmes offered that they really appreciated and which helped them develop new interests in life. However, some learners remained fragile and had some way to go in gaining sufficient confidence to cope with personal and social challenges.
Learning to be Learners
Programme staff noted that learners’ progress in developing and using new skills was dependent on their confidence and emerging identity as learners. Most programmes focused on helping clients to develop a sense of themselves as learners which included teaching study skills.
Basic Skills Development
The case studies revealed that the ACLF programmes visited had had a positive impact on learners’ basic skills. Programme staff reported that there had been improvements in the literacy and numeracy for the majority of learners. Some had also acquired IT skills. Examples of the gains made in literacy included reading more in general and reading more advanced material, improvements in spelling, and using new vocabulary in writing. Examples of the gains made in numeracy included improved skills in calculating weight, length and capacity and in using percentages and ratios.
Practical Skills
Enhanced practical and coping skills were important gains made by some learners from their participation in the programmes. These skills included cookery, DIY, welding, servicing and riding motor cycles, taking and developing photographs, pottery and recycling household waste. Some learners had gained an increased ability to deal with minor problems. Knowing where to get information meant that some learners were in a better position to make informed choices about the future.
Progression
The evaluation found that the programmes had had a positive impact on some learners in terms of increased awareness, raised aspirations and changes in values and attitudes. This included broadened horizons about what they might do in life and increased ambition.
Programmes reported that some learners had progressed to further education, employment or volunteering. Examples included:
Further education: child care courses, pathways courses for English, maths and IT, a creative media and basic skills course, a creative writing course, and a job search course.
Employment: part- and full-time work, including jobs in retail, in the building trade, and in the service sector.
Volunteering: on the programme itself, in church activities, doing some teaching, and placements with agencies for the homeless and drug prevention.
Impact on Organisational Capacity
The evaluation found that the ACLF had had a positive impact on participating organisations. Marketing the programmes had helped to raise their community profile through increased contact with people and local agencies, and in some cases, through coverage in the media.
Programme managers reported an increase in the profile of basic skills provision within their organisations. This was linked to an enhanced awareness of what basic skills were and how to deliver them amongst staff and volunteers.
The ACLF had enabled organisations to acquire resources, such as teaching materials, which they could use to deliver basic skills now and in the future. The ACLF also helped to develop organisations’ capacity to deliver basic skills by providing opportunities for staff to gain experience and by providing the resources to buy in teaching expertise.
There was evidence that the ACLF had strengthened organisations’ links and partnerships with other organisations, including community groups, local authorities, drug referral agencies, youth offending teams, careers services, employment agencies, and colleges.
The evaluation found that the case-study programmes were attempting to secure funding which they considered to be essential for them to sustain the developments and progress made in providing opportunities for people, especially adults unaccustomed to education, to learn skills and improve their literacy and numeracy.
Future Developments
The experience gained through ACLF projects is now feeding into new initiatives. Some of these address social exclusion, such as the National Strategy for Neighbourhood Renewal. Others focus on improving basic skills, through the Skills for Life strategy. The latter include the Link-Up national volunteer project and the National Voluntary Organisations Partnership Programme. The capacity-building work of ACLF in the voluntary sector is influencing the content of training schemes being developed for volunteers and front-line workers. Lessons learned about delivering ‘embedded’ basic skills are feeding into an Adult Basic Skills Strategy Unit/Learning and Skills Council project exploring this area. The new literacy, numeracy and ESOL curriculum will also be a valuable resource for current and future ACLF projects delivering basic skills, which should enable them to identity learners’ progress in specific skills more clearly.
Finally, this evaluation has shown that the range of experience and expertise gained in designing and running ACLF programmes is a valuable resource which can be used to develop the skills of individuals and contribute to the renewal of their communities.
1
1.INTRODUCTION
1.1Background
A recognition of the value of learning underpins the policy drive to offer opportunities for people to learn throughout their lives. This is illustrated in the Adult and Community Learning Fund (ACLF) Prospectus (2001)[1] which stated that: