Boys Into Higher Education Usingfootball Project

1

A Sporting Chance

Boys into Higher Education usingFootball Project

Report of the evaluation of the second phase

July 2010

Steve Porter, BSc, MEd, DMS

Executive Summary

This is the report of the evaluation of the second phase of the ‘Boys into Higher Education through Football’ project, initiated, organised and coordinated through Leicester City and Leicestershire Aimhigher. It is intended to review the recommendations made in the report of the first phase of the project and investigate its impact on the Higher Education (HE) aspirations and awareness of the footballers. It is a particularly apposite time to submit this report, written as the 2010 World Cup reached its final stages.

The central concept of the project is to bring current students from the three Leicestershire universities into local football clubs to help with the coaching of the players and, at the same time, to deliver information, advice and guidance (IAG) about HE to the boys. These students are either studying sports science courses or are members of university football teams: all are very competent players. Other activities, notable university visits, have been organised to reinforce the student coaches’ work.

The footballers’ opinions about the project were surveyed by questionnaires and focus groups. These gave a consistent message that the boys had enjoyed their participation in the project, both in terms of the benefit to their football and to their HE aspirations and awareness. They had particularly liked the visits to local universities and had learned a great deal about what universities are and what they as students would do there. They had related well to their student coaches and their opinion of students had been improved by contact with them. There was a generally increased intention to consider HE study and some evidence of improved performance at school as a result of participation in the project.

The student coaches had also benefited from the experience. They were paid for their efforts; they gained experience of coaching (some were working towards coaching qualifications) and they developed generic skills such as communication and working with others. They also generally enjoyed the experience of working with the boys and were aware that they were role models, helping the Aimhigher cohort think about their future in new ways.

The parents’ views on the project were positive and some parents’ comments were enlightening and encouraging. They agreed that the project was worthwhile and had achieved many of its objectives.

The qualitative and quantitative data obtained show that issues identified by the evaluation of the first phase had largely been resolved although some remain. They also strongly indicate that the project continues to be a most valuable way of raising awareness of and aspiration to study in HE. The evaluation methodology, which is the same as that used to gather data for the first report as far as possible, survives critical appraisal and, with some shortcomings (which are discussed), is robust.

Recommendations for making further improvements to the project in the future are made.

Thanks are due to all who participated in the evaluation and to all who have worked hard to deliver, support and coordinate the project’s activities over the 2009-10 season.
Contents

Page Number
Executive Summary / 2
1 Introduction / 4
Investigation of recommendations / 4
Investigation of the impact of the project / 4
Evaluation Methodology / 5
2 Player’s Questionnaire Responses / 7
Discussion of Findings / 8
General comments / 9
3 Players focus group responses / 10
Discussion of the focus group responses / 17
4 Student coaches questionnaire responses / 18
Discussion of the Student Coaches’ Responses / 22
5 Parent’s Questionnaire Responses / 24
Discussion of Parent’s Questionnaire Responses / 26
6 Organisers’ Questionnaire / 28
Discussion of Organisers’’ Questionnaire Responses / 33
7 Discussion of Evaluation Methodology / 35
8 Recommendations / 36
Appendix: Student Coaches’ Questionnaire / 39
Acknowledgements / 40

1 Introduction

Aimhigher Leicester and Leicestershire have commissioned a second evaluation of the ‘Boys into HE using Football’ project. The evaluation was carried out by Semita Education and Trainingand took the evaluation on from Carpenter and Kerrigan’s report of the first phase of the project[1].

Semita was charged with focussing attention on two aspects of the project:

a. To investigate how effectively the recommendations made under Section 4 of Carpenter and Kerrigan (subsections 4.1-4.5, pp 23-27)have been put in place and whether any further recommendations for change need to be made;

b.To investigate the continuing impact of the project on the HE aspirations and awareness of the cohort.

1.1 Investigation of Recommendations

The following issues were under particular scrutiny during the evaluation:

1. Organisation and communication: this was highlighted in the first report as a significant problem, given the number of parties involved and the spread of institutions from which they came.

1. Awareness of the project: the issue appears to be lack of awareness of the aims and objectives of the project amongst the boys, their parents and the student coaches.

1. Recruitment and training of student coaches: the main issue appears to be with the training of the coaches but any recruitment difficulties were scrutinised as well.

1. Engaging Parents and Carers: the first report emphasised the importance of parent/ carer involvement; this evaluation looked at how (if at all) it had developed.

1. HEI visits: the first report raises questions about whether players should visit HEIs from whence their coaches had come and the number of HEIs visited. This evaluation reviews the whole HE visit issue.

1.2 Investigation of the Impact of the Project

As Carpenter and Kerrigan recognise, ‘as the report is based on research carried out shortly after the completion of the project, an assessment of the short-term impact only can be presented. Medium- and long-term impacts such as variations in attainment and HE progression rates will not become apparent for a number of years. Thus measuring these outcomes is not yet feasible. Nevertheless, short-term impact is important evidence in its own right and contributes a “layer of evidence” as advocated by the HEFCE guidance[2]’.

Bringing the 2009-10 cycle of the project into the evaluation providesanother ‘layer of evidence’. To ensure comparability with the first evaluation, the criteria and research methodology used in the June 2009 evaluation are used again.

1.3 Evaluation Methodology

The methodology used for the evaluation followed standard research techniques. It wasconsistent with that used in Carpenter and Kerrigan’s evaluation, using the same questionnaires and focus group protocol that they did, to ensure comparability with their findings. The main features are briefly discussed below.

1. Players’ Questionnaire

The questionnaire used was a mixture of closed questions (Yes/ No/ Don’t Know), attitudinal questions (Strongly Agree/ Agree/ Don’t Know/ Disagree/ Strongly Disagree) and open questions (see Table 1). This format works well because it allows numerical values to be assigned to responses to closed and attitudinal questions, generating quantitative data. It also enriches the data with the boys’ thoughts expressed in their own words. For justification of this methodology, see Likert[3]and others, eg.[4].

1. Player’s Focus Group Research

This also used a standard focus groups protocol (see Table 2). Whilst focus groups are much maligned because of the criticism of their use in the political arena in the press, they remain a valid research tool and can provide valuable qualitative data. This is the case in this evaluation, where the boys’ responses provide highly enlightening comments and suggestions for the further development of the project. For a discussion of focus group research, see [5].

1. Student Coaches’ Questionnaire

The responses to the student coaches’ questionnaire (see Appendix) are here presented as a series of case studies. This was felt to be the best way of reporting the opinion of the student coaches, since there were only a small number of questionnaire returns from them. The resulting qualitative data are very informative. This is the only questionnaire shown as an appendix since the format of the others is obvious from the tables which report the findings.

1. Parents’ Questionnaire

The questionnaire was again a mixture of closed and open questions (Table 3). Several parents took the opportunity to answer the open questions with very informative comments.

It was decided to amalgamate the ‘Strongly Agree’ and ‘Agree’ responses in all the questionnaires – similarly the ‘Strongly Disagree’ and ‘Disagree’ responses. This was because the small sample size in all of the questionnaire investigations would have meant very small numbers if the data had been expressed separately. In scoring the closed question responses, the ‘Strongly Agree’ and ‘Strongly Disagree’ answers were weighted so that they scored double the ‘Agree’ and ‘Disagree’ responses. The raw data were then converted to percentages.

1. Organisers’ Interview

This was completed jointlyby the two chief organisers of the project.They suggested that they might submit a single combined response, which was agreed, so interview the questions were sent to the organisers by email. The questions asked and the responses to each question are shown in full on pp28-32. The data obtained were purely qualitative.

2 Player’sQuestionnaire Responses

Question / Y8 / Y9 / Y10
% Yes/ Agree / % No/ Disagree / % Yes/ Agree / % No/ Disagree / % Yes/ Agree / % No/ Disagree
Q 1-3 were ‘Yes/ No/ Don’t Know’ responses
1 Had you considered HE as one of your options before you started this project? / 46.7 / 13.3 / 25.0 / 50.0 / 20.0 / 20.0
2 Would you consider it now? / 80.0 / 20.0 / 25.0 / 50.0 / 90.0 / 0
3a Have the student coaches discussed HE with you during your football sessions? / 66.7 / 6.7 / 87.5 / 0 / 90.0 / 0
3b If yes, have these discussions been of benefit to you? / 40.0 / 20.0 / 50.0 / 0 / 70.0 / 0
Q 4-10 were ‘Strongly agree/ Agree/ Don’t Know/ Disagree/ Strongly disagree’ responses
4 This project has increased my understanding about university / 75.0 / 6.25 / 55.6 / 22.2 / 50.0 / 0
5 This project has increased my confidence about studying at university level / 47.1 / 11.8 / 22.2 / 44.4 / 54.5 / 0
6 This project has encouraged me to do well with my studies / 56.25 / 12.5 / 33.3 / 44.4 / 63.6 / 0
7 This project has increased my confidence about ‘fitting in’ at university / 58.9 / 0 / 44.4 / 11.1 / 54.5 / 9.1
8 I have enjoyed the project / 84.2 / 5.3 / 55.6 / 11.1 / 92.3 / 7.7
9 I have found the project useful / 62.5 / 0 / 55.6 / 11.1 / 84.0 / 0
10 The project has been well organised / 81.25 / 0 / 66.7 / 0 / 75.0 / 0
Q 11-13 were ‘Yes/ No/ Don’t Know’ responses
11 Do you think you will study at university in the future? / 46.7 / 0 / 12.5 / 62.5 / 20.0 / 0
12a Has the project influenced your thoughts about studying at university in the future? / 53.3 / 13.3 / 0 / 75.0 / 50.0 / 20.0
12b If so how? Please add your own thoughts / Don't know;
It's given me more information about life at university;
All of the facilities;
Because of the facilities;
That there's lots of stuff to do;
That there's lots of stuff to do. / Not sure. / Feel more interested;
Sports facilities good;
The sports equipment;
It looks interesting and involves more than just studying;
By the ways they have shown me what they can do.
13a As part of this project, have you visited a university? / 100 / 100 / 90.0 / 10.0
13b If yes, which university? / Both Loughborough and De MontfortUniversities / LoughboroughUniversity / LoughboroughUniversity
14 Any other thoughts or comments? / There were none. / There were none. / There were none.

Table 1

2.1 Discussion of Findings

Note: the ‘Don’t Know’ responses have not been recorded in the table, explaining why the percentages do not add up to 100 for most questions.

These are a very encouraging set of responses, indicating overall a positive improvement in the boys’ attitude to HE as a consequence of the project. This is both in terms of aspiration- and awareness-raising. There are some interesting variations on this theme, however, particularly involving the Y9 boys.

Q1and 2:over three times as many Y8 boys said they had considered HE as an option before the project, compared to twice as many Y9s saying they hadnotconsidered it and the Y10s being evenly split. The change in attitude over the project for Y8 (to four times as many) and Y10 (90% saying they’d consider HE now) was most encouraging; so it’s odd that the Y9 sample’s opinion remains unmoved. This may be a factor of the sample, a characteristic of Y9 boys in general or that the Y9s had less effective student coaches.

Q3a and 3b: however, examination of the responses to these questions indicate that all year groups had discussions about HE with their student coaches and all found these of benefit. The different responses of Y9 to Q1 and 2 thus remain unexplained.

Q4, 5 and 7:the responses to this set of questions on understanding what HE is, having more confidence of being able to cope with HE study and more confidence of ‘fitting in’ are consistently positive (some very much so), with the exceptiononce againof Y9, who indicate being less confident of being capable of HE success. This suggests that this is a factor of the Y9 sample; perhaps it is a less able set of boys than the Y8 orY10 samples. The positive aspect is that, although the change is negative, it is a change, an indication that the Y9 boys have had their awareness of HE raised and are therefore in a position to make a better-informed choice.

Q6: Ys 8 and 10 indicate that the project has had a positive influence on their attitude to study; Y9 disagree. The responses are consistent with others in the questionnaire.

Q8 and 9: all year groups agree that they found the project interesting and useful: but do they mean they just enjoyed the football? The way this question is worded doesn’t allow for differentiation between enjoying the football and enjoying finding out about HE.

Q10: all agree it was well organised from their perspective.

Q11: there were lots of ‘Don’t Knows’ here, as might be expected from pupils as far as 4 years away from a UCAS application. Again there was a difference in opinion, very marked in this case, between Ys 8 and 10 (where there were no ‘No’ answers) and Y9 (5 times more ‘No’sthan ‘Yeses’). It would be regrettable if these boys’ aspirations to study in HE had been reduced by participation in the project. The findings are again consistent.

Q12a: again some ‘Don’t Knows’ but only Y9 indicate that their thinking has not been influenced by the project. Note that the Y9 responses to Q12a contradict those for Q4, 5 and 7. Note also that the wording of the question does not allow determination of whether the influences for Y8 and Y10 were positive or negative.

Q12b: unfortunately Y9 did not add any useful comments to elucidate their responses to Q12a. The Y8 and 10 boys seem to have had their perceptions of what it’s like at university changed – this is supported by the focus group responses (eg. see Q27, p14). They were impressed with what they saw at both DMU and Loughborough. Note that there is a strong indication that these perceptions arise from the university visits rather than from the student coaches’ inputs during the sessions at the football clubs.

Q13a and 13b: attendance at the university visits was excellent. The Y8 boys attended two visits (it’s unclear from these responses whether the ‘Leicester’ university they refer to was DMU or Leicester University itself; however, the focus group responses make it clear it was DMU). It might be that the very positive responses from the Y8 boys are at least in part a factor of having two visits.

Q14:it is a pity that there were no comments here, but the focus group responses fill this gap to some extent.

2.2 General Comments

1. There appeared to be some collaboration in the filling in of the questionnaires but this was not significant (they were completed on the bus journey from Loughborough to Leicester).
2. The sample size was small, with only 33 boys responding; and they were all from one club, Shepshed Dynamo FC. It proved to be quite difficult to get the questionnaires returned.
3. Thus the findings must be taken as indicative but, with this caveat, the questionnaire responses overall do quite strongly indicate that the project has largely achieved its aims: the boys’ aspirations to study in HE and awareness of different aspects of HE have been developed (NB the non-study elements, particularly the sport: several of the boys were clearly very pleasantly surprised by the sports facilities they saw).
   3 Players Focus Group Responses

Question / LancasterSchool
NOTE: A focus group session was held at LancasterSchool on 4th May 2010. There were 4 boys in attendance, one from Y8 (under 13 football level), one from Y9 (u14) and two from Y10 (u15). Time with the group was limited so not all of the questions were answered but a useful response was elucidated. Q 29-36: time ran out so these weren’t covered. / ShepshedSchool
NOTE: A second focus group session was held at ShepshedSchool on 11th June 2010. This group comprised 9 boys, all from Y8 (u13). More time was available for this event and better coverage of questions was achieved, although some questions were not asked to save time. The same question protocol was used at both schools.
Q7: what can you tell me about the football project that you were involved with? / Y8: I thought it was just a uni trip, that’s all I was told. I found out more about uni at the Mon 26th April event [the social evening held at Aylestone Park FC].
Y9: We went to this uni – I didn’t know much about it – it was DMU.
Y10s: We went to two different unis (Loughborough and DMU) – we found out what courses you could do and what it’s like to live there. The student coaches who helped out with the training told us a bit. / Not asked to save time
Comment: / Indicates a necessity for better preparation of the boys prior to the university visits. An activity-based approach (for example that used in GIGO[6]would be appropriate.
Q8: what do you think the project hoped to achieve? / Y8: Uni can be fun and it’s not boring.
Y9: How to improve our football. At uni we learned some new information, like tactical stuff. Our student coaches did this on the uni visit.
Y10: To persuade us to go to uni, that uni’s not boring. I thought it might be boring but it’s OK now. You live with your friends, study what you want to. It was to see how good uni was and what you did when you went.