An analysis of change in learning sites in four FE colleges in England[1]
Keith Postlethwaite*
Senior Lecturer in Education
Wendy Maull
Research Fellow
School of Education and Lifelong Learning
University of Exeter
St Luke's Campus
Heavitree Rd
Exeter
EX1 2LU
Introduction
The 'Transforming Learning Cultures in FE' project (TLC(FE)) is currently exploring cultures of learning within the FE sector by looking in detail at, initially, 16 learning sites[2] - four in each of four colleges in England. General details of the project, and of the four partner colleges can be obtained from the project website located at
The sites included in the project were not intended to be a representative sample of the range of activity that is common in FE. There were, for example, no sites that were concerned with the higher education provision, which is a feature of some FE college work. However, the range of sites was wide. The sites varied on a number of dimensions such as the level of the target qualification (some were, for example, concerned with A-level, AVCE, or BTEC Higher National Diploma, whereas others were concerned with personal support for IT skills or English language support), mode of attendance (eg some were full time, some part time), length of study (some were short roll-on/roll-off sites where the period of study was negotiated between student and tutor, some were one year, and some two year programmes), location of learning (eg classroom or workplace or distance learning), student profile and tutor profile. For further details of all the sites, see Appendix 3 and James, (2004).
2) Methods
The main focus of the project was on detailed case studies of (usually) six of the students in each of the sixteen learning sites. However, in addition to conducting an extensive series of interviews with this sample of students, the project has collected data from all students in the sites by means of a questionnaire. This questionnaire was developed to include items about the students that would help us to locate them in a wider social context; it asked about their expectations of their course and, at the end of their studies, about the outcomes they had achieved; it included a scale designed to assess students' perceptions of their learning environment. The scale chosen for this purpose was the Constructivist Learning Environment Survey (Fraser & Treagust, 1986; Taylor, Fraser, & Fisher, 1997). This was selected because it matched the socially constructed view of learning that was central to the project, whereas other scales tended to be based on more individualistic views of learning. The questions from this instrument gave insights into 'shared control', 'critical voice' and 'student negotiation' (the details of which can be found in Appendix Two). More information about the development and piloting of the questionnaire can be found in (Postlethwaite & Maull, 2003).
The current paper is concerned with issues of change within the learning sites. It is based on findings from the questionnaire data, though we also draw on the interview data where appropriate to gain deeper insights into the issues identified. As noted above, sites were of different duration: short periods of study, one-year courses or two year courses. Given our current focus on change, this 'length of study' variable was particularly pertinent. Questionnaires were used at the start of each year, and at the end of the year (or end of the period of study if this was earlier). For students engaged in a period of study of one year or less, we therefore had data at the beginning and end of their programme. For students engaged in a two-year period of study, we also had data at the mid point of their programme - though these mid-point data are not discussed in detail in the present paper. At the time of writing, data were available on two cohorts of students. There is therefore also the opportunity to consider whether the sites were similar for two consecutive student groups.
Details of the numbers of students involved in the work reported here are given in Appendix One.
Data were analysed using, mainly, repeated measures t tests, and oneway ANOVA with post hoc tests. Analysis was done using SPSS v10. Since we make no claims to be studying a representative sample of the FE population of learning sites, and therefore do not seek to generalise our finding to that population, we are making non-inferential use of inferential statistics in this paper (Postlethwaite & Maull, 2003)
3) Aspects of change
This section discusses differences in students' questionnaire responses between the start and finish of their course of study. It begins with an analysis of overall responses for all students, then deals with individual learning sites. It therefore provides a picture of how the views of the students involved in the sites changed during their period of study. This section finishes with an analysis of the differences between interviewees and the other students in their sites.
Overall findings
In the tables that follow, results for Cohorts 1 and 2 are discussed, and courses of one year's duration and less, are reported separately from courses that lasted for two years.
Tables 1 and 2 show start and end results for the first cohort of students on one-year and two-year courses.
Table 1 Students' ratings of expectations and achievements for their course of study: Students on one year courses: Cohort 1 2001-2: Entry to exit
On this course, I expect I will…(On this course, I did…) / Strongly disagree
1 / Disagree
2 / Neutral
3 / Agree
4 / Strongly agree
5 / Means
(sig)[3]
…make friends / 1
(0) / 2
(1) / 6
(2) / 25
(21) / 21
(31) / 4.15 (4.49) (**)
…enjoy the experience / 0
(0) / 0
(0) / 4
(1) / 33
(27) / 18
(24) / 4.25 (4.35) (*)
…use the course to continue studies / 0
(2) / 3
(6) / 16
(8) / 21
(19) / 15
(20) / 3.89 (3.89)
(-)
…improve my job prospects / 2
(2) / 3
(8) / 12
(20) / 17
(11) / 21
(14) / 3.95 (3.49) (*)
From this table it is clear that students were optimistic about their engagement in their courses at the start of their year. In two respects (making friends and enjoying the course) this optimistic expectation was exceeded by their perceived achievements at the end. In both cases, the positive differences between expectation and achievement were statistically significant.
The two respects in which optimistic expectations were not exceeded by achievements were 'using the course to continue my studies' and 'improving my job prospects'. The negative difference in the case of job prospects was statistically significant.
Discussion with colleagues who collected and analysed the interview data for these students suggested that reasons for decreased optimism about employment prospects were complex. The decline may have reflected students' growing understanding of the general reality of the job market, or their growing awareness of the specific issues of a mismatch between the job demands and their abilities, or their growing sense that relevant employers did not seem to value the course on which they were engaged.
The responses to the question about students' intentions to "use the course to continue studies" appeared to polarise. The experience of the year seemed to help those who were undecided to make up their minds whether they expected to continue or not. This polarisation would not show as significant on a statistical test that was comparing means.
Table 2 Students' ratings of expectations and achievements for their course of study: Students on two year courses: Cohort 1 2001-3 Entry to exit
On this course, I expect I will…(On this course, I did…) / Strongly disagree
1 / Disagree
2 / Neutral
3 / Agree
4 / Strongly agree
5 / Means
(sig)
…make friends / 0,
(1) / 1,
(1) / 5,
(3) / 23,
(25) / 28,
(27) / 4.37 (4.33)
(-)
…enjoy the experience / 0,
(2) / 3,
(2) / 12,
(8) / 31,
(32) / 11,
(13) / 3.88 (3.91)
(-)
…use the course to continue studies / 1,
(16) / 5,
(11) / 18,
(11) / 16,
(10) / 17,
(7) / 3.73 (2.65) (**)
…improve my job prospects / 0,
(4) / 1,
(2) / 5,
(9) / 28,
(28) / 23,
(12) / 4.25 (3.76) (**)
In relation to making friends and to enjoying the course, students were (like their colleagues on one-year courses) optimistic at the start of their studies. At the end of their courses these early expectations were similar or had, for the group as a whole, been somewhat exceeded. However, unlike their colleagues on one-year programmes, the differences between views at the start and end of these two-year courses were not statistically significant.
Although mid-course figures are not reported in Table 2, it was the case that students' views about enjoying their courses were significantly lower at the mid point of their programme, but recovered during the second year.
In relation to improving job prospects, expectations for the cohort dropped significantly between the start and end of the students' two-year courses, matching the direction of the change for the students on one-year courses. Again, the mid point data for the two-year course students are not reported in Table 2, but these data suggest that a significant fall in students' views about their employment prospects was already apparent by the end of their first year. The factors mentioned in relation to one-year courses may be operating again here. It is interesting that they appeared to have their impact well before the end of the students' course, when the realities of job hunting (or, for those already in employment, seeking advancement in their career on the basis of their studies) may have started to have an effect.
Like their colleagues on one-year courses, students on two-year courses began their studies with similar, positive views of the likelihood that they would use their courses as a basis for continuing their studies. However, for these two-year students there was, by the end of their courses, a statistically significant drop in this expectation. This could be a reflection of two-year students' disillusionment with FE - though their positive comments about 'enjoying the experience' make this explanation unlikely. It could reflect a concern (particularly for students who came directly from school to FE) to move on from continuing education to employment. If this is the case, students' reduced expectation that their studies would help them improve their job prospects is particularly worrying.
Taking all four questions together, students' views changed less positively over two-year courses than one-year courses. They did become more positive about making friends and enjoying their courses, (though in contrast to their peers on one-year courses, the positive change in views was not significant); they became more negative about continuing their studies and securing jobs (and in contrast to their peers on one-year courses this negative change was even greater).
Table 3 and 4 show the results for the second cohort of students. At the time of writing full exit data for the students engaged on two year courses were not available for analysis, so the results reported are those for the start of, and for the entry to the second year of, these courses. An analysis of exit data will be discussed in a subsequent paper.
Table 3 Students' ratings of expectations and achievements for their course of study: Students on one year courses: Cohort 2 2002-3 Entry to exit
On this course, I expect I will…(On this course, I did…) / Strongly disagree
1 / Disagree
2 / Neutral
3 / Agree
4 / Strongly agree
5 / Means
(sig)
…make friends / 0,
(1) / 1,
(0) / 4,
(3) / 32,
(24) / 18,
(27) / 4.21 (4.39)
(-)
…enjoy the experience / 0,
(0) / 3,
(0) / 5,
(7) / 33,
(32) / 14,
(16) / 4.05 (4.16)
(-)
…use the course to continue studies / 3,
(1) / 2,
(3) / 13,
(17) / 15,
(20) / 21,
(14) / 3.93 (3.80)
(-)
…improve my job prospects / 1,
(2) / 3,
(11) / 12,
(17) / 25,
(15) / 13,
(9) / 3.87 (3.35) (**)
Like Cohort 1, students' expectations about making friends and enjoying their studies were positive and somewhat increased during the year though these changes were not significant.
Students' expectations to continue their studies were again quite high. Unlike Cohort 1 these expectations did not change significantly.
Students' expectations to improve their job prospects again began positively and again fell significantly over the year.
Table 4Students' ratings of expectations and achievements for their course of study: Students on a two year course: Cohort 2 2002-4
Entry to mid-course
On this course, I expect I will... (entry)On this course, I expect I will... (start of yr2) / Strongly disagree
1 / Disagree
2 / Neutral
3 / Agree
4 / Strongly agree
5 / Means
(sig)
…make friends / 0,
(0) / 2,
(4) / 5,
(4) / 39,
(42) / 16,
(12) / 4.11 (4.00)
(-)
…enjoy the experience / 0,
(2) / 1,
(5) / 11,
(13) / 41,
(35) / 9,
(7) / 3.94 (3.65) (**)
…use the course to continue studies / 1,
(2) / 1,
(9) / 18,
(19) / 21,
(14) / 21,
(18) / 3.97 (3.60) (**)
…improve my job prospects / 0,
(0) / 0,
(0) / 7,
(13) / 31,
(32) / 21,
(14) / 4.26 (4.00) (*)
Students again expected to make friends and found this expectation to be matched by their experience of the first year of their courses. They expected to enjoy the experience but the experience was significantly less positive at the end of their first year. This finding was similar to the mid point result for Cohort 1. In Cohort 1, as shown above, the positive view was regained over the second year. It will be interesting to see if this pattern is repeated in Cohort 2.
Students began with positive expectations to continue their studies but by the end of their first year were significantly less positive about this prospect. The large drop reported in Table 2 between entry and exit for Cohort 1 was already apparent in the Cohort 1 mid point data, so this finding is broadly consistent with the Cohort 1 result. Again, tt will be interesting to see if this fall in expectation over the first year for Cohort 2 is sustained during the second year as it was for Cohort 1.
For Cohort 1, views about improved job prospects fell significantly over the first year and over the whole two-year period. The first year fall was again evident for Cohort 2.
In summary, the overall findings suggest a consistent fall across the four groups of students in their sense of optimism about the impact of their studies on their job prospects. Also consistent was the expectation to enjoy the experience of study, and the report that the expectation was, at least, fulfilled. In both cohorts of students on two-year courses, the expectation that they would use the course to continue their studies fell.
Analysis site by site
As outlined above, in two year sites, comparisons were made, for Cohort 1 students, between CLES scores at the start of Year 1 and the end of Year 1/Start of Year 2; between the start of Year 2 and the end of Year 2; between the start of Year 1 and the end of Year 2. These three comparisons, made using related t-tests, could be interpreted as mapping a sequence of change through three points roughly one year apart. Clearly, each comparison can only use data from students who replied to the questionnaire at both points in the comparison. A student replying to a question at the start of Year1, start of Year2 and end of Year 2 would appear in all three comparisons. A student who omitted a question in the start of Year2 questionnaire could only appear in the overall comparison. The student numbers therefore vary in the different parts of the analysis. Since a complete data set for Cohort 2 students is not yet available, there is no opportunity to repeat this analysis for a separate cohort of students to investigate the stability of any reported effects.
In one year sites, comparisons for two student cohorts were possible: between the start of Year 1 and end of Year 1 for Cohort 1; between the start of Year 2 and end of Year 2 for Cohort 2. These are separate comparisons relating to different groups of students. They can give indications of the stability of any effects revealed in the analysis.
Similar analysis was conducted on students' expectations and achievements on a site by site basis.
Some examples of the site by site analysis for two year and one year sites are given below.
CACHE (PL) - two year site
Shared control scores rose steadily over the two years (start Yr1 = 1.76; start Yr2 = 2.00; end Yr2 = 2.51) with the overall difference reaching significance, suggesting an increasing tendency for students to negotiate with their tutor. This may be a consequence of a change, mapped through the qualitative data, in the tutor's sense of where responsibility for learning should lie. In parallel, there was a fall over the two years in student negotiation - ie students' tendency to discuss work with each other. At the start of Year 1, the mean score on this variable was 2.77; by the end of Year 2 it was 2.38. This difference was statistically significant.
There was a big drop in 'I expect to make friends' scores in Year 1, which was recouped in Yr2. A similar pattern was evident in 'I expect to enjoy the experience'. From the qualitative data it seems that this could have been related to the development of a split between students in the first year of this site leading to the formation of two sub-groups who might be labelled 'conforming' and 'challenging' and who did not seem to relate well to each other.
The 'I expect to use the course to continue my studies' scores were stable in Yr1 (perhaps reflecting the fact that this was a 2 year course), but these scores fell sharply in Yr 2. The 'I expect to improve my job prospects' scores fell in both years (significantly in Yr1 and overall). The qualitative data suggested that the falls in Yr2 may have been related to the fact that the group had a poor mock exam experience which may well have generated pessimistic feelings about the possibility of moving from the course into other study or into employment.