The Research in Teaching Environment (Rite) Initiative
The Research in Teaching Environment (RiTE) Initiative
Final Report
Date of completion: 20th September 2010
Completed by: James Wilkinson
Title of Project:
Evaluation of the ‘CLEAR’ Approach:
Combined Learning for Employability and Research,
an approach which integrates project management, creativity and cross-cultural team management, and where the tutor provides leadership by example to develop students’ research and team management skills.
(the original title was ‘Evaluation of the ‘Apprenticeship Model…’. This was changed to the ‘Combined Learning for Employability and Research’ Approach. This was to make clearer what the approach was for, and also because the previous title overemphasised the importance of the idea of ‘apprenticeship’, which was in fact only one of several integrated approaches.)
Contents
Page
- Project aims and purpose3
- Project Outline
- Stage 14
- Stage 2
- Stage 3
- Main findings
3.1.- 3.3. Skills, Knowledge and PersonalAttributes needed
for research, as identified by Research Methods tutors
3.4.Concerns identified by Research Methods tutors6
3.5.Stage 1 student responses7
3.6.Participant observations
3.7.Link to theory9
3.8.Summary of the CLEAR Approach
3.9.Stage 3 findings (post-intervention) 11
3.9.1.Disconfirming evidence
3.9.2.Reported benefits of the CLEAR Approach
3.9.3.Comparison of cohorts before and after CLEAR (fig.1) 13
- Evaluation of the project:
- Strengths 14
- Weaknesses
- Potential
- Limitations 15
- Main outcomes (publications/ conferences/ dissemination etc)
- References 16
Appendix 1: Summary conceptual framework for the CLEAR Approach 19
Appendix 2: Dynamic representation of the CLEAR Approach 20
Note
The CLEAR Approach, which is the subject of this evaluative research, continued to evolve during this project. In particular, the research permitted greater engagement with relevant literature to provide fuller theoretical underpinning, and findings from the early stages of the research further informed the approach. For this reason, the approach is not described until page 9. The approach is also summarised in the two appendices on pages 19 and 20.
- Project aims and purpose
This project had the following two main aims:
1.1.To better understand pedagogical issues relating to teaching, learning and assessment for research and employability education;
1.2.to evaluate the ‘Combined Learning for Employability and Research (CLEAR) approach, as applied on the Work Related Learning and Research (WRLR) module (HND Business)
Two further aims related to dissemination and the production of learning materials:
1.3.To disseminate research outcomes in the form of papers in journals and presented at conferences; and
1.4.To produce learning materials allowing the approach to be applied by other practitioners
- Project outline
Before and during students’ experience of the CLEAR approach, the first aim was pursued using a mix of approaches and at different stages:
2.1.Stage 1
participant observations, based on field notes written after WRLR classes, and drawing on previous years’ experiences with the module;
interviews with three TVU research methods tutors;
value grids, designed following the tutor interviews, were completed by 14 participants and used as a basis for interviews with students.
interviewswith eight of the total 15 WRLR students, prior to their experience with the CLEAR approach. These were transcribed and organised to facilitate analysis, and to provide baseline data for comparison with Stage 4 interviews.
Observations and interview transcripts from the above activities were reduced and organised using a grounded, hermeneutic, editing style, as described by Addison in Crabtree and Miller (1999).
2.2.Stage 2
Core categories identified in Stage 1 were next analysed with reference to relevant literature.
2.3.Stage 3
interviewswith eight of the total 15 WRLR students, shortly after their experience with the CLEAR approach. These were transcribed and analysed in association and comparison with the previous sets of data.
- Main findings
Findings from the Stage 1 interviews with tutors and from the author’s participant observationsyielded a set of skills, knowledge and personal attributes which tutors emphasised as being needed for research. Interviews also revealed a range of concernslinked to the above skills, knowledge and attributes, and relating to students’ readiness to undertake research. There were also concerns relating to pedagogical issues. Several of these mirror students’ comments and can be linked to the Stage 2 analysis and linking to theory.
3.1.Skills
Research skills
-Finding and evaluating sources of information and using these to focus research
-Handling complexity and uncertainty
-Producing a synthesis of ideas
-Writing critically and academically
-Reasoning and presenting arguments
-Editing and frequently improving written text (writing and re-writing)
-Selecting, justifying and applying data collection methods
Generic, transferable, employability skills
-Self management
-Time management
-Project management
-(if working in groups): team / group management
3.2.Knowledge and understanding
-Knowledge of the process(es) of research
-Research methods
-Analytical methods
-Sufficient subject knowledge to be specific
-Ethics
3.3.Personal Attributes
-Patience / perseverance
-Diligence – especially with reading
-Ability to take criticism
-Autonomy / self-managing
-Motivation
-Confidence
3.4.Tutor Concerns
Tutors expressed a range of concerns relating on the one hand to the students and on the other to issues linked to the necessary skills development. From the general tone of what tutors were saying, as well as from specific examples that they gave, many of their concerns can be interpreted as relating to a range of student shortcomings:
-lacking sufficient knowledge to be specific
-not reading enough (a key factor and linked to the previous point: conversely, ‘good’ students were those who do read and who use the knowledge thus acquired to inform their research focus and methods)
-lacking confidence
-lacking motivation
-difficulty working independently / autonomously
-procrastination
-poor time management
-lacking project management skills and knowledge
-poor self management
-lacking awareness of the need to edit and re-write work
From the tutors’ comments, it is possible to interpret many of these issues as being interconnected. For example, issues concerning procrastination, autonomy, self and time management and insufficient reading may relate to confidence and motivation, as is illustrated by the following comment:
Tutor 2: “What I feel I know enough about is to give them the guidance and structure to do it, and they should then be able to go and research it and take it from there, but they don’t. You know, you’d like them to come back next week and say: “Ok, this week, now I’ve got something, come on smart ass: let’s see you pick holes in this!” And that’s what you want from them, but they don’t seem to have that innate confidence, knowledge, and ability to want to do that or be prepared to do that”.
Concerning skills development, there was broad agreement among the three tutors that such development needs to start earlier in students’ programmes, as is illustrated by the following comment:
“…they arrive at this point, Level 6, with a lack of confidence, a lack of knowledge, a lack of skills in many cases, and we do cram it in there. And what we do, I think, is actually really good, … , but it is still a hell of a load to give some of these students and I suspect if you did a full research methods programme at Level 5 it might bring about a different situation, with students coming to this in a completely different state of preparation”. (Tutor 2)
Tutors were also concerned about the practice of teaching research methods in the abstract, before students needed to apply what they were learning. Tutor 2 suggested that a ‘drip-by-drip’ approach could help students if they were required immediately to apply the skills and knowledge they had been learning to their own research.
3.5.Stage 1 Student Responses
Many of the students’ responses reflected similar concerns to those expressed by the tutors. These related to:
-Getting started
-Studying independently: several said they preferred being told what to do rather than having to find out for themselves
-Being unclear about academic elements, such as drawing on ideas from different authors, being critical, dealing with controversy, producing a synthesis of ideas, and developing a conceptual framework for research.
Working in groups is also a key feature of the WRLR module, and here opinions were mixed. Some professedenthusiasm for group work while others said they preferred working on their own. Many had had the negative experience of working with colleagues who did not contribute. A few also said they did not feel confidence in expressing their views openly, or that their motivation was adversely affected if they felt that their opinions were not being listened to by others in the group. Two of the eight students mentioned that they were afraid to ask for help, both from other team members and from the tutor.
3.6.Participant Observations
Many of my own participant observations mirrored concerns expressed by both the tutors and the students. These related, on the one hand, to research related skills and their development:
-overreliance on lectures and virtual learning platforms to deliver research capability does not appear to provide sufficient skills development.
-Students either do not take in or have difficulty learning and applying the complex skills required for critical analysis, for explaining and justifying research methodology and methods, and for developing coherent, reasoned arguments.
-Linked to these difficulties, many fail to grasp the purpose and process of engaging critically with the literature and the need to devote sufficient time for reading.
On the other hand, there were also issues related to generic, transferable, employability skills gaps relating to:
-self, time and project management
-communication problems
-group management problems, such as dealing with free-riders and sometimes conflict.
3.7.Link to theory
Discourse on research and employability education reflects similar concerns reported by colleagues working in different disciplines and from across the Higher Education sector. These concerns include leaving research-based learning until the final year of the degree (e.g. Healey and Jenkins 2009), transmission-based approaches (e.g. Elton 2001; Laurillard 2002); learner passivity and dependence on ‘certainties’ provided by authorities (e.g. Boyer Commission 1998; Ramsden 2008; Baxter Magolda 2010); and insufficient development of generic skills and attributes relating to self, group and project management, as well as confidence and the related, counterproductive impact of high stakes assessment (Knight and Yorke 2003). Moreover, some of the strategies intended to address these concerns are themselves questioned. Student-centred approaches may be poorly thought through and applied, and fail to provide sufficient challenge (Furedi, in Gill 2008), and with social forms of learning, these risk students losing sight of individual responsibility (Sanger 2010).
3.8.The CLEAR Approach
As a response to theabove concerns, the ‘Combined Learning for Employability and Research’ (CLEAR) approach seeks to integrate pedagogical practices that promote learning which is:
-experiential and reflective (e.g. Laurillard 2002);
-inquiry based (e.g. Justice, Rice, Warry, Inglis, Miller and Sammon 2007; Spronken-Smith, Walker, O'Steen, Matthews, Batchelor and Angelo2008);
-socially constructed (e.g. Lave and Wenger 1991); and
-situated with reference to cultural discourses and involving a process of enculturation which transforms tutor and student identities (e.g. Wenger 1998; Quay 2003).
In order to achieve such learning, the CLEAR Approach draws on the cognitive apprenticeship model proposed by Collins, Seely Brown and Newman (cited by Woolley and Jarvis 2007). However, this includes not only showing students examples of best practice, but also rough drafts and subjecting these to students’ scrutiny, with the aim of developing their understanding of the process, confidence and critical skills.
To develop more generic, employability related competences, students also learn management techniques relating to project and time management (e.g. Sola and Wilkinson 2008) and are introduced to theory relating to cohesion (e.g. Rathje 2007), inclusivity and creativity (e.g. Ceserani and Greatwood 1995), uncertainty and anxiety management (Gudykunst 2004) to help them manage their own feelings towards unfamiliar others, and emotional intelligence (Goleman 1995).
For the past two years, the CLEAR approach has been used on the WRLR module.
Students work in groups to develop a research proposal and reflect on the experience, both during the project and afterwards, using Gibbs’ ‘reflective learning cycle’ (DAR 2009) and the above theory.To help them to monitor progress of the research-related elements, students are referred to the ‘inquiry process’ and ‘inquiry paper checklist’proposed by Justice et al. (2007, p.203 and 214). (This last element is in fact an innovation prompted by the project’s findings (see 3.9.1. below) and has not yet been used).
For a summarised conceptual frameworkand dynamic representation of the CLEAR Approach, see Appendices 1 and 2 on pages and .
3.9 Stage 3 Findings (post ‘CLEAR’ intervention student interviews)
With one exception, comments from the eight students interviewed shortly after completion of the module were predominantly positive.
3.9.1. Disconfirming evidence
The one student whose comments were not positive said they had found various elements confusing, in particular undertaking the literature review and discussing research methods:
“…there were so many sources … and we didn’t knowwhat was right and wrong and sometimes ended up concluding the wrong things and then not knowing what things needed changing.”
“…the research methods we used were quite difficult ... ‘cause nobody had done this sort of thing before, ....”
These comments may relate to issuesalready mentioned, such as confidence and also to some of the procrastinating tendencies of the student’s group colleagues which had hampered progress. The make-up of student groups is indeed a problem, and another student expressed frustration at the difficulty of achieving high grades when working with weak students. Whatever the underlying problems, this highlights the need for tutors to monitor student progress, and also to give students the tools needed to monitor progress themselves. To this end, studentswill in future be referred to the ‘inquiry process’ and ‘inquiry paper checklist’ designed by Justice et al. (2007, p.203 and 214).
3.9.2. Benefits of the CLEAR Approach
Positive responses predominantly emphasised benefits relating to generic, employability skills, notably group and project management, as well as to the development of personal attributes such as confidence, empathy and assertiveness. These benefits are illustrated by the following comments:
-on managing group projects :
“I know how to structure my group work...using pie charts and Gantt charts…”
-on confidence
“group work, that was my big problem, …(but now) I feel more confident”
-on group problem solving, empathy and assertiveness:
“ Before the module I used to be, well, ‘their problem is their problem, not
really my issue’ but now I … kind of look at the problem and kind of try and
help.”
“…(doing the module) help(ed) me understand other people’s situation better and at the same time not let them walk all over you…”
-on handling conflict
“…whenever I put my idea or opinion forward she would find fault in it and
give it no value….When things became excruciating, I confronted her. When we discussed the matters she realised that she was in the wrong and
agreed to take on board other people’s opinion… it made me feel much better and we were able to work smoothly”
In addition to these generic, group working skills and attributes, several students commented on the value of reflection. Interestingly, this related not simply to learning benefits, but also to practical problem-solving, an issue commented on by more than one student:
“…when we did those log books every week I really liked doing that ‘cause it … made you think about …the positive and negative aspects, and how you
could improve on them, so it was, like, problemsolving, how we can sort of solve our own problems”.
This problem-solving benefit can also be inferred from comments made when students were asked to say how the WRLR module differed from other modules they had studied:
“In ...(another module), ... we did not do that(i.e. keep a learning log) and most of the group work was really messy and people did not turn up and no one really wrote how they felt or what issue they had, like, no problems were solved.”
Having to reflect on progress on a weekly basis also appeared to help students apply the theory they were learning:
“ It worked well because it helped me learn, like, where I had gone wrong or if there were issues and how to deal with them using theory and learning about theory, ’cause in (another module) although we used theory quite a lot, like, let’s say ‘group work theory’ and ‘learning theory’ we did not really implement them in practice, and with work related learning we implemented them in practice.”
Reported benefits also related to social forms of learning, which also helped them to appreciate the value and the need for reading:
“…when I was working with the other girls, like K and M, that was really good,
‘cause I think they had more knowledge, they knew more…. We would always come in to do research in the library and just check out books and see what we could get, so… I think I have learnt from them how they do research… actually go to the sources and even, like, I don’t know, just reading books and stuff and I think that helped quite a lot, ‘cause it just broadened my knowledge a bit.”
3.9.3. Comparison of Cohorts before and after CLEAR
Finally, comparison of cohort grades for two years prior to adoption of the CLEAR approach, and the two subsequent years suggest that performance has improved, as is illustrated in the following graph:
