Peer review of learning and teaching: the grass roots translating policy into innovative and vibrant practice.
Clare Kell, Beatrix Fahnert, Victoria James, Keren Williamson.
Cardiff University.
Background and introduction
In response to calls from the Dearing Report (NCIHE, 1999), Quality Assurance Agency, funding bodies and the Higher Education Academy (HEA), Higher Education Institutions (HEIs) have been keen to demonstrate the effectiveness of the student learning experiences they offer. Focus on the student learning experience has placed emphasis on the teaching environment within which the learning is facilitated. Many HEIs operate the dual systems of student evaluation and feedback on their learning experiences, and colleague / peer review or observation of individual learning events or experiences. These latter peer-led sessions have formed the basis of much discussion both in the literature and in local academic departments, with rationale ranging from audit / judgmental purposes (QAA, 1996; Shortland 2004), to those focussed on dialogue for mutual benefit and enhancement (Martin and Double, 1998; Orland-Barak, 2005).
Developing a new approach
Following the merger of two HEIs, a Change Champion: Peer Review of Teaching was seconded to negotiate and develop a new policy that aligned with the identity, vision and strategic plan of the new institution. A year of observation, discussion, negotiation and piloting created the Peer Review of Learning and Teaching (PRLT) Policy Framework that was accepted by University Senate in February 2006. Staff and student involvement was crucial in developing an innovative approach to PRLT that valued the whole ‘teaching’ role, was inclusive of all staff who contributed to the student learning experience and was framed about the notion of a mutually beneficial evidence-based peer-assisted reflection. Output from the process would meet the needs of QAA, funding body and institutional audit systems and provide evidence that might be useful to inform staff promotion, appraisal, accreditation and other portfolio-building needs. Despite a positive ‘launch’ and initial flurry of activity, the policy quickly joined others on the shelf of well-intended policies that ticked boxes but had failed to turn from rhetoric into embedded practice.
Reinvigorating the project
New academic staff are required to undertake a postgraduate learning-to-teach module that involves a series of PRLT cycles with both discipline-specific mentors and cross-disciplinary cohort peers. Moving from the Change Champion role to the Programme Manager enabled the ‘new’ PRLT to be practised, experimented with and developed among a broad spectrum of junior and highly motivated colleagues. Working with each other, their mentors and their students these staff are pioneering PRLT across the University. Seeing PRLT not as a ‘meaningless chore standing outside [their] practice’, but as an opportunity to value and label as ‘discussible’ a whole range of ‘teaching’ activities, the staff have engaged their students and colleagues in dialogue about teaching evaluation practices and evidence creation to facilitate meaningful individual and team reflection. An evidence-based approach has prevented the peer-assisted reflections becoming judgmental or cloning in nature and has fostered evidence-based discussion, problem-solving and sharing of practice within and across academic disciplines. Widely recognised as one of the most positive elements of the Module, PRLT has resulted in shared teaching across disciplines, the undertaking of small-scale, inter- and cross-disciplinary enquiry projects and a developing understanding of the diversity of practice and contexts across the Institution.
With the number of participants on the Module, and thus the network undertaking PRLT, mushrooming and sprawling across the University, participants are in a position to effect real cultural change. Reflecting on their experience in a variety of roles within the PRLT process, participants have identified the following elements as essential for achieving a mutually-beneficial, non-judgmental outcome:
1) negotiation of process ownership;
2) overt discussion about individual’s perception of PRLT and their past experience of other ‘schemes’;
3) discussion of each other’s underlying rationale and conceptions of learning and teaching;
4) creativity in collecting appropriate evidence upon which the PRLT dialogue will be based;
5) dialogue framed about questions to facilitate reflective practice;
6) ability to use the output in several different ways.
The following section explores each factor in turn while recognising their interdependence.
Turning the rhetoric into useful reality
1) Ownership
Giving staff ownership of PRLT is seen as recognition of the autonomy with which academic staff carry out many of their roles (Shortland, 2004) and is likely to result in the development of a flexible process that could be tailored to individual / local contexts and needs (Blackwell and Preece, 2001). While recognising that any process has to operate within the framework of local and Institutional regulations, facilitating local and individual ownership is seen as a critical first step to embed PRLT in practice. Exploring these issues in the study institution suggests that nomenclature is a key element in the ownership process. While PRLT at Institutional level assumes that R = ‘Review’ (an assumption based upon the Institutions’ perception that, in a research-led environment, ‘review’ reflects developmental, non-threatening feedback), at local level there has been a need to deconstruct that term in relation to the norms and dialogue of each discipline. As a consequence, while the PRLT process is adopted across Schools, the ‘R’ may stand for review or reflection or simply represent a shared understanding of a process of ‘collaborative reflection’, ‘support’, ‘peer-assisted reflection’, ‘reflective dialogue’ etc. What has been important therefore is the local deconstruction of the spirit of the PRLT policy and its translation into meaningful local practice for local ownership and development to align with opportunities and challenges in the specific context.
2) Who am I and how do I teach?
When the initial PRLT process was in its pilot phase (Kell and Lloyd, 2006), it became clear that, in order to embark on an evidence-based peer assisted reflection of a learning opportunity, it was essential that each participant was able to articulate their own philosophical approach to teaching and their consequent conceptions of and approaches to both learning and teaching (Rosenfeld and Rosenfeld, 2004). Such self-analysis was seen as an opportunity to understand the perspective from which each participant was engaging in the PRLT process in order to facilitate curiosity in different teaching approaches while reducing the likelihood of the PRLT dialogue becoming cloning and judgmental. Specific resources that have been found to be useful in this reflection process are the work of Samuelowicz and Bain (2001), Rosenfeld and Rosenfeld (2004), Rezler and French (1975) and Wheeler and Marshall (1986).
While we of course acknowledge that such deep reflection is not appropriate for all PRLT participants, ‘getting-to-know-myself’ (what is my teaching philosophy, what teacher and learner type am I, what previous experiences do I have with peer review and with what might be reviewed, what were the results of my reflection about it, what focus would I like for the observation, what evidence/ data to inform my practice do I need, what are my expectations) is an important precursor to an initial ‘getting-to-know-each-other’ and how and why we teach dialogue. In practice participants are encouraged to engage in reflection at least at the following stages of the process:
· The reviewee (owner of the PRLT focus) analyses themselves to explore possible review foci that would be useful to their teaching development;
· The reviewer analyses themselves to identify common ground and differences and how that will affect the ensuing evidence-collection and reflective dialogue.
3) Perception of the process
Blackwell et al (2001) and Gosling (2002) recommend that departments help staff verbalise their perceptions about a PRLT process in order to make overt and then clarify issues of urban myth and reality. Expectations of the value of PRLT arise from past experiences of related teaching dialogues and perceptions of the principles underpinning the activity and the reasons for its undertaking. Each of these factors has a significant impact on an individual’s approach to the PRLT process and should be overtly considered within the PRLT pairs/ triads teams etc in addition to the ‘whole department’ discussions about the process that may have already taken place. While we would advocate this ‘step’ in the process for all participants, it is particularly important when new staff are being integrated into the culture of the department. This observation was especially striking in one PRLT pilot site (that drew new staff from health-related practice) where new staff had experienced similar-termed processes that were undertaken by ‘seniors’ with a specific competency-based and judgemental ethos (Kell and Annetts 2005, unpublished data). A useful tool to facilitate these discussions and unpack perceptions is the POT Model Grid (Gosling, 2002) with staff being invited to read and circle (in different colours) the words that match their past experience / current perception and those that represent their more ideal process.
Using the Gosling Grid or other similar resources has led PCUTL participants to recommend
· that participants should be able to articulate their personal teaching philosophy and understand the perspective of their partner(s);
· that participants recognise themselves and each other to be part of a PRLT team / pair / triad with different tasks/ roles and perceptions of the task at that time;
· that participants need to think about, identify and communicate the perspectives, skills, strengths and weaknesses they bring into the PRLT team in support of the task, and that these tasks/ roles can change over time (within a continuing peer-partnership and/ or within a community of practice to disseminate good practice). PRLT should therefore be placed in its unique context and not within an existing power / departmental hierarchy structure;
· that the focus of the evidence-collection and ensuing peer-assisted reflective dialogue need to be negotiated in advance and acknowledge the differing perspectives on the issue brought to the context by the participants.
In summary therefore, this element of the PRLT process results in the participants identifying and / or creating a common language and understanding relevant to their needs and context. While we would argue that such an understanding is essential whatever the PRLT foci and whoever the peers involved, it is particularly important for reviews framed about teaching observation and assessment / marking / feedback practices and those reviews undertaken across courses, disciplines and Schools / Faculties.
4) Collecting the evidence
Fundamental to the achievement of a mutually beneficial peer-assisted reflection of a learning activity is the collection of meaningful, robust, non-personal and relevant evidence (Pill, 2005). Shortland (2004) describes a common mismatch between PRLT rhetoric and practice when staff use closed question-, ranking-based forms of evidence to support formative rhetoric. The use of paperwork, or not, therefore is an important choice as checklists or prescribed ideas of what should be reviewed do not enable flexibility, can be judgemental and may not be aligned with the PRLT philosophy. Although many institutions will require some form of process completion confirmation, data collection tools can be individually and creatively designed by those involved to meet their specific process outcomes and goals.
Key questions for overt consideration by PRLT partners are:
· what is the focus of the review and what specific questions / issues are we hoping to explore?
· what ‘research’ skills do we have that we can use in this context? E.g. what form of evidence could be collected to address these questions? How will confidentiality and data protection issues be addressed?
· if the evidence is in the form of documentation (e.g. module outlines, subject benchmark indicators, assessment criteria etc) how are we going to interrogate the evidence to provide specific evidence for the ensuing discussion?
· and how will the data be presented as a resource for facilitating reflective practice?
Experience suggests that creativity at this phase of the PRLT process can have a major impact on the outcome. Indeed Shortland (2004) suggests that PRLT is a perfect opportunity to break down the assumed ‘no-go areas’ of teaching practice recommending instead that partners explicitly consider how to take peer review beyond the standard classroom/lecture hall based observation format. Within our research-led institution, the access point for making PRLT relevant to staff has been the overt linking of PRLT to the research process. For example, where observation of an innovative teaching practice-in-action is the focus of the review, packaging the reviewer’s role as one of ‘data collector / researcher’ has had a powerful empowering effect. Such has been the creativity, that PRLT has been effectively and robustly carried out with module design, assessment marking and feedback, media resources, PowerPoint presentations, fieldwork support etc all regularly featuring as foci. Examples of evidence collection ‘tools’ have included session transcripts, student: teaching activity grids and group working participation maps.
5) Facilitating reflective practice
Many authors advise that reflective practice for the mutual benefit of both the reviewer and reviewed is an essential component of any PRLT process designed to enhance and value learning and teaching diversity (Hammersley-Fletcher and Orsmond, 2004; Beaty, 1998). Reflective practice based dialogue, occurring in an environment of trust and mutual ownership, will help both parties ‘unpack’ their practises as instinctive teachers (MacKinnon, 2001) and go beyond the assumptions both take for granted in their approaches to teaching (Pill, 2005; Paris and Gespass, 2001).
These concepts manifest within the process of PRLT as specific behaviours that can be seen as positive or negative responses to the aforementioned predetermined beliefs and perceptions. Korthagen and Vasolos (2005) note that reflection undertaken by teachers commonly focuses on aspects of environment, behaviour, competency and beliefs and concur with Calderhead and Robson (1991) that these beliefs are often deep-rooted and persistent. Where these beliefs manifest in negative perceptions and behaviour, it may be assumed that change will be difficult to manage.
In practice creativity may be the answer, looking for ‘bottom up’ influence rather than ‘top down’ directives, identifying the value of PRLT in order to change perception and modify behaviour. Creativity, becoming more open to the advent of new possibilities in teaching practice, is facilitated by the process of core reflection (Korthagen and Vasolos, 2005). This process looks more to the inner circles of the ‘onion model’ of reflection (Korthagan, 2004) namely identity and mission, and seeks to identify ideals and limitations within the environment of teaching and learning. Core reflection is pivotal to the success of PRLT in identifying these constructs and actively promoting a positive, supportive environment for personal development and institutional change thereby improving the students’ learning experience.