BERA 2005.
Evolving diversity within a model of peer observation
at a UK university.
Ian M. Kinchin.
King’s College London.
E-MAIL:
Paper presented at the British Educational Research Association Annual Conference, University of Glamorgan, 14-17 September 2005
ABSTRACT
A formal programme for the peer observation of teaching was introduced across a UK university during the academic year 2003-4. This process has been evaluated across the 10 schools that constitute the university. The teaching and learning coordinators within each school were consulted about modifications to the basic model presented. Academics (below the level of Head of Department) were then interviewed about their perceptions of the process. From these interviews, seven themes emerged, representing choices made by departments when implementing the programme: efficiency vs. effectiveness; anonymity vs. focus; formative vs. summative; formality vs. informality; frequency of observation; equal vs. unequal partners and teaching vs. research. These themes were then described and given back to the 10 learning and teaching coordinators and the 10 Heads of Schools for their responses. The themes are described with reference to Gosling's (2002) models of peer observation, and Fullan's (1991) 'insights to change'. Patterns of choices made by departments may help them to negotiate the fear generated among academics by peer observation. A model is presented to summarise the central role of observation in university teaching, which considers the role of peer observation at varying levels: the students, the department, the institution and the profession. The implication from this work is that evolving diversity within the model can be viewed as a strength of peer observation, allowing departments to take ownership of the process and develop their own agenda for professional development. It is suggested that for the further development of this model, interdisciplinary peer observation may help to escape the restrictions created by the common focus on the content of teaching.
______
Introduction
This paper examines the introduction of a formal system of peer observation of teaching within a research-led university in the UK. The decision was taken to implement this programme within the academic year 2003-4. This was due, in part, to meeting external pressure of the quality enhancement agenda as well as meeting the university’s commitment to maintaining high quality teaching. The implementation of a peer observation of teaching scheme within the institution was a collective decision, which was ratified at the highest levels within the institution. Its aims were to engage as many staff in raising awareness to the processes, as well as, the content of teaching. This was considered to be a positive move to understanding the variety and complexity of teaching strategies and process engaged within the institution. Initially it was envisaged that the outcomes of the process would help the institution identify areas of good or excellent practice so that they could be disseminated across the institution. Equally important was that it could produce a mechanism by which developmental needs could be identified in a secure and supportive environment.
The peer observation of teaching (sometimes referred to by colleagues as ‘peer review’) as described in this paper, is defined as an intentional process of observation in which a university teacher (lecturer) sits in on a teaching session of a colleague with the express intention of offering feedback as a ‘critical friend’. In the context described herein, this is not linked to staff appraisal, does not contribute to a postgraduate teaching certificate, is not ‘graded’, and is not restricted to ‘new’ lecturers.
In preparation for the introduction of peer observation, a series of seminars was offered to staff. Over 500 members of staff attended these seminars, where discussion focused on possible models that could be adopted. The models put forward by Gosling (2002), (Table 1), were used as a focus for discussion and analysis.
Table 1
Models of Peer Observation of Teaching (after Gosling, 2002).
Characteristic / evaluation model / development model / peer review modelWho does it & to whom? / Senior staff observe other staff / Educational developers observe practitioners; or expert teachers observe others in department / teachers observe each other
Purpose / Identify under-performance, confirm probation, appraisal, promotion, quality assurance, assessment / Demonstrate competency/improve teaching competencies;
assessment / engagement in discussion about teaching; self and mutual reflection
Outcome / Report/judgement / report/action plan; pass/fail PGCert / Analysis, discussion, wider experience of teaching methods
Status of evidence / authority / expert diagnosis / peer shared perception
Relationship of observer to observed / power / expertise / equality/mutuality
Confidentiality / Between manager, observer and staff observed / Between observer and the observed, examiner / Between observer and the observed - shared
within learning set
Inclusion / Selected staff / Selected/ sample / all
Judgement / Pass/fail, score, quality assessment, worthy/unworthy / How to improve; pass/fail / Non-judgemental, constructive feedback
What is observed? / Teaching performance / Teaching performance, class, learning materials, / Teaching performance, class, learning materials,
Who benefits? / Institution / The observed / Mutual between peers
Conditions for success / Embedded management processes / Effective central unit / Teaching is valued, discussed
Risks / Alienation, lack of co-operation, opposition / No shared ownership, lack of impact / Complacency, conservatism,
unfocused
The pros and cons of each model were discussed within the seminars and it was agreed that the model described by Gosling as the ‘peer review model’ would be most appropriate. Staff were also given guidelines about how to review colleagues and a standard format for the supporting paperwork was provided and described. A three stage format was supported. A pre-observation meeting in which the reviewer and reviewee could discuss the session to be observed and issued that might be anticipated. This was followed by the observation itself and then a post-observation meeting in which the merits of the teaching session would be discussed. These discussions would be confidential between the pair and the paperwork would be the property of the lecturer being observed. An additional sheet of paper would record that the staff member had been reviewed and would be passed to the peer observation coordinator within each department. Organizing the mechanics of implementation was devolved to the departments across the university.
It was anticipated that staff would offer some resistance to the idea of peer review at the outset as it would be seen by many as a change to their normal practice. Comments were made about the time the process would take and the unclear nature of the benefits that participation would bring. Though it was never articulated explicitly by the staff at the seminars, there may well have been anxiety about embarking upon a process in which colleagues’ teaching (traditionally a private activity) is suddenly made more public. Such anxiety has been described by Atwood et al. (2000) as a major hurdle to overcome when implementing peer review:
… it appears that fear is one of the most compelling reasons to forestall the implementation of peer review. How ironic that disciplines that pride themselves on the peer review of their research … can let peer review of teaching be so immobilizing!
(Atwood et al., 2000)
This fear combined with the strong traditions of teaching as a private activity, have provided considerable resistance to the universal acceptance by university staff of peer observation as a means of developing teaching skills.
The use of peer observation of teaching is well documented, especially in Australia and the USA. Leading in the discussions of peer review have been such authors as Schulman (1987; 1998) and Hutchings (1994) from the Carnegie Foundation, and Angel Brew’s work on professional development (eg. Brew and Boud, 1996).
Hutchings (1994) recognised four arguments for engaging in peer observation of teaching:
· Student evaluations of teaching, though essential, are not enough; there are substantive aspects of teaching with which only faculty can judge and assist each other.
· Teaching entails learning from experience, a process that is difficult to pursue alone. Collaboration among faculty is essential to educational improvement.
· The regard of one's peers is highly valued in academia; teaching will be considered a worthy scholarly endeavour--one to which large numbers of faculty will devote time and energy--only when it is reviewed by peers.
· Peer review puts faculty in charge of the quality of their work as teachers; as such, it is an urgently needed (and professionally responsible) alternative to more bureaucratic forms of accountability that otherwise will be imposed from outside academia.
These ideas were promoted during the staff seminars described above. Some members of staff were clearly aware of the potential offered by peer observation. As one member of staff put it:
From the individual lecturer’s point of view, peer observation is by far the most informative form of feedback, particularly as nuances can be discussed and the exact manner of teaching delivery and its anticipated outcomes can be decided beforehand in a bespoke manner for each individual observation.’
______
Methods
This research takes the approach of hypothesis-generating rather than hypothesis testing (Brause and Mayher, 1991). Initial discussions were undertaken with Teaching and Learning Coordinators (members of academic staff with a responsibility for coordinating teaching and learning activities within the Schools). This helped to focus on key issues and allowed the tentative identification of emerging themes.
The next step was to conduct interviews with colleagues across the college. Descriptions given in the research literature of attempts to achieve blanket coverage of staff within an institution have been met with very low response rates (eg. Closser, 1998), making efforts to achieve generalisability non-viable. It was therefore felt to be more important to focus on the quality of data gathered rather than the quantity of data. Data gathered for this study has been sufficient to confirm and saturate categories (ie. issues of concern regarding the process of peer review across the college), generating an increasingly stable agenda to guide subsequent data collection (Guba and Lincoln, 1989). Further sampling confirmed the findings. Therefore, in deciding upon the size of the sample of staff consulted, there is simply a trade-off between generalisability and practicality. Interview data were collected from 20 colleagues (academic staff with a range of experience across the Schools – all below Head of Department level). Quotes from these are used below to illustrate points throughout the text.
Themes were then described and presented to Heads of Schools (and back to Teaching and Learning Coordinators) for their consideration, making the total number of academic staff consulted 40.
This research is designed to raise awareness of issues to be resolved, rather than to suggest a universal solution that will fit across the university or could be applied to other institutions. However much a researcher is aware of regularities and themes across the schools (when viewed from the outside), those within the schools focus on perceptions of uniqueness that make one school’s needs feel different to those of the next (ie. the view from within a discipline is unavoidably distorted). Disciplinary styles influence the way scholars approach learning and teaching just as it does their research methodology and perspectives of learning, (Marcus, 1998). A disciplinary style comprises, at its core, what Schwab so elegantly distinguished as ‘substantive and syntactic structures: the “conceptions that guide inquiry” and the “pathways of enquiry” (scholars) use, what they mean by verified knowledge and how they go about verification’ (1964: 25, 21). What Schwab is intimating is that disciplinary style influences the problems academics choose to engage in, the methods used to explore the problems and the nature of the arguments that develop from those explorations
The aim of this research is not to compare departments or conclude that one department runs a better observation programme than the next. Rather the point is to identify and illustrate the evolving diversity within the college that has arisen as a consequence of choices made. Whether these choices were made consciously or subconsciously, by raising the profile of these choices, it is hoped that departments will reflect upon them and use these reflections to justify the direction of future developments, enabling peer observation of teaching to make its contribution to ‘enhancing the student experience’. All interviewees were guaranteed total anonymity and so individuals (their status and departments) are not identified.
______
Themes.
Efficiency vs. effectiveness
In applying the model of peer review, departments have each made a choice between having a small team of reviewers within the department (‘development model’), or having everyone act as reviewer and reviewee (‘peer review model’). The use of a group of ‘specialist reviewers’ has been adopted in some departments and has allowed them to ‘complete’ the process quickly. This view of completion seems to overlook the developmental intention of the process. The research literature suggests that such an approach can be improved by rotating the group of staff who are trained as observers so that more of the staff within a department are involved in the process (Hammersley-Fletcher and Orsmond, 2004).
This specialist reviewer adaptation (allied to Gosling’s ‘development model’), may help to achieve consistency within the process, particularly if an appropriate discourse of peer review develops among the reviewers within (and possibly between) departments. However, time for such dialogue does not seem to have been given a high priority, and the absence of such a discourse may have an isolating effect upon the reviewers.
I can’t comment on what happened in any of the others, because I haven’t spoken to any of the other reviewers. It might be sensible for us to have a little session between us.
Such an approach also loses one perceived benefit to most members of the department – that of observing others teach as part of a mutual learning experience. This is seen to be of particular importance (and interest) to new and inexperienced lecturers who would like to see how others do it:
we will often take one of the younger, newer people in the department and send them in to review someone like X, for example. He is a star man … magician. He’s an excellent lecturer. Therefore the idea is that people can go in and learn from good lecturers.
Application of this model also implies that the process can be completed and set aside, as an adjunct to normal teaching rather than as a part of it: