ESRC Teaching and Learning Research Programme
Discussion Group
What might count as evidence of critical thinking in teacher education courses?
Rebecca Soden and Effie MacLellan
Senior Lecturers at the Faculty Of Education, University Of Strathclyde
Paper presented at the ESRC Teaching and Learning Research Programme, First Annual Conference - University of Leicester, November 2000
Address for correspondence
Dr. Rebecca Soden
Senior Lecturer,
Faculty Of Education,
University Of Strathclyde,
Jordanhill Campus,
Southbrae Drive,
Glasgow G13 1 PP
Tel 0141 950 3155
(mobile 0780 846 1820)
Abstract
Assessment criteria for teacher education programmes commonly imply that students are to provide written evidence that they can think critically about course content. Research literature implies that critical thinking involves understanding how evidence relevant to claims might be evaluated and encompasses such competencies as constructing arguments for preferred practices, envisaging counter arguments others might offer and considering how such opposition might be rebutted (Perkins,1989, Kuhn, 1991, Bensley, 1998).
Preliminary content analysis of assignments from students on teacher education courses revealed a very low incidence of critical thinking in the sense outlined above. This analysis suggested that tutors' (and students’) conceptions of good thinking about learning and teaching may be different from those in the critical thinking literature: high grades were awarded to students who wrote about how theories could be applied in teaching but who provided very little evidence of the ‘critical thinking’ competencies outlined above. It seems likely that teachers will be more effective in encouraging good thinking if they (and their students) confront their conceptions of critical thinking.
Discussion
The discussion focuses on problems in deciding what might count as critical thinking about learning and teaching. Exploring these problems involves exploration of the nature and purpose of theory in teacher education programmes. The overall aim is to begin to clarify conceptions of what it might mean to think critically about learning and teaching.
Starter questions
1.What might it mean to think critically about teaching and learning?
2.To what extent does thinking critically about teaching involve exploring the adequacy of theoretical positions cited in support of preferred practices?
3.What are the implications for teacher education (and other professional courses)?
4.What are the questions for research?
BACKGROUND PAPER (not for citation, meantime)
Introduction
The case has been well made that curricula which help learners to think effectively about whatever they are studying are likely to raise attainment in ways which enhance students’ ability to be effective employees and citizens (Perkins, 1983; National Committee of Enquiry into Higher Education, 1997). Several of the ESRC TLRP projects focus on enhancing students’ ability to think and learn (e.g. Sustainable Thinking Classrooms; Learning How to Learn - in Classrooms, Schools and Networks). McGuinness (1990) distinguished three strands in what has become an increasingly influential literature about what it might mean to think well and about what teachers might do to help students to do so. One of these strands was critical thinking.
The notion of core and key skills in school and post-school curricula draws on ideas in this literature. Teachers are now required to teach core skills in all Scottish 'Higher Still' courses (Higher Still Development Unit 1996). The English National Curriculum includes key skills. There is a similar requirement in all programmes in the further education sector leading to General National (or Scottish) Vocational Qualifications and key skills initiatives are proliferating in the higher education sector (Drew, 1998).
Critical thinking: literature outline
While acknowledging the importance of other strands in the literature about thinking, this paper specifically draws on the strand commonly described as critical thinking. (Those ideas are reflected in the coding scheme in Appendix 1.) This strand connects well with descriptions of general criteria in many teacher education course documents and with traditional notions about purposes of higher education. Underlying some of this literature is an assumption that critical thinking is a generic ability which can be used in many contexts beyond those in which it was learned (Brookfield, 1987; Ennis, 1987; Kuhn, 1991; Paul, 1987). Other conceptions of critical thinking imply that knowledge and thinking are more inter-dependent than is implied by conceptions of thinking as a transferable skill (Bensley, 1998; Drew, 1998, Hyland and Johnston, 1998; Bonnett, 1995; Gardner and Johnson, 1996).
Although there is serious debate in this literature about how knowledge influences thinking, there seems to be consensus that critical thinking involves understanding how evidence relevant to claims might be evaluated and that it encompasses such competencies as constructing arguments for preferred positions and practices, envisaging counter arguments others might offer and considering how such opposition might be rebutted (Perkins,1989, Kuhn, 1991, Bensley, 1998). Research suggests that such thinking is the exception in the adult population.
When Kuhn (1991) asked a sample of American adults, including college graduates, to explain causes of three social phenomena (school failure, unemployment and recidivists' return to crime) she found weaknesses in their 'skills of argument': poor differentiation of theory (which included personal theory) from supporting evidence and frequent generation of 'pseudo-evidence' (richly elaborated plausible scenarios that restate the theory rather than provide evidence for it). Perkins (1983; 1989) contrasted a ‘makes-sense epistemology’ with a ‘critical epistemology’. People working with a ‘makes-sense epistemology’ tend to use the criterion of whether a given proposition makes intuitive sense whereas people working with a ‘critical epistemology’ understand the pitfalls of justification and try to test their arguments against possible objections and modify them until they become more robust.
From their further education college research sample, Anderson, Howe, Soden, Halliday, and Low (2001) reported that verbal behaviours indicative of critical thinking were very rare in Social Care students’ assignments. Bloomer's (1998) research suggests that, although there was much student-centred activity in courses leading to the General National Vocational Qualifications, it rarely included critical enquiry. These findings resonate with Harvey, Moon, Geall and Bower’s (1997) report that UK employers were not convinced that employees with Level 3 General National Vocational Qualifications were able to engage in critical, analytic thinking about their work. In two of the studies reported below, although the students discussed throughout one module how they might justify their practice and knew that critical thinking would be valued in assignments, such thinking was rare.
What is being counted as evidence of critical thinking in teacher education courses?
Since teachers (and others working with people) are now expected to help students to learn to think well, it is not surprising that assessment criteria for teacher education courses commonly contain phrases which resonate with descriptions of critical thinking in the literature outlined above. Often, general criteria in teacher education course handbooks include: ‘analysis’, ‘evaluation of ideas’, ‘extensive critical coverage’, ‘cogent and sustained justification of a standpoint’. Plainly such qualities are demonstrated in relation to some sort of content. Typically, assessment criteria require that students are to embed their work in ‘theory’, in itself a problematic notion. The presenters’ and their colleagues recently carried out a preliminary content analysis of small samples of teacher education students' assignments from three courses which include similar general criteria. For each of the courses, provided in different universities, the general criteria included statements similar to those quoted in this paragraph. In each course, ‘theory’ seems to have some of the meanings discussed by Chambers (1992): an accumulating body of knowledge, close to grand theory, and a cluster of meanings surrounding the idea of a hypothesis, model, or heuristic.
MacLellan and Soden (in preparation) analysed 20 assignments written to meet the requirements of one double module about learning and teaching in a course leading to a post-graduate award for experienced, qualified teachers. This sample included all the assignments from one cohort. The 20 assignments had been written by students who had discussed throughout the module how they might justify their practice. They knew that such justification would be highly valued in assignments.
All of these 20 post-graduate students managed to use relevant literature to construct a case for changing aspects of their practice. However, very few of these students envisaged any counter argument and even those students did not manage to provide rebuttals of the counter arguments they did envisage. These last two competencies are central features of Kuhn’s 1991 description of ‘the skills of argument’. Very few of this cohort evidenced any thinking about the theoretical frameworks they cited: problematic assumptions and other possible limitations were left unexamined.
Drawing on the Perkins' research cited above, it might be said that almost all these post-graduate students were thinking with a theory rather than thinking about a theory. The students in this study demonstrated far greater understanding of powerful theoretical frameworks than the under-graduate students in the other two studies and they were much more competent at suggesting implications of particular theories for their practice, but they rarely questioned the limitations of the theories. What made these assignments seem good was the comprehensive account of theories and of what theories might have to say about their practice. However, the incidence of features included in descriptions of critical thinking in the literature is almost as low as that revealed by the preliminary analysis of data from the other two studies outlined below.
The following examples taken from the initial analysis of the assignments by MacLellan and Soden (in preparation) give a flavour of what was taken to be some indication of critical thinking:
Dispositions are described by Greeno et al( 1999) as the habits of mind which control how we put our beliefs to work. So powerful are these self-beliefs in our ability to master a wide range of skills suggest Sloboda et al (1999), that they provide better indicators of future success than even perceived IQ will. A belief in being good at something motivates and encourages someone to persist. Such views are consistent with Attribution Theory which is concerned with the explanations people give for their successes and failures.
This was counted as an example of constructing an argument for a preferred position.
- Research evidence (Butler, 1998) has shown that even pre-school children bring problem solving ability to school. This view is in contrast to Piaget’s suggestion that children at this age are egocentric and therefore unable to consciously reflect on the difficulties of a problem and resolve the apparent conflict.
2.A study by Butler (1998) described an approach used to promote self-regulation in post-secondary learners with learning difficulties, the Strategic Content Learning Approach (SCL). This was achieved by monitoring their own successful learning in discussion with tutors. This approach differed from other instructional models such as those described by Harris (1996) in that the learners were not taught specific strategies but devised their own.
These were counted as examples of envisaging counter arguments.
There were no examples of rebutting the counter argument.
The following exemplar is a discrete section in one of the assignments.
POSSIBLE ADVANTAGES OF SETTING
a)Lessons and resources need only be prepared for one group of children.
b)The teacher is able to give concentrated input throughout the maths session and children are not given ‘time-fillers’ in order to free up the teacher’s time to work with other groups.
c)A good, consistent pace of learning can be sustained, as the whole group is more or less at the same level.
d)Techniques and instruction can be more readily adapted to meet the needs of the group.
e)Interest and motivation are sustained because more able children are not held back by less able pupils.
f)Less able pupils take more part when not affected by those who are much brighter.
g)Individual and small-group instruction for children who are struggling with a particular concept becomes more feasible.
While this list might be regarded by some as arguing for a preferred position, we would want there to be some evidence for each of the points before counting it as such. Furthermore, there was no list of counter points to promote a critical view.
MacLellan and Soden explored possible reasons for the low incidence of what, according to research literature, counts as critical thinking. One possibility was that the students did not construct a meaning similar to that of the tutors, although the tutors had modelled what they counted as critical thinking throughout the module. It is possible that the students found it difficult to co-ordinate several challenging task demands: understanding theoretical frameworks, thinking about applications ofthese frameworks in their practice and thinking critically both about applications and about the adequacy of the frameworks themselves. As a follow-up to this study, MacLellan and Soden are gathering data on lecturers’ conceptions of critical thinking about learning and teaching. They plan to gather similar data on students’ conceptions and to use these data for development programmes.
Soden, Pithers and Hunter (in preparation) analysed 40 assignments from one B.A. teacher education programme. In the sample were eight assignments for each of five different modules which formed a group in the programme. The eight assignments for each module were randomly chosen from a total of 25-30 assignments submitted for each module. The (still provisional) coding scheme (see Appendix 1) reflected ideas in the critical thinking literature outlined above. These ideas are broadly similar to those in the wider literature on learning and teaching in higher education. In addition to 'strong' literature driven codes, 'lenient' codes were included to capture content which met even the most lenient description of critical thinking. In addition there were codes to account for data which did not fit any of the critical thinking codes. These included codes to capture students’ descriptions of theories, resonance of theory with students' personal experience and examples of how the theories might be applied in classrooms. An initial reliability check showed that acceptable reliability levels could be achieved. Some of the codes used in the analysis are shown in Appendix 1 at the end of the paper.
Every statement in each student’s assignment was coded. The total number of statements were recorded as well as the percentage of statements allocated to each code. Overall, less than 20 per cent of statements in the sample were allocated to codes describing critical thinking; this includes codes reflecting lenient definitions of such thinking. Students whose scripts exhibited a low incidence of statements often were awarded high marks and often the feedback notes included enthusiastic congratulations from tutors on an 'excellent' performance. These marks counted towards classification of honours degrees. Inspection of tutors’ comments suggest that high marks were being awarded to students who wrote well about how theories could be applied in teaching. There is little reason to suppose that it is unusual in similar teacher education and in other professional courses to award marks in first class bands to students who provide very little evidence of the sorts of critical thinking described by the codes used in this study.
Soden, Ross and Allison (in preparation) analysed 22 assignments written to meet the requirements for one module about learning and teaching in a course leading to a Certificate in Education. The 22 assignments had been written by students in the post-school sector who had discussed throughout the module how they might justify their advocacy of teaching in some ways rather than others and what might count as justification. These students knew that such justification would be highly valued in assignments.
A first reading of the transcripts indicated that codes which reflected notions of critical thinking in research literature would not account for much of the data. In an attempt to develop understanding of how participants justified their practices, the 89 pages of text generated by the teachers were subjected to a form of inductive content analysis. In total 971 meaning units were extracted from the participants’ writing. Following Tesch (1990) a meaning unit was defined as a segment of text that is comprehensible by itself and contains one idea, episode or piece of information. The reliability levels achieved were acceptable for each section of the meaning units (Kappa co-efficients for each section ranged from .82 to .92).
In this sample, 567 of the 971 meaning units (59 per cent) which were extracted from the assignments were described as justifications for preferred ways of teaching. The remainder of the meaning units contained descriptions of teaching/learning methods to which the justifications related. Typically, the justifications took the form of citing research evidence to support a preference. Of the 567 units classified as justification only 52 units (9 per cent) mentioned anything problematic about the research they cited or about the assertions they made. Teachers who made the highest number of references to research were no more likely to mention such matters.