Understanding Student Disengagement in Peer-Instruction Classrooms.
Steven Pollock
Dep't of Physics, CU Boulder, CO 80309
Abstract: Peer instruction techniques in large-classrooms are demonstrably effective in terms of improved student learning, but simple flaws in implementation can have a noticeably negative impact on the effectiveness of the technique. Through a variety of student surveys and interviews, we investigate and categorize the underlying cognitive and procedural issues which limit the effectiveness of peer instruction methods, and suggest classroom strategies at a variety of levels to improve them.
Physics Education Research [e.g. Arons97, Hake98, Hestenes92, McDermott91, 93] argues compellingly that traditional large lectures can fail with regard to student attitudes about physics, classroom engagement, and performance in tests of conceptual underpinnings. A promising approach advocated by Eric Mazur's group, and others, involves "Peer Instruction" [Fagen02, Mazur97, Meltzer02, Wyckoff01] which involves frequent in-lecture multiple choice conceptual questions, with paired ("peer") discussion and votes. Mazur's research [Crouch01] has focused on the resulting gains in conceptual understanding. However, at our institution, the half dozen faculty experimenting with these interactive engagement classroom techniques have been uniformly disturbed by a noticeable fade in student participation with time. We have begun an action research program to investigate this fade, in order to provide a theoretical basis for improvement of this promising classroom technique. Our approach is largely survey based, which at a minimum provides rich and detailed student comments. Our results are meant to guide further investigation and suggest possible interventions.
Our data came from two large introductory physics courses: an algebra based Electricity and Magnetism course (Fa'01, 136 students) aimed at health science majors, and a calculus based course (Sp'02, 307 students), aimed at engineers. We used a "colored card" method of student voting: multiple choice answers are colors, and students raise an index card to vote. This provides a quick visual survey of the class, with minimal expense or effort. Using electronic feedback can have a positive effect on student participation, but adds significant costs to both students and departments. Since many schools do not yet have this option, we consider it important to investigate student participation effects without the benefit of expensive technology. Our data consisted of six distinct types: classroom observation, an end-of-semester anonymous survey with Likert-scale questions, end of semester quantitative assessment of conceptual learning, faculty interviews, focus groups of former students, and online open-ended "participation questions". The latter provided by far our richest source of information. More details about our data collection can be found elsewhere [Pollock03]
The bottom line is the following: in all cases we studied where peer-instruction was implemented using colored cards (rather than electronic feedback), student participation tended to fade with time, and faculty were uniformly discouraged by their perception that the classroom dynamic was changing with time. However, our student performance on CSEM (Conceptual Survey in Electricity and Magnetism [Maloney01]) exam questions exceeded the comparable base rate from the CSEM survey by an average of 20-30%, indicating that conceptual understanding was not objectively hindered by this classroom approach. Student surveys about the use of concept tests in lecture were also overwhelmingly positive, both in terms of "affect" (over 90% positive feelings about concept testing in lecture) and in their perception of the benefits of such a technique to learning physics (just under 2% "not beneficial" in both classes.) See Pollock03 for more details. Faculty interviews from our institution, and also from faculty nationwide via a web forum on peer instruction [Crouch02] tended to focus on the single idea that students simply get tired over time - that repetitive use of a single classroom technique is bound to "fade". Although it may be tempting to concede that the fade is thus just a result of unavoidable "human nature", our discussions with students brought out hints of deeper and more varied explanations of the phenomenon, many of which allow for classroom methods that could improve student participation levels and maintain the effectiveness of peer instruction.
Our open ended online survey questions provided the richest data, which allowed us to code and classify a variety of distinct effects which are involved in the fade. In the end, we have organized these into five categories, each of which has distinct implications for classroom approaches to peer instruction. The first category, "fatigue", is the primary assumption of interviewed faculty. Interestingly, almost no students specifically mentioned this in their written responses to our questions about their participation, and it was discussed only briefly and in passing during focus groups. There is no question that fatigue must play some role: when one uses the same classroom approach for 15 weeks the novelty certainly wears off. It should be no surprise that students who are increasingly stressed, studying for other classes, and suffering from persistent lack of adequate sleep will not show up as consistently for class, and will be more passive and quiet when they're there. If this is the sole issue, varying the pace and style of concept tests, adding different in-class activities, and creating novel elements to the course and activities as the semester progresses will all help. Nevertheless, this effect is quite difficult to mitigate entirely. Since the students still claim to enjoy and benefit from peer-instruction activities right up to the end of the semester, we have found the simple act of asking the students if they're tired and want concept tests to end can shake them up and reinvigorate their participation level.
The second category is "logistical" in nature. When using cards to vote in a class with hundreds of students, there is no easy mechanism for making their participation explicitly count. Individuals are quite anonymous, and students quickly deduce that it is not worth the effort required to talk and vote. One solution to this problem is switching to an electronic feedback device. These are becoming common, and dropping in price, but add their own logistical complications - one of which being that it takes longer to collect votes with a large audience response system than a quick flash of cards. Short of this expensive and complicated solution, we have begun experimenting with e.g. telling students that the online participation question will involve telling how their neighbor's solution differed from their own. Credit for these online questions is pass/fail - we don't try to grade 500 essays, but automatically give credit for any reply. Nevertheless, this encourages students to participate at least on those occasions.
We have developed another approach to the logistical problem of anonymity, which has also proven to be fairly effective. When one starts to notice some "fade" (or preferably even before) a variation on the voting procedure can be introduced. Students are required to get into small groups, "voting blocs", and the blocs are required to vote the same color. If they cannot agree on an answer, they have no choice - majority rules! With electronic feedback, students forced to vote "wrong" because of their neighbors then don't get credit, but even with colored cards, the frustration of having to vote "wrong" often induces students to participate more actively, argue more firmly, and defend their opinions.
Yet another important, and distinct logistical issue, is that of time allotted to group discussions in class. It is difficult to monitor the progress of a large number of small pairings or groups in a big lecture hall, and the tendency amongst most faculty is to end the discussion as soon as the sound level begins to drop noticeably. It did not occur to us that this had a negative impact until I began soliciting student written feedback, asking them e.g. "if you did not participate in class today, why not?" Fully 25% of student in the class spontaneously brought up the issue that they felt pressed for time during concept tests. Most argued that without enough time, it was frustrating and pointless to start a discussion, and many gave up and just waited quietly. This is clearly important to know about, and a solution may be to give more time earlier in the semester when the conversations are still more energetic. When using electronic feedback, this issue is easier to handle - the systems indicate how many students have replied, and if one allows the discussion to continue until some significant fraction have answered, the time problem may be self-regulated by the students themselves.
The third broad category of effects impacting the fade is "social issues". Several students in the focus groups argued that it was not considered "cool" to raise cards and participate actively. This is not part of the standard social climate in typical classrooms at our large public institution. In the beginning, many participated due to the novelty and excitement, but as time went on, many started to notice the small but growing group of "hold-outs" who don't participate. This fed on itself - the fewer people who talked and voted, the less socially acceptable it was to continue doing so. There were other, distinct social issues involved too. Most students informed me that they quickly became embarrassed talking to strangers if they were at all confused about a point. Sooner or later, there came a time when they didn't feel comfortable talking, and after that it was difficult for them to start up again.
One cannot easily change social constructs in a classroom, but being aware of them suggests some useful strategies. We now begin the semester with "ice-breaking" activities, in which students are required to find out something about their neighbors, including their names. This may not come naturally to physicists, but from our classroom observations it had a marked impact on participation levels. We now repeat such activities at regular intervals - students don't always sit in the same groups, and if attendance begins to dwindle later in the semester, people are physically farther apart and may suddenly arrive in class without their usual support structure of friends. One can place remote parts of the large lecture hall "off limits" late in the semester if attendance drops, to maintain a higher density, and actively encourage reticent students to speak to their neighbors. Singling out a student in front of the class is counterproductive, but quiet individual cajoling often brings students together into groups.
These social issues also provide fodder for what I call "classroom propaganda", the essential explanation and verbal motivations one provides the students with to encourage participation. Explaining why discussion with neighbors is a useful tool is critical. Mazur and others emphasize the necessity of explaining why we use these classroom techniques[Mazur97, Meltzer02]], but I didn't appreciate how important this can be until I began reading e.g. students statements that they refused to talk unless they "knew the answer", so as to avoid embarrassing themselves in front of strangers.
A fourth category is "epistemological" in nature. Not surprisingly, few students were able to articulate this idea explicitly in focus groups or online responses, but their replies often indicated they didn't yet understand what it means to learn physics. Unfortunately, the very nature of a multiple-choice question does not emphasize the idea that the process of discussion is important to learning. Why should any student bother with a discussion if the answer is all that matters? One must constantly remind students that articulation is an essential part of learning, that hearing alternative viewpoints can strengthen understanding, and that nobody is expected to know all solutions right away. It may be that the multiple choice format should not be the sole source of "in-class peer discussions", despite the ease of collecting the results. One can have students occasionally discuss more open ended, even subjective issues, and collect their work either by large-class "call out" techniques, or by having them write up their ideas in an online participation activity due after class.
Developing "how do you know" questions (which can still be multiple choice, or might dominate the important followup discussion to concept tests) is essential. If students are expected to be able to discuss their reasons on homework or exams, the in-class peer-discussion becomes more useful and meaningful to them. A related issue, explicitly mentioned by many students in their online writings, was their growing realization that they could just sit quietly during discussion periods and wait for "the answer". In response, I carefully chose occasions where I would not tell the answer to a multiple choice question. I let them know in advance - the question mattered, but the peer discussion was their only opportunity to work on it (besides office hours). This was a powerful motivator, especially when it became clear that those were more likely to show up on exams. Although this can be extraordinarily frustrating for some students, choosing questions that they are prepared to understand on their own, and explaining to them that they need to make sense of the material rather than simply memorizing answers, can help ease their annoyance at not having an answer handed to them every time.
Awareness of these epistemological issues may encourage the professor to cajole, encourage, and demonstrate why participation is acceptable, useful, and necessary. Asking my students in online questions to explain the value of in-class discussions yielded many insightful answers, and participation after instituting them was measurably less "faded" than in previous classes. Encouraging discussion, and teaching what “arguing” means to a physicist, are necessary steps for the students to engage appropriately with the material and each other.