From errors to stereotypes: Different levels of cognitive models in algebra

Elisabeth Delozanne1, Christian Vincent1, Brigitte Grugeon2,,

Jean-Michel Gélis2, Janine Rogalski3, Lalina Coulange2,

1CRIP5 Université René Descartes

45 rue des Saints Pères, 75270 Paris Cedex 6 France

2DIDIREM Université Paris VII

2 Place Jussieu, 75 251 PARIS Cedex 5 France

3C&AF Université Paris8

2 rue de la liberté, 93526 Saint-Denis Cedex 2 France

Abstract: Developing complex student models based on advanced cognitive and educational studies appeared to be efficient to create new learning opportunities. In This paper proposes an approach and a software to support teachers dealing with the complex problem of articulating whole class management with the necessary personalization of teaching in the domain of school algebra. Within the Pépite project, a multidisciplinary project that aims to diagnose high school students’ algebraic competence, our experience with teachers has led us to propose several levels of cognitive models according to usage and actors concerned. We defined three levels. The first level is the coding of the student’s answers according to a multidimensional analysis grid derived from cognitive and educational research. This very fine-grained level is the base for a quantitative and qualitative description of the student’s competence. We called it the “student’s cognitive profile in algebra”. This profile is made up of the other two levels. The second level expresses personal features of the student’s competence and supports tactical decision-making. It is based on a categorization of students also derived from educational research and expressed by learning levers and weaknesses. The latter are instantiated for each individual student by her/his success rates and lists of characteristic errors. The third level is the stereotype level that reifies the categorization of students and supports strategic decision-making. We expose the mode of calculation and the use of each model in a software we implemented. We then discuss our models in comparison with related work and we evoke experimentation in progress and the various prospects.

Introduction

Developing rich models based on advanced cognitive and educational studies appeared to be efficient to create learning opportunities (Kay 2000, Burkhardt et al. 2003, Mitrovic et al. 2003, Pellegrino 2001, Schoenfeld 2002, Stacey 2003), . In the Pépite project, our objective is to design an intelligent assistant that supports math teachers’ activity when they have to monitor learning in a classroom context taking into account their students’ cognitive diversity. In the first stage of the project we based our work on mathematical educational research (Artigue et al. 2001, Kieran 1992, Sfard et al. 1994) and we set up a multidimensional model of the students’ algebraic competence (Grugeon 1995). We called it a student’s cognitive profile in algebra; it gave an account not only of students’ errors but also of coherences in their algebraic thinking. To reach the expected level of algebraic competence some of these coherences must be developed by learning activities while others, that are inadequate, must be destabilized by other learning activities (Brousseau et al. 1997). We assumed that pointing out these coherences would help teachers to adapt their teaching to their students’ actual competence and therefore to be more efficient. We developed a software, also called Pépite, which makes it possible for teachers to study their students’ individual profiles (Jean et al. 1999, Delozanne et al. 2003). We present this first model in section 2.

Then, in the second stage, we tested our software with teachers in classrooms and in training sessions and we started to define teaching strategies adapted to the profiles diagnosed by Pépite. We restructured and supplemented the previously defined models with two goals in mind: first to facilitate decision making by providing several levels of analysis and, second, to make the models easier to be used by teachers. In section 3, we present the three levels of modelling we defined. The first level is the coding of the student’s answers according to a multidimensional analysis grid derived from educational research. This very fine-grained level is the basis of a quantitative and qualitative description of the student’s competence we called the “student’s cognitive profile in algebra”. This profile is made up of the other two levels. The second level expresses personal features of the student’s competence and supports tactical decision-making. It is based on a categorization of students also derived from educational research expressed by learning levers and weaknesses in algebra. The latter are instantiated for each individual student by her/his success rates and list of characteristic errors. The third level is the stereotype level that reifies the categorization of students and supports strategic decision-making. In section 4 we summarize the results of an ergonomic study about math teachers’ diagnosis activity. Section 5 presents the PépiStéréo software that implements these multi-level modelling processes. In section 6, we discuss this work in comparison with related work. Then we examine some prospects for this research.

Cognitive profiles in elementary algebra

This work is based on an educational research in mathematics. It is grounded in theoretical work and in empirical studies of the activity in elementary algebra of groups of students over several years (Artigue et al. 2001, Grugeon 1995). The third author established a multidimensional model of students’ expected algebraic competence in secondary schools (9th and 10th grade). The diagnosis of a student’s competence intends to situate the student along four dimensions: meaning of letters (unknown, variable, generalized number, abbreviation or label), algebraic calculus, connection and translation between various representations (graphical, geometrical, algebraic, natural language) and type of justifications (proof by example, proof by algebra, proof by explanation, proof by incorrect rule).

Software / Individual Student’s model / Task and algebraic competence models
PépiTest / Student’s answers (data) / Types of exercises (technical, modelling, recognition)
PépiDiag / Local coding of student’s answers
·  Diagnosis Matrix or xml file / For each exercise, a multidimensional criterion analysis grid :
·  correctness, use of letters, algebraic calculus, translation, justification
PépiProf / Overall Profile :
·  success rates (overall, questions answered, type of exercises, most frequent type of processing)
·  qualitative description
·  articulation between representations / Analysis of the whole test
·  Types of processing
·  Settable Thresholds
PépiStéréo / Stereotype
·  Level on 3 dimensions (usage of algebra, translation between representation, algebraic calculus)
·  Levers and fragilities
Personal features
·  Success rates
·  List of errors / Characterization of stereotype in relation with PépiProf overall profile.
Lists of errors from the model of tasks and PépiDiag coding

Table 1 : Different levels of modelling as implemented in the Pépite software

A paper and pencil diagnosis tool was provided to teachers and to researchers. It consists of three parts. A set of exercises designed to go over the whole abilities expected at this school level was proposed to students. A teacher (or a researcher) coded students’ answers to each exercise using an analysis grid derived from the multidimensional model of competence. Then, carrying out a global analysis of the coding results, the teacher (or researcher) built a cognitive profile in algebra for each student (Table 1). It is a three part description of individual student’s competence: a quantitative description expressed by success rates on mastered abilities, a qualitative description of the student’s characteristics along the four dimensions of the model of competence, and a description of flexibility between representations expressed by a diagram indicating the connections between the representation modes the student mastered.

We developed a software to implement this work (Jean et al. 1999)[1]. PépiTest is the student software. It proposes twenty-two exercises and collects students’ answers expressed by multiple choices, by algebraic expressions and by using their own words. PépiDiag codes 80% of the answers: : every answers expressed by multiple choices and by one algebraic expressions, most algebraic reasoning expressed by several algebraic expressions, some answers in student’s words. The result of this automatic coding is a matrix of fifty-five lines and thirty-six columns of Boolean numbers: the line represents the question, the column the criterion and the Boolean number whether or not the criterion is assessed for the student’s answer to the question. PépiProf is the teacher software. It computes the student’s profile by a transverse analysis of the matrix, presents it to the teacher and provides an interface to scrutinize the profiles, check, complete or correct the coding carried out by the software, and to navigate between the student’s answers and profile.

From profiles to stereotypes

The next stage of this research is about defining learning situations adapted to students’ profiles and setting learning goals to be worked upon in priority. The students’ profiles set up by PépiProf with the three hundred students we had tested with Pépite were very different and were made of a great number of parameters. Although the student model implemented in PépiProf was necessary to understand students’ difficulties, it was too complex to support decision making for a whole class. Educational researchers and teachers needed to define groups of students according to the learning objective they estimated the student should work on in priority.

Educational researchers in our team first used an empirical approach to study, two hundred different profiles set up by Pépite. They came to an agreement on a classification of these profiles according to the same first learning objective. Each group was called a stereotype and characterized by a level on three dimensions. Then we worked on making the grouping process explicit and systematic. Finally in order to make the result of the diagnosis easy to use for teachers, we organized some workshops with ergonomists and with math teachers. To sum up a stereotype is an equivalence class of profiles : two student's profiles are said equivalent and belong to the same stereotype if they are advised to work on the same learning objective

For instance for Mickael’s stereotype (Table 2) we advise (strategic decision making) emphasizing the tool dimension of algebra through problems involving generalization and proof. To tune the problem characteristics to Mickael’s cognitive profile (tactical decision making) we need a more accurate description of Michael’s competence. As Pépite detected incorrect uses of parentheses, we will choose a task to prove a property involving a parenthesized expression.

A stereotype is characterized by three or four levels of competence on three dimensions: use of algebra (UA), translation from a representation to another (T) and algebraic calculus (AC). Levers and weaknesses are associated to stereotypes. They give an external representation of how the stereotype is linked to the student’s answers and are illustrated by students’ personal features such as success rates on different kinds of tasks and a classified list of errors.

Mathematics teachers and diagnosis

In (Delozanne et al. 2003) we described uses of Pépite in different contexts for several years and the lessons we drew from these experiments. An ergonomics study was conducted with ten math teachers using Pépite in different French "college" (secondary schools)(Rogalski et al. 2003). Here we only present the results of these studies relevant to our work on stereotypes. Teachers were strongly interested in PépiTest: it collects more open and richer answers than usual computer-based assessment test, and it covers the broad spectrum of content and thought processes represented in the algebra curriculum. They were also interested in the teaching approach based on competence and in an automatic and intelligent paper-grading system. However these studies stated a certain number of obstacles to the widespread use of Pépite in classrooms.

D. Mickael 24/09/2004
grade: 10 th (2nd 10) / Print
Learning activities for Mickael
Stereotypes / Personal features
UA3 / Usage of algebra: level 3
This student does not use algebra enough either as a tool to formulate equations or to produce formulae or to justify / Modeling exercises: Success rate : 18 %
Lever:
The student is beginning to use algebra to prove
Weakness
Too few algebraic justifications
For instance
Exercise 2.1 : Justification by giving incorrect rule a² a3 = a6
Exercise 4.3 : Justification by incorrect explanation in natural language : You are not allowed to have apples and bananas
T3 / Translation: level 3
It is difficult for this student
·  To articulate relation between variables with algebraic expressions
·  Or to link an algebraic expression to another representation (or vice-versa)
At least once, the student used algebra as a shorthand i.e. without finding the relationship in the situation / Recognition exercises: success rate 48 %
Levers:
Some links: (algebraic ßà geometric)
and (algebraic à natural language)
Weakness
Uses of symbolic writing to abbreviate
For example:
Incorrect translations
Exercise 3: No use of parenthesis
Exercise 5 a: Confusion between product and sum
Use of symbolic writing to abbreviate:
6S = T (6 times more students than teachers). Etc.
AC3 / Algebraic calculus: level 3
The student showed low ability in algebraic calculus and used incorrect rules to form or to transform expressions / Technical exercises: success rates 14 %
Weakness:
Low ability in using operators
For instance:
Errors with parentheses
Exercise 3p2: a+3(a+b) for (a+3)(a+b)
Incorrect rules
Exercise 2: 3 + 5a = 8a
Exercise 9a: a²- b² = (a-b)²
Exercise 9c: ax = b gives x = - b/a
Exercise 9 : (x + 2)²- 5(x+2) = (x + 2)(2-5)

Table 2 : Mickael’s profile (English translation of a screen of PépiStéréo)

Teachers’ diagnosis activity is determined by tradeoffs between the different requisites of the institution: "to go through the curriculum", "to make the class progress as a whole ", and "to ensure every student learns". Their diagnosis often focuses on the class or on a group of students rather than on individual students. For instance whole class diagnosis occurs when a teacher is planning a lesson for the class. Teachers' major objective is the monitoring of the class activity with respect to the knowledge content to be taught. Their individual diagnosis articulates an overall evaluation of the class and an individual evaluation of the students. This diagnosis is often expressed by a categorization of the students in subclasses, for example "good ones", “rapid regular ones ", "slow regular ones", "weak ones". The teachers’ diagnosis is often based on students’ recurring errors and focuses on algebraic objects (equations, powers, identities etc.) whereas Pépite focuses on competence and is based on the “tool and object dimensions of algebra” (Artigue et al 2001). Indeed this approach to algebra teaching derived from educational research is not usual for teachers. These studies showed also that the more experienced teachers were, the more their diagnosis was closely linked to actions that they could implement in the class to capitalize on it.