The Validity of Formative Assessments
Terry Crooks
Educational Assessment Research Unit
University of Otago
Box 56, Dunedin, New Zealand
()
2001 Annual Meeting of the
British Educational Research Association (BERA)
Leeds, England, 13 - 15 September
Note
This paper is incomplete as yet. The major points to be included in
the paper are presented here, but the linking text is yet to be written.
Contact the author to receive the full paper when completed.
10
Defining Assessment
Any process that provides information about the thinking, achievement or progress of students
Defining Validity
Messick, 1989, p.13 (Validity)
Validity is an integrated evaluative judgement of the degree to which empirical evidence and theoretical rationales support the adequacy and appropriateness of inferences and actions based on test scores or other modes of assessment.
Distinguishing Between Summative and Formative Assessment
Summative assessment is intended to summarise student attainment at a particular time, whereas formative assessment is intended to promote further improvement of student attainment.
Assessment OF learning versus assessment FOR learning.
The Formative Impact of Summative Assessment
Crooks, 1988, p. 467(The impact of classroom evaluation practices on students)
Classroom evaluation affects students in many different ways. For instance, it guides their judgement of what is important to learn, affects their motivation and self-perceptions of competence, structures their approaches to and timing of personal study (e.g. spaced practice), consolidates learning, and affects the development of enduring learning strategies and skills.
It appears to be one of the most potent forces influencing education. Accordingly, it deserves very careful planning and considerable investment of time from educators. Many of the skills and attitudes that are goals of education take years to develop, and their development can be undermined by lack of consistent support for them in the educational experiences of the students.
Adapted from Crooks, 1988, p. 443 (The impact of classroom evaluation practices on students)
Short Term Effects
• Focussing attention on important aspects of the subject;
• Giving students opportunities to practice skills and consolidate learning;
• Guiding further instructional or learning activities within the course;
Medium and Longer Term Effects
• Influencing students' motivation as learners and their perceptions of their capabilities (self-efficacy);
• Communicating and reinforcing (or in some cases undermining) teaching goals, including key performance criteria and desired standards of performance;
• Influencing students' choice of (and development of) learning strategies, skills and study patterns;
• Influencing students’ subsequent choice of courses, activities or careers
Definitions of Formative Assessment
Ministry of Education, New Zealand, 1994 (Assessment: policy to practice)
A range of formal and informal assessment procedures (for example, the monitoring of children’s writing development, anecdotal records, and observations) undertaken by teachers in the classroom as an integral part of the normal teaching and learning process in order to modify and enhance learning and understanding.
Bronwen Cowie & BeverleyBell, 1996/1999 (A model of formative assessment in science education)
The process used by teachers and students to recognise and respond to student learning in order to enhance that learning, during the learning.
Black & Wiliam, 1998, p. 2 (Inside the black box)
Assessment’ refers to all those activities undertaken by teachers, and by the students in assessing themselves, which provide information to be used as feedback to modify the teaching and learning activities in which they are engaged. Such assessment become ‘formative assessment’ when the evidence is actually used to adapt the teaching to meet the needs.
Key Points about Formative Assessment
Sadler, 1989, p.119 (Formative assessment and the design of instructional systems)
A key premise is that for students to be able to improve, they must have the capacity to monitor the quality of their own work during actual production.
This in turn requires:
• that students possess an appreciation of what high quality work is,
• that they have the evaluative skill necessary for them to compare with some objectivity the quality of what they are producing in relation to the higher standard, and
• that they develop a store of tactics or moves which can be drawn upon to modify their own work.
Instructional systems which do not make explicit provision for the acquisition of evaluative expertise are deficient, because they set up artificial but potentially removable performance ceilings for students.
Crooks, 1988, pp. 468-9 (The impact of classroom evaluation practices on students)
The approach that leads to the most valuable feedback is nicely captured by Easley and Zwoyer (1975):
“If you can both listen to children and accept their answers not as things to just be judged right or wrong but as pieces of information which may reveal what the child is thinking you will have taken a giant step toward becoming a master teacher rather than merely a disseminator of information. (p. 25)”
Crooks, 1988, p. 469 (The impact of classroom evaluation practices on students)
Feedback should be specific and related to need. Simple knowledge of results should be provided consistently (directly or implicitly), with more detailed feedback only where necessary to help the student work through misconceptions or other weaknesses in performance. Praise should be used sparingly and where used should be task-specific,
whereas criticism (other than simply identifying deficiencies) is usually counterproductive.
A Summary of Findings from Research on Formative Assessment
Educational research has shown that providing high quality feedback on student work is a very powerful way of raising the standard of student work. Paul Black and Dylan Wiliam (1998), in a 10 year review of research on assessment, stated that "We know of no other way of raising standards for which such a prima facie case can be made." John Hattie (1999), in his inaugural professorial lecture as Dean of Education at the University of Auckland, summarised his wide-ranging review of research on “what works” in education with the statement “the most powerful single moderator that enhances achievement is feedback.”
Royce Sadler (1989) identified three elements that are crucial to the effectiveness of formative assessment:
• helping students to recognise clearly the desired goal (understand what is required);
• providing students with evidence about how well their work matches that goal;
• explaining ways to close the gap between the goal and their current performance.
Self-assessment is a vital component in learning. Feedback on assessment cannot be effective unless students accept that their work can be improved and identify important aspects of their work that they wish to improve. Self-monitoring is a key component of the work of all professionals, so if we want our students to become professional learners and professionals in their fields we should actively promote self-assessment. If students are asked and encouraged to critically examine and comment on their own work, assessment can become more dialogue than monologue, and can contribute powerfully to the educational development of students. As Wynne Harlen and Mary James (1996) put it,
“…students have to be active in their own learning (teachers cannot learn for them) and unless they come to understand their own strengths and weaknesses, and how they might deal with them, they will not make progress.”
Marks or grades alone produce no learning gains. Indeed, there is some evidence that students gain the most learning value from assessment when feedback is provided without marks or grades. Where marks are provided, they often seem to predominate in students’ thinking, and to be seen as the real purpose of the assessment.
Student motivation is crucial to learning. Assessment is one of the major influences on student motivation. It is important, therefore, to anticipate and try to optimise the motivational effects of feedback on assessment. The research evidence available suggests that the greatest motivational benefits will come from focussing feedback on:
• the qualities of the student’s work, and not on comparisons with other students;
• specific ways in which the student’s work could be improved;
• improvements that the student has made compared to his or her earlier work.
Five points summarise the key lessons from research about formative assessment.
Assessment that promotes learning:
• involves learning goals understood and shared by both teachers and students;
• helps students to understand and recognise the desired standards;
• involves students in self-assessment;
• provides feedback which helps students to recognise next steps and how to take them;
• builds confidence that students can improve their work.
Some Other Summaries of Requirements for Effective Formative Assessment
Black & Wiliam, 1998 (The Black Box)
Feedback to any pupil should be about the particular qualities of his or her work, with advice on what he or she can do to improve, and should avoid comparisons with other pupils.
For formative assessment to be productive, pupils should be trained in self-assessment so that they can understand the main purposes of their learning and thereby grasp what they need to do to achieve.
Opportunities for pupils to express their understanding should be designed into any piece of teaching, for this will initiate the interaction whereby formative assessment aids learning.
The dialogue between pupils and a teacher should be thoughtful, reflective, focused to evoke and explore understanding, and conducted so that all pupils have an opportunity
to think and to express their ideas.
Tests and homework exercises can be an invaluable guide to learning, but the exercises must be clear and relevant to learning aims. The feedback on them should give each pupil guidance on how to improve, and each must be given opportunity and help to work at the improvement.
Assessment Reform Group, UK, 1999, p. 7 (Assessment for learning: beyond the black box)
Assessment that promotes learning:
• is embedded in a view of teaching and learning of which it is an essential part;
• involves sharing learning points with pupils;
• aims to help pupils to know and to recognise the standards they are aiming for;
• involves pupils in self-assessment;
• provides feedback which leads to pupils recognising their next steps and how to take them;
• is underpinned by confidence that every student can improve;
• involves both teacher and pupils reviewing and reflecting on assessment data.
Formative Assessment of Completed or Draft Work
Student-Centred Contextual Components
Who is this student? What cultural or other considerations require special attention?
How confident is this student about this curriculum area or type of task? (self-efficacy)
How does this student respond to praise, criticism, and guidance? What works best?
Task-Specific Contextual Components
Did the student understand the task?
• its broad thrust
• the specific qualities desired, and their relative importance
• the standards that would be applied.
How much effort has the student put into completing the task?
• Were there mitigating commitments or circumstances?
Did the student receive substantial help (or distraction!) while working on the task?
• from school staff, peers, or family members
What use did the student make of help available from others?
• ignored or rejected it
• learned from it, and incorporated it into his/her work
• used it essentially verbatim, without evidence of learning
Summative Components
Overall, how well does the work meet the desired criteria and standards?
Should a mark/grade be recorded for the student? If so, where and how?
Components with Both Summative and Formative Aspects
What are the strengths of the work? (criterion referenced)
What are the weaknesses of the work? (criterion referenced)
Should the student be given “knowledge of results” (in a criterion referenced way)?
Formative Components
How does this student’s performance match the teacher’s expectations for that student?
• distinctly better than expected
• largely as expected
• distinctly worse than expected
What evaluative feedback should be given to this student?
• what, if anything, should be commended?
• what, if anything, should be criticised?
• what should be passed over without comment?
What corrective suggestions/requirements should be given to this student?
• mainly focussed on improving this particular piece of work
• mainly focussed on better performance on future work
Should the student be required or encouraged to resubmit the work with improvements?
• if so, is summative assessment of the work then appropriate
Formative Assessment of Work In Progress
Student-Centred Contextual Components
Who is this student? What cultural or other considerations require special attention?
How confident is the student about this curriculum area or type of task? (self-efficacy)
How does the student respond to praise, criticism, and guidance? What works best?
Task-Specific Contextual Components
Is this the sort of task that the student usually finds motivating?
Does the student understand the task?
• its broad thrust
• the specific qualities desired, and their relative importance
• the standards that will be applied.
Does the student show signs of existing or emerging beliefs or understandings, related to the task, which will obstruct the desired learning or performance?
How much effort is the student putting into completing the task well?
Are other people, events, or circumstances interfering with the student’s work?
What use is the student making of help from others?
Formative Components
What questions should be asked to better understand what the student is doing?
What should be done at this point to encourage the development of self-assessment skills?
How well is the work progressing towards meeting the desired criteria and standards?
What are the strengths of the work thus far? (criterion referenced)
What are the weaknesses of the work thus far? (criterion referenced)
How is the student’s performance matching the teacher’s expectations for that student?
• distinctly better than expected
• largely as expected
• distinctly worse than expected
What evaluative feedback on the progress made should be given to this student?
• what, if anything, should be commended at this stage?
• what, if anything, should be criticised at this stage?
• what should be passed over without comment at this stage?
What corrective suggestions/requirements should be given to this student?
• focussed on improving this particular piece of work
• intended to enhance the student’s work more generally