THE NATIONAL ACADEMIES
NATIONAL RESEARCH COUNCIL
NATIONAL SCIENCE RESOURCES CENTER
Math/Science Partnerships Workshop
Assessment of Student Learning
May 18, 2004
The National Academies
Keck Center Room 100
500 Fifth Street, NW
Washington, DC
Proceedings By:
CASET Associates, Ltd.
10201 Lee Highway, Suite 160
Fairfax, VA 22030
(703) 352-0091
TABLE OF CONTENTS
Page
Equity and Access: The Implications of Accountability 3
Diana C. Pullin
Professor of Education Law and Public Policy
Boston College
Planning for Change in Assessment 62
Mark Kaufman
Co-Director of the Center for Education Partnerships
TERC
Committee Reflections and Participant Discussion 110
1
P R O C E E D I N G S (8:30 a.m.)
DR. KAUFMAN: Good morning. We’d like to try and start pretty much on time today, so that we can help you to get out on time as well.
I hope you had a good evening, found good places to eat last night, got a little bit of rest and relaxation after a long day yesterday.
This morning, we are going to have two sessions. The first one will be with Diana Pullin from Boston College School of Education, and she is a Professor of Education Law and Public Policy at BC. She was formerly the Dean of the School of Education at Boston College, and she’s been at Michigan State University and was a staff attorney, Co-Director and then President of the Center for Law and Education in Cambridge and Washington, and I had the pleasure of chatting with her yesterday, and I’m sure you’ll enjoy her discussion and presentation today.
I have been asked to tell you that the CD that was distributed didn’t include the biographical sketches of presenters, and so that will be run off and distributed, in case you want to know who the people are.
Terry is looking, in back, as if to say, I didn’t know that.
Word has it that we’ll get the bio sketches of the presenters in case you want to know a little bit more about the people who presented.
After Diana has had her opportunity to present, I am going to come back up and not just be the meeter and greeter. I’m going to do sort of a wrap-up session with the group to look at some of the issues about trying to take this back, take this work assessment back into your MSP projects on what kinds of things you want to be considering in order to do that. So I’ll be back here.
And before I go -- has a quick comment that you wanted to make.
SPEAKER: Yes. I would like to invite any and all of you who are interested in assessment to possibly consider being involved with the ultimate in summative assessment, and that is the reading of AP exams, and the AP program, which is run out of the College Board, is always looking for readers who would be interested in reading AP exams in their various disciplines. The science disciplines are biology, chemistry, physics, environmental science, math disciplines or calculus and statistics, and if any of you would like to do that, it’s usually a glorious week-long event in the summer with an opportunity to interact with really good teachers from around the country and around the world who come and do this.
So if you would like to know more about the AP reading and to be an AP reader, I would encourage you to talk to me or to Walt Jimenez, who is from the College Board, but we would really like to encourage you to think about taking a pleasant week. I’ll have a pleasant week in Lincoln, Nebraska, in mid-June. So -- but think about it, if you might be interested.
Thank you.
DR. KAUFMAN: And with no further ado, I would like to introduce Diana Pullin.
Agenda Item: Equity and Access: The Implications of Accountability
DR. PULLIN: Good morning. It is a pleasure to be here with all of you and to spend a little time thinking about these issues with you.
I will start off at the beginning and confess it is true, I am an attorney. I admit that -- (laughter) -- and guilty as charged.
I am frequently exposed to the opportunity to be the butt of lawyer jokes, most of them funny, but many of them misplaced, and I have noticed over time that one of the most successful social barometers in our culture is who becomes the butt of the most frequent sort of bar jokes, and I kind of introduce myself at cocktail parties to people I don’t know differently, depending on what the current trend of derogatory jokes would be.
When I was a dean of education, I always used lawyer jokes as a way to expose myself to the crowd, and then when the legislature started making noises in Massachusetts about imposing conditions -- new kinds of conditions on educators -- teacher educators in particular -- I suddenly discovered that my little line that you never heard a derogatory joke about a teacher was no longer true. Now -- shifted back to lawyer jokes.
So most of these lawyer jokes are somewhat useful and give an illustration of what people are thinking about things, and this is one example very relevant to you.
Now, the law says nothing about this particular kind of circumstance, and I hope no one would ever contemplate using a lawyer to contest this sort of distraught reaction on the part of a little child about math or science knowledge, but it does give a highlight -- does give an opportunity to highlight some issues about equity that are very important in our considerations of improved achievement in mathematics and science.
Sometimes, the law has some things directly to say, not to those of you who fear having a lawyer sicced upon you, but the lack of particular lenses for looking at how we address issues of equity and adequacy in the provision of educational opportunity, and I am going to use, this morning, some examples -- I’ll also use a much larger collection of examples of important ways to think about equity issues that arise from the professional standards of practice within the education professions and that arise from the previous work at the National Academy of Sciences National Research Council.
Many of these things, I think, are useful ways for those of you who don’t frequently have to think about equity issues or for those of you who think about them more frequently -- having to highlight a way of looking at issues of equity and adequacy, particularly in this era of very high-stakes accountability.
So this morning, I am going to talk about a series of issues that relate to assessment, and I’ll talk about assessment from two different perspectives.
Certainly, in this era of “No Child Left Behind” and many state-imposed reform and accountability systems, the major use and the major consideration of assessment issues focuses on assessment for institutional or individual accountability. Assessments used in state testing systems, many of them increasingly associated with high stakes, but I also talk about assessment in terms of what goes on in local classrooms and in local schools for school-based decision making, either about the school itself, about classrooms or about individual students.
And in talking about assessment, I want to consider factors associated first with equity, and when I talk about equity issues, I am concerned about equity for minority students who are traditionally poorly served by our schools, for low-income students or those who are located in schools that are poorly resourced. Most often, schools in low-property-wealth school districts.
I am also concerned about equity issues as they relate to students who are English language learners, and there are an increasing number of those students, and they come from an increasing variety of language backgrounds.
Next, I’ll talk about equity issues that relate to students with disabilities, both students who have disabilities and students who may be misclassified as having a disability, when, in fact, they do not, or people who have a disability that has not yet been appropriately assessed.
And then we also need to take into consideration, particularly because you are talking about math and science, issues of equity that relate to students who are in a traditionally disadvantaged gender group in terms of either the learning or participation in assessments.
And then, in addition to all these equity issues, I would also like to talk about what I’ll call fairness for all students. Now, that is also an equity issue, but under the broad umbrella, I want to talk about fair treatment of every student in a system for improving mathematics and science learning.
We also need to talk a little bit about educator quality issues. Most of the people in this room are educators, and most of us have been relatively free from some of the recent high-stakes initiatives that have occurred, but I am going to suggest that we have to look closely at those initiatives as well, and then all of this is done in the context of thinking about an incredible public press for more accountability, not only from K-12 schools -- a primary focus -- but also more accountability from higher education institutions, and I am of the view that this accountability movement is extremely important and could be extremely influential in a positive way, but could also be used for some very insidious political purposes that may not, in fact, inure to the benefit of improved instruction for all learners.
So with that broad overview, let me turn, then -- oh, and by the way, since we are talking about assessment, there will be a couple of assessment instruments that we will use for you. So please have a blank piece of paper and a pencil ready for the assessment items that I will impose on you.
So let’s begin.
In this context of an effort to improve educational achievement across the board, we have as the primary, most visible driver for education reform right now the federal No Child Left Behind Act, and, in addition, we have many states, independent of NCLB, imposing their own education reform systems. Now, all of these systems are being aligned with “No Child Left Behind,” because all of the states want to get their Title I money from the federal government, and in order to get your Title I money you have to comply with “No Child Left Behind.”
But in some states, such as my own, Massachusetts, we have been doing standards-driven education reform since 1993, and in many states -- in most states - there is some form of standards-based reform that has been going on.
And there are two kinds of goals that I will draw your attention to in the “No Child Left Behind” system.
The first is that the system is designed to promote high standards for learning, and it is designed to promote broader public accountability mechanisms for those whose children are in school and for citizens more broadly who are concerned about schools.
Now, my view of “No Child Left Behind” is that it is loaded with promises and it is also loaded with potential pitfalls. So I think we need to look at both of those and assess both appropriately.
“No Child Left Behind,” and the state systems that result, rely heavily on the definition of standards for students, widespread use of testing to both assess the quality of education reform and to drive the reform system, and “No Child Left Behind” is relying -- as these systems come on board, as the programs are rolled out, “No Child Left Behind” is relying more and more frequently on a system of public information distribution to try to both press schools to change and to allow the public and parents an opportunity to know what is going on in schools particularly their local school that their child attends.
Now, as more and more of that public information becomes available, there will be more and more consumer push about the performance and accountability of particular schools and the educators who work in them.
Now, in many respects, that is a very useful tool for improving learning, particularly in areas like mathematics and science or literacy for individual students and groups of students in K-12 schools, but many people predicted and we are now beginning to see some of the pitfalls associated with this kind of high-stakes publicly visible test-driven system for improvement.
The testing industry is, in some reports, in its moment of glory because all these systems are driven by standardized, broadly administered tests, but those within the testing industry would also tell you that it is going to be nearly impossible for them to keep up with these mandates and to keep up with them at a high level of quality.
We are also placing upon tests and the data we receive from tests an incredible amount of weight in driving the systems and in driving the accountability reports, because there are high stakes associated with many of these systems -- high stakes for individual children in grade-to-grade promotion or in the receipt of the high school diploma, but also high stakes attached for educators, either educators within a particular building or system or individual educators themselves, and, in most states, you will see an increasing number of designations of certain schools as low performing, many of those designations weighted heavily toward the test-score performance of students within those schools, and, in some states, there will be the capacity to actually dismiss educators -- particularly school administrators, principals and the like -- on the basis of these low-performance indicators.
So there is more and more pressure on educators, both teachers and administrators to accommodate themselves to this system and to recognize and incur the consequences associated with test performance. So among the pitfalls are efforts to manipulate the system, sometimes constructively, but sometimes not so constructively.