Teaching from strengths: a new research base
Sondra G. Stein, National Institute for Literacy, USA
Brenda Bell, University of Tennessee-Knoxville, USA
Paper presented at SCUTREA, 31st Annual Conference, 3-5 July 2001, University of East London
EQUIPPED for the Future (EFF) is a national customer driven, standards-based adult literacy system reform initiative. It was developed by the National Institute for Literacy in response both to Congressional mandates to measure U.S. progress toward the national adult literacy goal and to concerns, like the ones expressed above by adult learners, about what it would take to meet that goal: 'to be literate and possess the knowledge and skills necessary to compete in the global economy and to exercise the rights and responsibilities of citizenship.' Since 1994 researchers and practitioners participating in EFF have been mapping a new conception of adult learning based on research on the development of expertise and the social construction of knowledge. This paper presents the findings from the current phase of EFF field research.
· One of the earliest initiatives of a new federal agency, Equipped for the Future took a different route from other standards-based initiatives established to improve the quality of public education for school-age children (kindergarten to 16 in the U.S.). Rather than beginning by defining knowledge and skills within key academic subject matter domains like science, math, English, and social studies, EFF partners began by defining what adults do in the primary roles identified by the national goal - the role of worker, citizen, and parent/family member. Through an iterative, structured feedback process conducted over a twelve-month period we built consensus on broad areas of responsibility and key activities in these three socially constructed roles. Then we turned to the task of identifying a set of knowledge and skills that supported effective action in those roles (Merrifield, 2000). Our definition of effective action both within and across these roles was shaped by a set of broad purposes for learning, identified in the first phase of our research, through qualitative analysis of writings from more than 1500 adult learners (Stein, 1995) These learner-defined purposes are:
· Access to information, so adults can orient themselves in the world;
· Voice, so adults can speak with the confidence that they will be heard;
· Independent Action, so adults can make decisions without having to rely on others to mediate the world for them; and what we called Bridge to the future, learning to learn so adults can keep up with the world as it changes, and leave behind their fears of 'moving nowhere'.
Applied learning standards
From the beginning Equipped for the Future's approach to defining and elaborating necessary skills was congruent with the situated cognition branch of cognitive psychology.
Keeping our focus clearly on what adults need literacy for, we identified 16 core skills that supported effective performance as workers, family members, and citizens and members of communities. Then, through two years of iterative field and expert review, we defined Content Standards that describe what adults need to know and be able to do to use these 16 skills in everyday life activities.
EFF Content standards include a broader range of skills then adult basic skill programs are typically expected to cover. There are five communication skills: Read with understanding; Convey ideas in writing; Speak so others can understand, Listen actively, Observe critically. Three decision-making skills: Use math to solve problems and communicate, Solve problems and make decisions, Plan.
Four interpersonal skills: Cooperate with others, Advocate and influence, Resolve conflict and negotiate, Guide others.
And five lifelong learning skills: Take responsibility for learning; Reflect and evaluate; Learn through research; Use information and communications technology.
As the names of the standards suggest, our focus in defining them went beyond the declarative and procedural knowledge that typically define basic skills like learning how to read to include explicit focus on the cognitive and metacognitive strategies that enable us touse knowledge to accomplish some purpose. The EFF standard Read with understanding, for example, integrates necessary components of the reading process such as phonemic awareness and decoding strategies into five key 'components of performance' that keep the learner's reading purpose central. These components are: determine the reading purpose; select reading strategies appropriate to the purpose; monitor comprehension and adjust strategies; analyze information and reflect on its underlying meaning; integrate it with prior knowledge to address the reading purpose.
Our goal in proposing this range of standards and in framing them as we did was to shift the focus of adult literacy and basic skills instruction and assessment away from a decontextualized skills-based curriculum toward a contextualized, practices-based curriculum that was better matched to and firmly grounded in learners' own purposes for returning to school. EFF's basic argument was that adult basic education programs could only be successful in helping adults achieve their purposes - whether related to work, family, or community achievements - if they worked with adults on the full range of skills that are critical to success in those activities. Starting from the assumption that many teachers and programs already did try to address the full circle of EFF Skills, we argued that they could not be fully successful in these efforts until the national reporting framework for English as a second language and adult basic education included more than the three Rs. As long as program funding continued to be based on assessments of student progress and success only in numeracy and in reading, writing, and speaking English, the primary focus of the system would continue to be on building expertise in these areas only. Therefore, in order for Equipped for the Future to achieve its goals of refocusing the adult basic education and English as a second language system, we would have to develop a new assessment and reporting framework that was congruent with contextualized, purposeful instruction.
Building the assessment framework
In 1999, as we began the third and final field review of the standards, we launched a two-pronged effort to define an assessment framework for the EFF Standards. We began by conducting a review of existing standards efforts throughout the English-speaking world, and invited a range of experts on standards and assessments to provide advice and direction. At the same time, we prepared data collection protocols to focus teacher reports from our third round of field review on what student performance looked like, using the EFF Standards, and on teacher and student descriptions of progress.Through these two efforts we identified and built consensus on a set of guiding principles for our assessment framework. These guiding principles included the following: 1. The EFF Assessment Framework must address multiple purposes for assessment.
2. The EFF Assessment Framework must support a multidimensional, flexible, and systemic approach to assessment.
3. The EFF Assessment Framework must address learning over a lifetime.
4. The EFF Assessment framework must address a single continuum of performance for all adults-including those with only minimal and those with many years of formal education.
5. Each level in the EFF Assessment Framework must communicate clearly what an adult at that level can do.
6. The levels in the EFF Assessment Framework must be explicitly linked to key external measures of competence so that adults and systems can rely on them as accurate predictors of real-world performance.
7. The levels in the EFF Assessment Framework must be the products of a national consensus-building process that assures portability of certificates and credentials.
8. Work on the development of this framework must maintain the strong customer focus that has distinguished the EFF Standards development process to date.
Taken together, these principles all pointed to building the assessment framework on cognitive science research on the development of expertise. This research base enabled us to conceive of a single continuum of increasingly skilled performance that included all adult performance - from novice to expert; it also provided a starting point for defining a small number of key dimensions that distinguish performances along the continuum. Drawing on core elements of expert performance synthesized by the National Research Council Committee on Developments in the Science of Learning, we identified four key dimensions for rating performance of adults on the 16 EFF standards (Bransford et al, 2000). We defined these dimensions as: Structure of Knowledge Base: The literature on expertise and transfer asks us to think about a knowledge base as not only what and how much an individual knows (declarative and procedural knowledge) but how that knowledge is organized. The goal is to assure that, as an individual's knowledge relative to a particular domain or skill grows, the structure of the knowledge base becomes increasingly coherent, principled, useful, and goal oriented (Glaser, 1991). This means that what a person knows - at whatever levels of knowledge - is organized for efficient retrieval and application in every day life.
Fluency: the level of effort required for an individual to retrieve and apply relevant knowledge.
Independence: the extent to which an individual can select, plan, execute, and monitor performance without reliance on the direction of others.
Range: the kind and number of tasks and contexts in which an individual can use the skill (Stein, 2000).
· Identifying these four dimensions enabled us to define an EFF Continuum of Performance that was consistent with our guiding principles. Once practitioners confirmed that these four dimensions provided useful tools for assessing what students knew and how well they could use that knowledge to perform everyday life tasks, we were ready to begin work on the remaining tasks necessary to building the EFF assessment framework. These tasks include:
· Developing a continuum of performance for each EFF Standard, with levels that benchmark key performances;
Identifying and developing tools to assess performance of each standard for a range of assessment purposes.
Developing a broad "qualifications framework" that focuses on integrated performance across standards, with levels that represent real-world benchmarks.
In order to be sure that the continuum of performance we defined for each standard reflected actual contextualized learner performances we began work on these tasks with a large-scale field research project involving more than 100 practitioners. This phase of the research aims to define the continuum for each standard by engaging teachers (working in one-on-one and classroom settings with both ABE and ESL learners) in collecting data on student performance relative to the four dimensions of performance.
To be sure that the data we collect is 'standardized,' focusing directly on the standard and the dimensions of performance so that it can be aggregated, teachers in the project have been trained to use templates - developed by the EFF Assessment Team - to construct what we call 'wellstructured performance tasks.' 2 The tasks combine two key ingredients of EFF: an emphasis on meaningful, real-life tasks and a sharper focus on the knowledge and skills underlying successful use of a standard in carrying out such tasks. Task templates help teachers make sure that the tasks they have created clearly center on the target standard, so that successful performance will provide evidence of how well a student can use the standard.
Task templates also enable teachers to rate the complexity of the task they have created, so they can be sure it is within a range that is appropriately challenging for their students, and to identify the underlying knowledge and skill the task requires, so they can provide instructional opportunities for students to develop and practice these underlying skills prior to carrying out the task. Learner performance templates guide teachers in using the four dimensions to describe and rate the evidence of performance they have collected, both during the practice periods and while students are carrying out the task. These performance templates ask teachers to describe and rate each aspect of performance separately - to deconstruct what we often aggregate - to aid us in constructing as precise a picture as we can of the adult developmental continuum for any one standard.
Preliminary results
The data we have collected so far does suggest an adult developmental continuum for EFF skills that is different at the lower end than the scope and sequence based on children's development, which shapes our current approach to adult basic skills instruction.3 There are two trends in the data that support this. First, teachers are constructing tasks that outsiders consider to be far above the abilities of students at the targeted level, but which teachers and students both believe are appropriate to the level. Second, students are in fact performing at this 'higher' level, validating the judgment of teachers and learners.
Let's look at both of these trends in the data more closely.
The tasks that outsiders consider too difficult for learners at a particular level to perform are, in fact, meaningful real world tasks that adults need to perform, regardless of their skill level - like shopping for groceries, asking for assistance from someone in authority, understanding workplace benefits, understanding childhood illnesses. When outsiders rate these as too difficult, they are, for the most part, focusing on the 'what' of knowledge base - what in EFF we call 'vocabulary and content knowledge' (defined to include declarative, procedural and conceptual knowledge related to the skill and to the subject of the task). And, judged solely in these terms, the task may be too difficult. That is, coming into the class, students may not have the specific vocabulary or content knowledge required to carry out the task. What they do have, however, that children at an equivalent level of knowledge don't have, is a brain in which the areas that support cognitive and metacognitive processes are fully developed. They also have considerable experience - both successful and unsuccessful - in using that organic brain capacity to organize their knowledge for use. Every adult student has some experience in recognizing or creating patterns, determining what are the important features of a problem or situation to pay attention to, and determining when a particular problem-solving approach might be effective. In building the EFF continuum of performance on cognitive science research on expertise, we have provided teachers in EFF research sites with a definition of 'knowledge base' that enables them to pay attention to and build on this area of 'strategic knowledge.' There are three ways in which teachers in our research sites are using this focus on 'organizing knowledge for use" to support a higher level of performance among their students. First of all, since the performance tasks teachers construct are 'meaningful real world tasks,' they call upon adult students to learn new content in the context of a pattern or schema that is appropriate to the task. Hence, as students learn new content knowledge, they are simultaneously organizing it for use in a particular context.