Appendix:
The enGauge 21st Century Skills Continua of Progress
Once the enGauge 21st Century Skills were identified, a second wave of research was conducted that drew on a vast body of work dealing with each skill (see References). After analyzing and summarizing that work to date, we sought to characterize what each skill would look like in practice. The goal – to provide teachers with explicit criteria by which to gauge students’ progress – was tackled in three steps. We:
- Identified specific behavioral, cognitive, and affective qualities of each 21st century skill
- Described levels of progress toward each of these qualities
- Considered each skill in the context of today’s Digital Age – updating them when necessary
Early in the process, it became clear that many of the skills identified were not new constructs; existing research was able to substantially inform the qualities that make up those skills. What these sources typically did not address (with some exceptions) was the shape these skills can take, either in 21st century environments or in the context of the technology tools available today. Thus, the strategy for developing the continua was two-fold: we drew on existing work as much as possible to inform the content of the continua, and we supplemented this content with our own expertise on teaching and learning in digital-age classrooms.
A wide range of sources was used in the research part of this process. When possible, empirical studies and theoretical work from psychology and education were used for generating specific facets of the skills. Sometimes, this process itself involved repeated passes through the literature (for example, when they authors began researching self-direction, they found that self-directed learners were likely to make specific attributions for successes and failures; this led to another wave of research on students’ attributions). Publications by organizations specializing in a skill (e.g. the American Library Association’s work on information literacy; the National Academy of Sciences’ work on scientific literacy), as well as publications by experts in a field (e.g. Daniel Goleman’s work on emotional intelligence), were also used to build the profiles and continua. These sources typically represented field-based expertise. Together, these three types of sources represent “best thinking” by researchers, theorists, and organizations about what qualities characterize an individual possessing each of the skills.
Not surprisingly, empirical studies that describe degrees of progress on each quality, particularly in a digital-age context, simply do not exist. To develop the progression of each quality contained within the continua, from novice to advanced, and to do so in a way that reflected a “digital-age” view of these skills, the authors drew on their own expertise, including:
Years of experience teaching students in classrooms, and thus “hands-on” experiences with the range of development on these skills;
Years of experience providing professional development to teachers, specifically on enhancing these skills through the use of technology, but also on technology-supported teaching and learning in general;
Fieldwork in classrooms across the nation, observing teachers and students as they work on transforming traditional teaching and learning to digital-age, technology-supported practice.
It is important to note that these continua are in fact, continua – not rubrics. They depict levels of development, but are not specifically targeted to any one task, assignment, content area or grade level. Naturally, the precise form some of these skills will take will depend on these contextual factors. However, the principles of rubric development – outlined by educational expert Bernie Dodge on his website – were also used to guide the development of these continua. Specifically, these:
Focus on measuring a stated objective (performance, behavior, or quality)
Use a range to rate performance
Contain specific performance characteristics arranged in levels indicating the degree to which a standard has been met (Pickett and Dodge, 2001: For more details about rubrics, their purposes, and their development, see this site).
Finally, specific sources used to generate each of the student profiles and continua are listed in this document by skill (see References – Selected).Continuum of Progress: Basic Literacy
Basic Literacy is language proficiency (in English) and numeracy at levels of proficiency necessary to function on the job and in society to achieve one’s goals, and develop one’s knowledge and potential in the Digital Age
Reading / Novice / Basic / Proficient / AdvancedAbility to read and understand day-to-day written materials / Without substantial assistance, the student is unable to read and comprehend even the most basic prose* or documents** / The student is independently able to read and comprehend some basic prose* (e.g., simple written instructions) and documents** (e.g., simple forms), but requires assistance with more complex items. / The student is usually able to independently read and comprehend many forms of prose* and documents** / The student is almost always able to read and comprehend a wide range of prose* and documents**, sometimes beyond his/her developmental level.
Writing / Novice / Basic / Proficient / Advanced
Writing to accomplish day-to-day functions / The student is unable to complete day-to-day tasks that require writing; written responses and communications contain errors that make them misleading or difficult to understand. / The student is able to produce very basic, short, written responses and communications (e.g., grocery lists, brief notes) clearly and accurately. He/she requires substantial assistance for more complex items. / The student is able to produce basic written responses and communications and he/she is sometimes able to independently complete more complex writing tasks clearly and without errors. / The student is almost always able to produce clear and error-free written responses and communications, sometimes exceeding expectations for his/her developmental level. He/she uses writing skills effortlessly to complete day-to-day tasks.
Listening/ Speaking
/ Novice / Basic / Proficient / AdvancedSpeaking and understanding English / The student has severely limited English proficiency; he/she frequently misunderstands spoken instructions and often provides misleading/erroneous information as a result. / The student can understand and speak basic, conversational English. However, he/she does not have sufficient grasp of English to participate in discussions that are technical or academic. / The student has an adequate grasp of English and is able to participate in a range of conversational, technical, and academic discussions. However, he/she does not have a grasp of more subtle nuances of word meaning or colloquialisms. / The student is fluent in English, and has command of even subtleties in meaning.
* Prose includes (but is not limited to) literature, editorials, newspaper articles, poems, and stories
**Documents include print and media-based artifacts, such as job applications, bus schedules, maps, checks, tax forms, and tables
Numeracy
/ Novice / Basic / Proficient / AdvancedComputing and solving everyday math problems / The student is unable to perform basic math functions, and as a result is hindered in his/her ability to complete day-to-day activities effectively (e.g., determining if he/she has enough money to purchase lunch and dessert). / The student is able to perform basic math functions independently; however, he/she requires assistance in more complex math computations. / The student is independently able to perform many day-to-day math functions of intermediate difficulty; he/she is sometimes able to use these skills in real-world contexts (e.g. deciding to purchase a product based on unit price rather than actual price). / The student has an excellent practical understanding of math and is able to use these skills frequently, efficiently, and in real-world contexts (e.g. to make decisions about loans and interest).
Information / Novice / Basic / Proficient / Advanced
Basic Information Literacy / The student has little practical knowledge of where and how information (e.g., bus schedules, reviews of product quality) can be found; he/she is frequently unable to distinguish between information that is relevant or irrelevant for tackling day-to-day prose, document, or computation activities. / The student is able to independently find and access information with which he/she has prior experience; he/she is able to evaluate simple information adequately. / With some effort, the student is able to independently find and access information; he/she is able to evaluate most information adequately, but is not always able to synthesize and use this information effectively. / The student is able to independently find and access information; he/she often knows what sources of information are more likely to be reliable than others, and when necessary, he/she is able to synthesize and weigh conflicting information to make wise decisions.
Technology
/ Novice / Basic / Proficient / AdvancedBasic Technological Literacy / The student is essentially computer-illiterate; he/she does not use technology to enhance efficiency on day-to-day tasks (e.g. checking movie listings, communicating with friends). / The student is able to perform some very basic tasks with technology. He/she uses technology occasionally to make day-to-day life more efficient; however, technology use is highly limited. / The student uses technology independently to enhance efficiency of a number of day-to-day tasks (e.g., checking movie listings, communicating with friends, finding information). / The student has integrated technology into his/her life in a way that makes many aspects of day-to-day functioning more efficient (e.g., he/she has automated schedules and calendars and uses technology regularly to complete schoolwork or to make decisions about purchases).
Continuum of Progress: Scientific Literacy
Scientific literacy is knowledge and understanding of the scientific concepts and processes required for personal decision-making, participation in civic and cultural affairs, and economic productivity
Knowledge and Understanding / Novice / Basic / Proficient / AdvancedKnowledge and understanding of scientific concepts and processes / Student does not have knowledge of scientific concepts and processes as appropriate for his/her grade level. Many understandings of science that the student does possess are riddled with errors and misconceptions. / While the student has some knowledge of scientific concepts and processes as appropriate for his/her grade level, there are still a number of errors and misconceptions to overcome. / Student has a level of scientific knowledge that is appropriate for his/her grade level. Errors and misconceptions that are still held are typical for this level and will likely be overcome. / Student has an extensive knowledge of scientific concepts and processes. This depth of understanding allows him/her to perform consistently above grade level and to confront most misconceptions independently.
Scientific Curiosity
/ Novice / Basic / Proficient / AdvancedAbility to ask, find, or determine answers to questions derived from curiosity about everyday experiences / The student, while occasionally curious about phenomena around him/her, lacks the ability to formulate answerable questions and cannot conceive of a method to determine answers to questions related to science. When pressed, he/she relies on the rote application of steps presented by others. / The student can structure questions about phenomena around him/her with significant assistance. He/she finds it difficult to create strategies for answering questions independently, but may do so with assistance. / The student can construct effective questions as appropriate to his/her developmental level. He/she has strategies for finding or deriving answers to those questions, needing assistance only with more difficult or complex questions. / The student is extremely effective in framing researchable questions and often asks meaningful questions that go beyond his/her developmental level. He/she has multiple strategies for investigating those questions, demonstrating a sophisticated understanding of scientific method and how that method applies to everyday experience.
Natural Phenomena
/ Novice / Basic / Proficient / AdvancedAbility to describe, explain, and predict natural phenomena / The student lacks the conceptual framework, vocabulary, and understanding to describe, explain, or predict natural phenomena around him/her. Attempts to do so illuminate a host of misconceptions and gaps in understanding. / The student has some ability to describe, explain and predict natural phenomena with considerable support and direction. Explanations often contain evidence of some misconceptions and gaps in understanding. / The student is able to describe, explain, and predict natural phenomena as appropriate for his/her developmental level. With guidance, gaps and misconceptions are overcome. / The student is able to describe, explain, and predict natural phenomena at a level of sophistication that seems beyond his/her developmental level. He/she has developed independent strategies for identifying and resolving gaps and misconceptions and makes connections between phenomena that suggest a deep understanding.
Science Content
/ Novice / Basic / Proficient / AdvancedAbility to read articles with understanding and to engage in conversation about the validity of conclusions / The student avoids reading articles or engaging in conversation on topics related to science. He/she lacks strategies for analyzing the quality of scientific content. / While willing to read articles or engage in discussions on topics of science, the student lacks confidence in his/her ability to understand or analyze content and usually requires significant support. / The student is able to read articles about science, but may need some support in the interpretation of the content. With limited support, the student can analyze the quality of scientific conclusions in written and verbal contexts. / The student readily seeks out and reads articles about science in the popular press. He/she is insightful and easily able to engage in conversations about content, often showing significant insight into its quality and applicability.
Identifying Issues / Novice / Basic / Proficient / Advanced
Ability to identify scientific issues underlying national and local decisions and express positions that are scientifically and technologically informed / The student does not recognize issues underlying national and local decisions and is easily led to adopt positions that he/she cannot defend. / The student can identify the scientific basis for issues underlying social decisions with significant guidance. With support, he/she is able to express positions that are fairly well informed. / The student can independently recognize the scientific issues within many national and local decisions. He/she is able to formulate and express opinions with minimal assistance. These opinions are usually well informed, with occasional misconceptions. / The student is able to identify the scientific issues within national and local decisions with surprising perception. His/her analysis and positions are insightful and solidly grounded in scientific reasoning.
Evaluating Information, Methods, and Arguments / Novice / Basic / Proficient / Advanced
Ability to evaluate scientific information on the basis of evidence, sources, and the methods used to generate it / Student cannot articulate meaningful reasons for accepting/rejecting information, arguments, or findings. / Student can articulate some rudimentary reasons for accepting/rejecting information, arguments, or findings. / Student can articulate compelling reasons for accepting/rejecting information, arguments, or findings; reasons are based on facts and findings, but miss some important points. / Student can articulate compelling reasons for accepting/rejecting information, arguments, or data; reasons are sophisticated and based on facts and findings.
Continuum of Progress: Economic Literacy
Economic Literacy is the ability to identify economic problems, alternatives, costs, and benefits; analyze the incentives at work in economic situations; examine the consequences of changes in economic conditions and public policies; collect and organize economic evidence; and weigh costs against benefits
Informed Choices / Novice / Basic / Proficient / AdvancedUsing knowledge of costs, benefits, and the limitations of resources to make informed choices / Student does not have the skill or criteria to making informed economic choices. He/she is easily misled by simplistic and misrepresentative claims, and often does not even realize when a bad decision has been made. / The student has a basic understanding of the criteria for making informed economic choices. With considerable assistance and direction, he/she is able to apply these skills to simple choices. / The student has a significant understanding of the criteria for making informed economic choices. With minimal assistance and direction, he/she is able to apply these skills to choices of simple and intermediate complexity. / The student has an advanced understanding of the criteria for making informed economic choices. Without assistance, he/she is able to apply these skills to choices of varying complexity, demonstrating considerable insight.
Allocating Goods and Services / Novice / Basic / Proficient / Advanced
Ability to evaluate methods for allocating goods and services by comparing costs and benefits / The student is unfamiliar with varying historical strategies for allocating goods and services and lacks the criteria for evaluating systems. / The student has limited awareness of historical strategies for allocating goods and services, particularly of the market system. With considerable direction, the student can conduct a rudimentary analysis of other systems of allocation. / The student has a significant awareness of historical strategies for allocating goods and services. With minimal direction, he/she can conduct an analysis of these systems at a variety of levels (e.g., national economic systems, personal or family decision making structures, or allocation of medical services). / The student has an exceptional awareness of historical strategies for allocating goods and services. Independently, the student can conduct an analysis of these systems at a variety of levels (e.g., national economic systems, personal or family decision making structures, or allocation of medical services).
Economic Incentives / Novice / Basic / Proficient / Advanced
Ability to identify and explain economic incentives that affect behavior / Student has little or no usable knowledge related to economic incentives and is unable to identify examples of incentives at a personal or societal level. / The student has a basic level of knowledge related to economic incentives and is able to identify and discuss examples of incentives at a personal or societal level with significant assistance. / The student has a significant level of knowledge related to economic incentives and is able to identify and discuss examples of incentives at a personal or societal level with minimal assistance. / The student has an advanced level of knowledge related to economic incentives and is able to identify and discuss examples of incentives at a personal or societal level independently and with considerable insight.