Current Research Stresses the Importance of Evidence, Particularly Measurable Outcomes

Formative Portfolio 1

Running Head: FORMATIVE PORTFOLIO DEVELOPMENT

The Effects of Evidence-Based Practices on Formative Portfolio Development among Social Studies Preservice Teachers

Craig E. Shepherd

University of Georgia
Abstract

Portfolio development has received considerable attention in recent years as a method to document evidence of preservice teacher practices. Portfolios have been touted as a method to promote teacher reflection, hone skills, and document effectiveness. However, many portfolios focus on summative assessments. Some researchers have also questioned the value of portfolio production, wonder whether practices learned through portfolio development are continued after hire, and doubt the validity and reliability of portfolio assessments. This paper presents an alternative method for portfolio development that emphasizes systematic evidence collection and examination for formative portfolio construction. Results indicate that while preservice teachers identify benefits of such portfolios for examining practices, creating development plans, and critically reflecting on classroom practices, additional supports are needed to help preservice teachers follow through with these plans.

The Effects of Evidence-Based Practices on Formative Portfolio Development among Social Studies Preservice Teachers

Current research stresses the importance of evidence, particularly measurable outcomes of student achievement, to assess teacher quality (Cochran-Smith, 2005; NCATE, 2002). While several researchers contend that randomly-assigned experiments are the best source of evidence (Cochran-Smith & Fries, 2005; Cook, 2002; Slavin, 2002), this method may be impractical for preservice teachers because they do not have their own classrooms, may not ethically be able to assign students to experiment and control groups, and may not teach long enough to accurately measure the effects of their teaching experiences on student performance and achievement. Preservice teachers may benefit from alternative methods to develop and demonstrate professional development. Portfolio development has received considerable attention in recent years as a method to document evidence of preservice teacher practices. A portfolio is an collection of student work samples, reflections, lesson plans, and other materials (collectively referred to as artifacts) organized around professional standards or frameworks to demonstrate teacher knowledge and skill. Such standards and frameworks are believed to represent effective teaching (Delandshere & Petrosky, 2004; INSTASC, 1992, 2005).

Although portfolio purposes vary among teacher education programs, the majority document preservice teacher practices over specified program milestones. Portfolio production is believed to facilitate reflective practices, aid self assessment, and demonstrate professional knowledge (Borko, Michalec, Timmons, & Siddle, 1997; Fallon & Watts, 2001; Richert, 1991). Additionally preservice teacher portfolio development has recently shifted from paper-based documents to electronic databases where information about standards mastery can be more easily stored, aggregated, and analyzed for accreditation purposes (NCATE, 2002; Reis & Villaume, 2002).

However, portfolio production is not without limitations. Despite promises of portfolio uses for continual growth and reflection, most preservice teachers’ portfolios remain summative in nature—documenting specific skills as fixed milestones (Grossman, 2005). Additionally, some researchers question the time and resource requirements of portfolio production, wonder whether practices learned through portfolio development are continued after hire, and doubt the validity and reliability of portfolio assessments. If portfolios are to become useful tools in teacher education programs, these limitations must be addressed. The purpose of this paper is to critically analyze current portfolio trends in teacher education, present a methodology for transforming summative portfolios into formative tools for growth, and present preliminary findings about preservice teacher perceptions and uses of formative portfolio development.

Adopted in the 1980s as a means to demonstrate holistic teacher practice (Bird, 1990; Wolf, 1991), portfolios are commonly used in teacher education to demonstrate current skills for graduation and licensure (Delandshere & Arens, 2003; INTASC, 2005), institution accreditation (NCATE, 2002), and advanced certification (Burroughs, 2001; NBPTS, 2004). Portfolios have also been adopted by teacher education programs to enhance technology skills (Keefe, et al., 2002; Wright, Stallworth, & Ray, 2002), aid reflective practices (Borko, et al., 1997; Land & Zembal-Saul, 2003), and demonstrate professional knowledge and expertise (Delandshere & Arens, 2003; Fallon & Watts, 2001). Regardless of their application, portfolios are assumed to provide evidence of teacher practice that is unavailable through other methods of inquiry (Barton & Collins, 1993; Bird, 1990; Wolf, 1991).

Despite the adoption of portfolios in several teacher education programs, comparatively little evidence exists regarding their validity for assessment purposes (Borko et al., 1997; Burroughs, 2001; Delandshere & Arens, 2003). In a review of portfolio programs at three teacher education institutions, Delandshere and Arens (2003) found low reliability among portfolio evaluators. They also found that the majority of portfolios failed to adequately explain how artifacts aligned with identified standards. Indeed, the researchers stated that most preservice teachers took “projects developed as part of courses they have taken and retrofit[ed] them into such categories as ‘instructional strategies,’ ‘inquiry,’ ‘planning,’ and so on, without further explanation (p. 65).” Portfolios appeared to comprise documents randomly clustered around professional standards. Because these portfolios failed to identify why artifacts were included under specific sections, assessment was based more on course instructor knowledge of preservice teachers than evidence of their performance. Similar problems were reported by Reis and Villaume (2002). Following portfolio implementation, Reis and Villaume found that only 35% of interviewed faculty members believed that portfolios were valid assessments of teacher preparation because of differences in artifact representations and portfolio assessment practices. Although preservice teachers were asked to reflect upon all artifacts in their portfolio, for example, some faculty believed that these reflections were inadequate and poorly constructed.

Yet, some portfolio projects have proven of significant benefit. In a study by Borko et al. (1997), preservice teachers indicated that portfolio development helped them reflect more deeply about current practices, a major goal of the project. However, researchers also noted that a mentor or coach was needed for adequate reflection and that mandating specific artifacts limited perceptions of individual ownership. Land and Zembal-Saul (2003) reported similar results in an electronic portfolio study of preservice science teachers’ abilities to identify properties of light through classroom activities. While reflection increased among preservice teachers, inadequate content knowledge led some to focus on erroneous aspects of their artifacts; these teachers became increasingly confused by attempting to synthesize and reflect upon faulty evidence. Thus, while portfolios have been shown to be effective tools for encouraging reflective practices, ongoing support is necessary to guide preservice teachers.

While several studies identify portfolio benefits on reflective practices, most focus on portfolios for summative assessment (Carroll et al., 1996; Whipp, 2003; Wright et al., 2003). Portfolios often assume the form of capstone projects that are required for student teaching, graduation, or program accreditation. Although these portfolios document current knowledge, they often fail to establish a trajectory of teacher growth. Additionally, it is unclear whether portfolios constructed for these purposes are sustained or extended beyond course requirements. Although preservice teachers have used graduation portfolios for employment purposes (Loughran & Corrigan, 1995; Theele & Tallerico, 2004), they may not use or maintain them once hired. Indeed, these practices have led some students to consider portfolios as a required assignment rather than a tool for professional growth (Delandshere & Arens, 2003; Wilson, Wright, & Stallworth , 2003). Such portfolio uses would appear to have little prospect to influence teaching and learning.

Because portfolios are becoming increasingly popular within teacher education programs, it is important to establish methods that help preservice teachers collect and interpret artifacts for program purposes, encourage formative assessment, and demonstrate professional growth over time. The need to develop and document professional growth is especially relevant given current legislation calling for increased teacher accountability and evidence of student learning (Cochran-Smith & Fries, 2005). Although some have used standardized tests to assess student learning, portfolio development may add important information about teacher practices, especially their ability to create environments conducive for active student engagement.

Active student engagement is the thoughtful, reflective, or mindful activity by which learners receive, process, manipulate, judge, or interpret knowledge to enhance their understanding of subject matter (Black, Sileo, & Prater, 2000; Dewey, 1910; Wiggins & McTighe, 1998). Active engagement requires more than rote memorization and recall; learners gather, evaluate, and organize information to uncover complex, contradictory, and abstract ideas, or apply knowledge in flexible ways (Dewey, 1910; Shepardson & Britsch, 2006; Vonderwell & Turner, 2005). Because of the role active student engagement is believed to play in learning, it is posited as a basis for self-study and reflection in teaching (Dinkelman, 2003; Shepardson & Britsch, 2006).

In summary, although teacher education programs are adopting electronic portfolios at increasing rates in the United States, many students view them as capstone projects rather than tools for development. Additionally, little evidence exists about their validity in assessing teacher practices and several researchers have described their limited reliability in scoring. If portfolios are to gain a permanent place in teaching and learning, these issues must be addressed. Enacting portfolio development using Evidence Based-Decision Support (EBDS), a methodology for systematically examining teacher practices may help alleviate some of these concerns, particularly those dealing with artifact alignment, reliability, and establishing a trajectory of growth. EBDS is designed to help preservice teachers iteratively identify, generate, and interpret evidence of practice in their own teaching (see Figure 1). Through this method, preservice teachers select a specific aspect of their teaching for inquiry, gather and analyze evidence related to their selected practice, and enact changes as needed for improvement (Recesso et al., in press). The following section will describe how EBDS methods may transform portfolio development. The researchers will then describe how EBDS-based portfolios were developed during a capstone seminar for preservice social studies teachers.

Evidence-Based Decision Support

Rather than have preservice teachers identify projects, lesson plans, and other artifacts that represent their mastery on given standards, include it in their portfolios, and write reflections about them, portfolios using EBDS methods begin by having preservice teachers focus on a specific teaching method or technique. Once this method is identified, preservice teachers collect evidence regarding its implementation in their classroom. If, for example, individuals are interested in increasing wait time between questions and responses, they may video record themselves attempting to increase wait time, review current literature regarding wait time and active engagement, obtain written feedback from field instructors and cooperating teachers, and so forth.

While preservice teachers collect evidence relating to their specified method, they concurrently organize and examine it using professional standards, comparison techniques, and reflective questions to map specific practices onto professional attributes of teaching, gauge the effectiveness of method implementations, and identify strengths and areas in need of improvement. Based on identified deficiencies, preservice teachers then select an aspect of their practice in need of development, plan implementation strategies, and employ them. For example, after reviewing video tapes on wait time, reviewing literature on effective questioning strategies and planning with a cooperating teacher, a student teacher may determine that while their wait time was sufficient, they only asked discussion questions requiring rote memorization and recall. They may therefore decide to focus on asking questions requiring the comparison and contrast of several sources of information for future implementation and begin by including potential questions of that nature in lesson plans. Preservice teachers then enact their plans, collect and examine evidence related to their implementations, and determine whether they benefited from them. In this way, EBDS is an iterative process of data collection, examination, and change.

In the previous example, EBDS methods may help a preservice teacher systematically examine the degree to which active student engagement occurs in the classroom. Furthermore, EBDS methods may provide researchers with additional insights into the formation of such communities over time and across teacher experience levels. The potential for EBDS to enrich traditional portfolio content and functions appears considerable. The remainder of this paper will describe the implementation of EBDS portfolios in one social studies teacher education course at a large university in the southeastern United States. Specifically, the researcher addresses how the portfolios were implemented, what supports were provided, student perceptions towards these portfolios, and the ability of the program to transform traditionally summative portfolios into formative portfolio tools. Following this discussion, the paper concludes with several implications for future research on the viability of formative portfolios in teacher education.

Methods

Setting

As a capstone project for the Secondary Social Studies Teacher Education Program, all preservice teachers create an electronic portfolio. This portfolio fulfills course requirements in a seminar taken concurrently with a twelve week field experience. Historically, the e-portfolio focused almost exclusively on reflective writing. The bulk of the assignment consisted of writing a rationale statement for teaching and documenting its relevance to six state-based standards of teaching through written reflection. Although preservice teachers included one lesson plan, document, picture, or student work sample representing proficiency for each standard, these artifacts were rarely assessed. Additionally, seminar instructors identified several limitations with the e-portfolio model despite its implementation for five years. Many preservice teachers assembled their e-portfolios during the final weeks of the course, limiting the extent to which they documented progression through student teaching. Additionally, many faculty members felt that artifacts included in e-portfolios did not adequately represent their intended standards nor reliably depict teaching practices. Lastly, most faculty members doubted that students continued portfolio development beyond graduation.

To address these problems, additional portfolio requirements were added to one section of the seminar during Fall Semester, 2006. Using EBDS, preservice teachers selected a method believed to affect active student engagement from a list of possibilities provided to them by the course instructor. They then collected evidence regarding their implementation (including at least one video recording of their teaching—if possible), examined this evidence while filling out a series of reflective questions, and identified areas related to their teaching practices in need of further development. They were then given the option to either continue examining that method in more detail or select another one from the list of possibilities. During each step of EBDS portfolio development, reflective questions within the e-portfolio helped preservice teachers document their decisions and consider alternative explanations from their evidence (see Table 1). Each preservice teacher used these strategies to complete three inquiry projects embedded within their e-portfolio assignment during their twelve-week field experience. In addition to these inquiry projects, the e-portfolio retained the rationale statement, resume, and reflective responses regarding each of the six state standards. Preservice teachers were told that they could use these inquiry projects as artifacts for proficiency in one or more of these standards so long as they made the connection explicit.