Massachusetts School Redesign Grant
Report of Preliminary Statewide Findings
Submitted to the Massachusetts Department of Elementary and Secondary Education
September 2012
Report of Preliminary Statewide FindingsSeptember 2012
Project Staff
Greta Shultz, Project Manager/Senior Research Manager
Lonnie Kaufman, Senior Research Manager
Annette Hunt, Research Manager
Molly Breitbart, Research Analyst
Steven Ellis, Director, Research and Evaluation
This report is prepared by the project evaluator for the School Redesign Grant project, Fund Codes 511 – 767, Massachusetts Department of Elementary and Secondary Education
The University of Massachusetts Donahue Institute is the public service, outreach, and economic development unit of the University of Massachusetts President’s Office. Established in 1971, the Institute strives to connect the Commonwealth with the resources of the University through services that combine theory and innovation with public and private sector applications.
The Institute’s Research and Evaluation group specializes in applied social science research, including program evaluation, survey research, policy research, and needs assessment. The Research and Evaluation group has designed and implemented numerous innovative research and evaluation projects for a variety of programs and clients in the areas of education, human services, economic development, and organizational development.
University of Massachusetts Donahue Institute
Research and Evaluation Group
100 Venture Way, Suite 5
Hadley, MA 01035-9462
(413) 587-2400
(413) 587-2410 to send a fax
www.donahue.umassp.edu
/ UMass Donahue Institute
Research and Evaluation Group
School Redesign Grant / Contents
Contents
Tables and Figures i
Executive Summary i
I. Introduction 3
Organization of the Report and Notes on Style 7
II. Methodology 8
Evaluation Design 8
Data Collection Activities 9
Document Review 9
Study Participant Interviews 10
District Leaders 10
ESE District Liaisons 10
Principals, Teachers, and School Staff 10
Principals’ Survey 11
Analysis 11
Limitations 11
III. Results 12
A. Overview: Perspectives on Redesign 12
B. Perceptions of SRG Associated with Improvements 17
B.1. The District’s Role in Redesign 17
District support tends to take the form of passionate and engaged – yet isolated – individuals 17
Call for coordinated, system-wide efforts 18
Perceptions of the extent and nature of district involvement in redesign, and implications 20
Autonomy: Principals’ staffing autonomy is seen as critical to redesign success (sustained momentum and stable workforce) 21
Budget, curriculum and scheduling autonomy valued by principals 23
Other valuable aspects of district involvement: Networking opportunities, partnerships and coaching 23
Partnerships: valued district component and need for increased district involvement 23
Coaching: a multi-purpose solution to challenges inherent in redesign (staff turnover, principal mentorship, teacher growth, student growth) 24
Additional views on effectiveness—and needed changes—in district-level support 25
“The district is like a ghost”: Call for improved understanding of schools (increased time and more direct interaction between districts and schools) 28
Additional suggestions for effective district support: targeted feedback and differentiated PD 28
Sensitivity to political issues 29
B.2.Leadership in Redesign 29
Principals’ Role: Expectations and Burnout 30
Distributed Leadership 32
Challenges of Distributed Leadership 33
Implications 34
Strategies to enhance the effectiveness of distributed leadership models 35
B.3. School Redesign and Instruction 36
Coaches and Specialists 36
Synergy: Results accrue through integrated efforts and strategies 37
Formal Collaboration and Planning Time 38
Extended Learning Time and Restructuring the School Day 39
Restructuring the Schedule: Principals’ Needs for Technical Assistance and Coaching 40
Tiered Instruction: Calls for PD and for Differentiated Support 41
Collection and Use of Data to Inform Instruction 42
Professional Development: One Size does Not Fit All 43
B.4. Approaches to Supporting Social-Emotional Health under Redesign 43
Ambiguity about schools’ capacity to influence change in this arena 44
Perceived effectiveness of SRG-supported strategies 44
Examples: key factors include onsite services, key support staff positions, professional development, and partnerships 46
Synergy of SRG strategies yields perceived impact on school culture and students’ attitudes and beliefs about school 47
Additional areas of need 47
B.5. Focus on Sustainability: Planning Needed 47
C. Feedback to ESE: Implementation of the School Redesign Grant program 49
Sensitivity to Competing Demands on Level 4 Schools 50
Sensitivity to Politics and Public Perception 50
ESE’s Accountability to Schools and Districts 51
Requests for ESE to Spend More Time in Schools 51
School Redesign Tools and Processes 52
Monitoring Site Visits 52
Measurable Annual Goals (MAGs) 54
SRG Applications 54
Renewals 54
A call for Support with the Application and Reapplication Processes 55
Role of the ESE Liaisons: Opportunities for Enhanced Roles and Responsibilities 55
Further support 56
IV. Conclusions and Recommendations 57
Appendix A: Research Plan 61
Appendix B: District Leaders Protocol and Interview Guide 69
Appendix C: Survey Instrument 74
Appendix D: Open-ended Responses 93
/ UMass Donahue InstituteResearch and Evaluation Group
School Redesign Grant / Tables and Figures
Tables and Figures
Table 1. Effectiveness of SRG/Turnaround Strategies 13
Table 2. Staff Turnover in the First Year of Redesign 14
Table 3. Extent of Positive Impact of Staff Turnover 14
Table 4. Financial and Human Resources are Sufficient for Turnaround Goals 15
Table 5. Perceptions of Three-year Time Limit in Relation to Change and Goals 16
Table 6. Supports Provided to the Principal by the District 18
Table 7. Autonomy: Need for District Involvement 21
Table 8. Effectiveness of School- and District-based Coaches and Specialists 24
Table 9. Frequency of Activities Needed to Further Progress Toward Goals 25
Table 10. Usefulness of District Supports 26
Table 11. Elements of District Support Found Useful 26
Table 12. Limitations of District that Make Support Less than Effective 27
Table 13. Expectations of Redesign Principals 31
Table 14. Effectiveness of School- and District-based Coaches and Specialists 36
Table 15. Extended Time Option Used by School 39
Table 16. Impacts of Extended Time Strategies 39
Table 17. Primary Challenges to Consistent Implementation of Tiered Instruction 41
Table 18. Effectiveness of Strategies for Students’ Social, Emotional, and Health Needs 45
Table 19. Usefulness of ESE’s Support in Progress Towards Change 49
Table 20. Usefulness of ESE’s Redesign Tools for Promoting Change 52
UMass Donahue InstituteResearch and Evaluation Group
School Redesign Grant / Executive Summary
Executive Summary
The Massachusetts Department of Elementary and Secondary Education (ESE) contracted with the University of Massachusetts Donahue Institute (UMDI) to design and conduct a program evaluation of the School Redesign Grant (SRG) program. The purposes of the evaluation are:
§ to provide ESE with the type of formative feedback that would support continuous improvement in school, district, and ESE implementation of SRG
§ to capture evidence of short- and (eventually) long-term changes, developing explanations of changes (progress, success) that are likely associated with elements of SRG
From January through June 2012, the evaluation conducted the following data collection activities: document review of extant program materials, including School Redesign Plans and relevant application and monitoring reports; background interviews with ESE program staff, including district liaisons; individual and small group interviews with district leaders responsible for aspects of redesign efforts; site visits to a small sub-set of SRG schools for the purpose of conducting interviews with principals and school staff members; and an online survey of School Redesign principals. A full discussion of the results is contained in the body of the report; selected key findings are presented below.
Key Findings
· The power of the school redesign initiative lies in the synergy of a range of turnaround strategies, employed within the framework of a clear vision and focused goals.
One of the goals of the evaluation is to capture educators’ perceptions of the effectiveness of various elements of the school redesign approach to turnaround. Of interest are the various elements of the federal turnaround models, ranging from replacement of school leaders to efforts to reconfigure teacher collaboration, and a host of other strategies. Overall, study participants tend not to point to any one strategy as solely explanatory of their experience with redesign. In fact, the degree of similarity in response to questions attempting to distinguish between the federal models—“transformation” and “turnaround,” for example—is remarkable. A notion emerges from the data that the integration of the multiple turnaround strategies counts most. Analysis suggests that while certain strategies do tend to rise to the surface––exemptions and allowances negotiated with teachers unions, for example, or the principal’s hiring and budget autonomy—it is, largely, the synergy that results from turnaround strategies being employed together in multiple ways in the service of a broader goal that matters.
· School redesign calls for a systematic approach to school change: coordination, integration, and alignment are the cornerstones of significant and sustainable improvement. To date, districts demonstrate underdeveloped system-wide capacity with respect to school redesign. Rather, district support tends to take the form of passionate and engaged––yet isolated––individuals.
While efforts to systematize supports to schools are evident across the Commonwealth, the study suggests that, largely, redesign efforts still hinge on the efforts carried out by designated—and, typically, isolated-—individuals in districts. The degree to which these individuals are integrated into more coherent and cohesive district systems necessarily varies. Overall, however, evidence suggests a need to reinforce district-level capacity so that knowledge and awareness are disseminated more broadly and are translated into efforts to effect change at the school level. The study suggests that, while schools have had to undergo a process of comprehensive self-analysis and reflection, parallel processes have not yet been introduced at the district level.
· “The principal can’t do it alone”: redesign calls for new models of school leadership, differentiated professional development, and a variety of leadership capacity-building steps.
School redesign principals demonstrate a broad range of approaches to leading turnaround processes, including the adoption of distributed leadership models. A range of distributed leadership models is being developed across the Commonwealth, and while the effectiveness of these models varies from site to site, distributed leadership is cited by principals as one key mechanism to protect against “burnout.” Additionally, distributed leadership models serve as mechanisms to support teacher growth, communicate and foster buy-in to a school-wide mission, and reinforce the likelihood that changes will be extended and sustained in schools. Given the complexity of demands facing redesign principals, calls for increased and differentiated support are evident: customized tailored professional development that takes into account principals’ specific skill sets, professional development/mentoring targeted toward administrative and managerial skills (such as how to budget effectively and how to manage a complicated school schedule), and the establishment or expansion of collaborative forums/networks that allow principals and other instructional leaders to learn from one another’s experience.
· Sustainability of change after the funding period is a top concern for principals: the loss of key staff figures high on their list of concerns. Principals tend to feel that the three-year time limit has negatively influenced their ability to plan for sustainability.
While principals and teachers point to certain practices that they believe will be sustained after the funding period (e.g., collegial practices, cultural shifts), SRG has created and supported critical staff positions, such as instructional specialists and coaches. Many principals, staff, and district leaders worry about losing staff members who are instrumental to the redesign effort and have contributed significantly to success. Addressing the possible loss of instructional specialists, staff in several schools explained a perceived need for more, not fewer, specialists in the schools, especially in ESL and special education. The possibility of losing specialists is demoralizing for teachers who recognize the cycles of improvement and decline that have historically plagued their schools. Overall, leaders believe that three years is an insufficient amount of time to achieve rapid growth and work toward the sustainability of change. Principals typically possess insufficient knowledge about “what happens next” (after the grant period, after their schools exit Level 4 status). They look to the districts and to ESE for information, a plan, and/or answers to their questions. Leaders also call for a longer funding period.
· Overall, study participants appreciate the ongoing support that ESE provides. Broadly, district and school leaders request that ESE develop greater understanding of the realities that plague Level 4 schools.
School and district leaders appreciate ongoing communication with ESE and the technical support provided. Against that backdrop, there is a call for ESE to spend more time in schools so as to better understand individual school needs and to provide ongoing, customized support. Principals highlight the need for ESE to be aware of a) the political sensitivities surrounding school redesign (e.g., perceived inequities, experiences of failure) and b) the multiple initiatives at work in many Level 4 schools.
UMass Donahue InstituteResearch and Evaluation Group / 26
School Redesign Grant / Introduction
I. Introduction
In 2010, Massachusetts launched its first Request for Proposals inviting districts to apply for School Redesign Implementation Grant funding. These funds were made available initially through a combination of Massachusetts’ FY09 federal School Turnaround Grant (STG) and funds allocated through the American Recovery and Reinvestment Act (ARRA)/US Department of Education School Improvement Grants (SIG) program. The funds were intended to support districts in their implementation of one of four federally defined school intervention models: “turnaround,” “restart,” “transformation,” or “closure.”
Massachusetts had recently instituted important policy changes relative to school improvement. The 2010 Act Relative to the Achievement Gap signified a renewed and expanded focus on improving student learning in traditionally underperforming schools. Specifically, the Act provided districts greater power and authority with respect to these schools, and it designated districts as the official point of entry for the Massachusetts Department of Elementary and Secondary Education (ESE). Largely, the Act signified that districts would be held accountable for reversing the patterns of underperformance that had plagued their systems in the past. “Turnaround” was the new order of the day. Additionally in 2010, ESE established the Framework for District Accountability and Assistance. The Framework introduced a five-level scale to identify schools based on need and incorporated the Conditions for School Effectiveness, thereby linking accountability and assistance measures.