i3 Learning Community Kick-Off Meeting Summary

On August 31 and September 1, 2011, 34 representatives from 17 federally-funded Investing in Innovation (i3) projects were joined by guests from several organizations, foundations and the U.S. Department of Education for the initial meeting of an i3 Learning Community. The goals of the two days of intensive small and large group discussions were to:

  • Promote sharing and reflection across projects, roles and disciplines;
  • Provide targeted support/consultation on practical and substantive challenges;
  • Build collective knowledge on topics of mutual interest; and
  • Create opportunities to network and build relationships.

This document summarizes the primarytopics discussed at the meeting and some of the themes that emerged over the course of the discussions.

Fidelity

Grantees face a similar set of challenges related to fidelity:

•Determining the critical elements of the innovation

•Determining how much variety of implementation is acceptable or encouraged

•Sustaining fidelity across fluid environments

•Measuring fidelity

In discussing these challenges, several themes emerged:

  • There are a variety of models within i3, from very loose to very prescriptive.
  • Teachers need flexibility and autonomy to feel engaged and develop ownership of the intervention. Training and coaching systems need to focus on core elements and be flexible enough to allow for variation.
  • Interventionsmust plan for change in schools and districts.
  • Conceptualizations of fidelitydiffer in school vs. classroom level interventions.May be easier to identify core elements and implement with fidelity in classrooms thanin whole school models that have common structures (i.e., principal training) but not common practices.
  • Some models are “tight at the top, loose at the bottom,” where things look different at the classroom level but are tightly structured at the administrative level.
  • Measuring fidelity can surface tensions between implementers and evaluators and ongoing conversation about core and acceptable practices, how to prioritize core elements, and ideal/acceptable levels or thresholds of service is important.

Coaching

Two panel presentations and several small group discussions focused on coaching, a common component of many of the interventions. The following challenges and strategies surfaced:

  • Coaching begins with buy-in for change – teachers have to be willing to participate. Making observations as objective as possible and building buy-in and engagement from the outset are critical.
  • Good coaching is about building relationships – recruiting and developing strong coaches, making sure they understand the model, and letting them do their thing.
  • Fidelity in training is important - both training of the coaches and training teachers in being coached - in order to achieve fidelity in implementation. In some projects a focus on the who, what, where, and when of coaching helpedaddress variability in what coaches were doing Coaching logs also help with monitoring and supporting coaches who got off track.
  • Coaching is expensive. Many projects areinterested in testing out strategies for reducing cost with virtual coaching and other models in particular as they look to scale up coaching efforts.

The role of research in education innovation

The importance of building relationships between evaluators and implementers was a theme that emerged from several discussions at the meeting, and one workshop focused explicitly on the use of research in practice settings. Discussionsrelated to this topic addressed the following themes:

•Research often does not get used by practitioners for several reasons:

  • It is often inaccessible, physically and intellectually, to the average person
  • It is expensive and time consuming to conduct
  • There is a perception that data and research evidence is often misused (that people only use research that confirms what they already believe or can help them politically)
  • There is a perception that evaluators and researchers do not understand the challenges at the school level and vice versa

•Opportunities to increase the use of research in practice may include:

  • Creating more forums for programs to share lessons about what is working
  • Finding ways for researchers and practitioners to co-design research
  • Creating more opportunities for evaluators and implementers to talk about the core elements of interventions, fidelity issues, and how much flexibility and adaptability is desirable

Managing relationship with teachers, school, and districts

Throughout the meeting there was discussion about the importance of building positive and lasting relationships with schools and districts engaged in this work. The following challenges and strategies were raised:

•Selling randomized trials is particularly challenging as schools feel like they have too many requirements and big urban districts are spread too thin

  • It is important to allow for flexibility whenever possible and to talk with districts about their needs and how to adjust the trial to fit their situation

•Student, teacher, principal and superintendent turnover make interventions and evaluation challenging, particularly randomizing at the classroom level

  • Think of change as inevitable and not as an irregularity
  • Look for ways to embed interventions so schools will want to sustain regardless of who comes and goes

•Teacher morale is very low and principals do not want to force teachers to do things

  • Key is to build relationships – learn about the schools, understand how your intervention fits, not over promise, visit, meet teachers and administrators, be honest
  • Be clear about the benefit to the school or district
  • Take time to learn the context, the district priorities, personalities and politics

Reflections

Responses to the meeting were very positive. Participants valued sharing strategies with people working in similar contexts, getting out of their “silos” to share ideas and learn about other types of interventions, and discussing the broader purposes behind i3. Many participants reported leaving with specific practical ideas, especially related torecruitment and measuring fidelity.

The i3 Learning Community is funded by the William T. Grant and Spencer Foundations and supported by the Forum for Youth Investment.