Setting Outcomes for the District Plan
What are Outcomes?
The Massachusetts Planning and Implementation Framework defines outcomes as the plan’s expected results: what they will be, how they will be measured, and when they will occur. Outcomes are the SMART goals for the district: specific and strategic; measurable; ambitious and action-oriented; rigorous, realistic, results-focused; timed and tracked. Outcomes measure the district’s success in achieving its vision and include specific targets for the improvement expected as a result of multi-year plan implementation.
While districts will set both process and early evidence of change benchmarks through the annual action planning process in order to measure progress and impact of initiatives as they are implemented, it is important to set some district outcomes that will track the overall impact of the strategy defined by the multi-year district plan.
Identifying Outcomes
Outcomes are a natural outgrowth of the district’s vision and should be fully aligned with that vision. An effective way to begin conceptualizing outcomes for the plan as a whole is to focus on brainstorming outcomes for each strategic objective. The Planning Team can work in small groups, each focused on one objective, to address the question: Which outcomes are most meaningful and will best measure results for this objective? The team can then review the outcomes proposed for all objectives and address the question: What combination of outcomes will best measure success in achieving our vision?
Criteria to Consider: Public Understanding and District Capacity
As the Planning Team reviews proposed outcomes, the team should assess them in light of the following two questions:
· Will these outcomes be clear and accessible to the public?
· Do we commit to developing the capacity to monitor and report on these outcomes?
Since outcomes measure district improvement and success, it is imperative that they are framed in a way that helps make them meaningful to and easily understood by the public—and that district leaders have the capacity to effectively monitor and report on them. The Planning Team should consider what is realistic and manageable when deciding upon the total number of outcomes for the plan. The team should also consider the data collection needs and requirements of proposed outcomes, and evaluate these needs in light of current district practices and the steps that might need to be taken to strengthen these practices.
For example, as teams work to identify meaningful outcomes for a new vision and plan, they may propose outcomes that require the district to collect new types of data that the district currently does not collect. Or teams may propose outcomes that require the district to develop new practices in working with data that is collected but not consistently monitored. Outcomes such as these require additional planning on the part of district leaders to ensure that the necessary data is collected and monitoring processes are in place.
The “Planning for Success Outcomes Worksheet,” included below, is a helpful resource for teams to use in developing outcomes and identifying data sources and issues related to these outcomes. This resource is also helpful to teams as they prepare to gather and review historical data in order to set specific targets for outcomes.
Setting Outcome Targets
Once the team has identified outcome measures it believes will best measure district success in achieving the vision, the team sets specific targets for these outcomes. Setting targets is a delicate balancing act, as the team strives to strike the right balance between what is achievable and what is ambitious—between what district history may reveal and what district vision aspires to. During the target setting process, the team must address the questions: What degree of improvement is realistic? How much improvement is enough?
To develop a sense of what may be realistic, the team should create a data context for target setting by reviewing and analyzing any existing data related to the outcome. Doing so ensures that targets are grounded in reality rather than in wishful thinking. To create such a context, teams should begin by reviewing the Planning for Success resource, “Selecting Outcome Measures and Setting Targets.”
This PfS resource provides information about how districts might think about and select specific targets. It also provides detailed information about how to set targets for four outcome measures commonly used in district plans, for which comparable data is available statewide:
· Student achievement on MCAS
· Student growth on MCAS
· High school graduation rates
· Chronic absenteeism (the share of students who were absent more than 10 percent of their days of enrollment)
After reviewing this resource, districts will want to review and analyze their own data by:
· Reviewing two or more years of district data related to the outcome and analyzing gains/losses and patterns of district performance
· Comparing district data and performance to state performance using the state’s Profiles
· Comparing district data to the performance of similar districts and schools using the state’s District Analysis and Review Tools (DARTs). The DARTs allow comparisons of up to 10 districts of the user’s choice and also include suggestions for comparable districts based on district type, enrollment, and student demographics.
When considering how much improvement is enough, teams should be ambitious and also reassured that all improvement is valuable and valued. Teams should set targets that reflect accelerated performance each year, with an eye to decreasing any gaps in performance the team observes between the district and the state or the district and comparable districts. Whether these targets reflect a level of improvement that is statistically significant is much less important than whether these targets reflect an upward trend in improvement over time.
If the district has not yet collected the data necessary to measure an outcome, the team should include the outcome without a specific target and conduct the planning required to ensure the data is collected and monitored. The team may choose to indicate the timeframe in which baseline data will be collected and subsequent targets set for the plan.
Planning for Success Outcomes Worksheet
Strategic Objective ______
Instructions: Outcomes are the district’s SMART goals: specific and strategic; measurable; action-oriented; rigorous, realistic, results-focused; timed and tracked. Use this worksheet to draft one outcome for the specified strategic objective; then identify the data sources you will use in setting a target for this outcome. If the district does not currently collect data for this outcome, use this worksheet to begin planning for such data collection.
Outcome (SMART) / Example: The district will increase the 4 year graduation rate for all students to (X)% by 2018.Outcome
(in SMART format)
Note: Insert “X” in place of specific target, if target is unknown
Does the district currently collect the data needed to measure this outcome? / Yes / No
What is the existing or proposed data source/instrument?
Who is, or will be, responsible for collecting this data?
When will this data be collected, with what frequency?
What data will be used as baseline and when will it be collected?
Who will bring existing data for the team to analyze in setting targets for this outcome?
1