STUDENT LEARNING OBJECTIVES (Slos)

STUDENT LEARNING OBJECTIVES (SLOs)

Annotated Notes

M3-Reviewing

Slide No. / Notes /
Slide 1 / The Student Learning Objective (SLO) Process is comprised of three phases: Design, Build, and Review. Student Learning Objectives provide a valid assessment of teacher effectiveness through student performance outcomes based on standards.
Welcome to Training Module 3: “Reviewing”
Concept
Within the Review Module, the trainer will guide participants to implement a multifaceted, three-level review of Student Learning Objectives, including identified performance measures.
Key Points for Trainers
1. Explain in macro-terms the “big picture” of how SLOs are but one component of a much larger teacher effectiveness system.
a. As required by Act 82, an Educator Effectiveness Rating Tool must contain measures based on teacher observation and practice and multiple measures of student performance. Student Performance data will include the following: Building Level Data (15%), Teacher Specific Data (15%), and Elective Data (20% for teachers with eligible PVAAS data, 35% for teachers without eligible PVAAS data).
b. This Educator Effectiveness Rating Tool functions as a framework for the evaluation and summative process for classroom teachers.
2. Clarify that the Review Module will guide participants through a three-level, quality assurance review of the created Student Learning Objective and the identified Performance Measures. The Review phase helps ensure that the operational and technical expectations of the process are met or exceeded.
3. Ensure participants understand that this phase is the quality assurance component, focusing on issues that range from completing the SLO Process Template correctly to examining the validity evidence of a vendor-made assessment/performance measure.
Learning Activity
______
Technical Notes
“Structure”
1. Concept – “What is this slide telling the audience?”
2. Key Points – “What/Where are the details ‘needed for teaching’?”
3. Learning Activity – “How can the participant’s learning be enhanced?” (This item will not be populated for every slide.)
Slide 2 / Student Learning Objective Review
Concept
As you work through the Review phase, you will need to access the following materials:
1) Handout #4 – SLO Procedural Cheat Sheet
2) Handout #5 – Quality Assurance Checklist-SLO
3) Template #6 – Performance Measure Rubric
4) Template #7 – SLO Coherency Rubric-PDE
5) Refinement Control Checklist (on-demand Performance Measures only, found in SLO/Build/Resources)
Key Points for Trainers
1. Assist the participants in conceptualizing that the SLO Process Template (Template #4) is organized into two major components: SLO Goals and Performance Measures on page 1 of the template, and Performance Indicators and Elective Ratings on page 2.
2. Clarify that the comprehensive evaluation of a new performance measure cannot be fully completed until after the measure is administered and data is collected. The current tools focus on the development and administration of the measures, with additional training being developed for the “post-administration” review. For vendor-made measures, there should be sufficient evidence on the inferences of scores, reliability, etc. that also includes how the performance measure was developed.
3. Stress to participants that the Review process includes a quality assurance check on each Performance Measure selected to ensure that they are aligned with the targeted content standards, free of bias and sensitive content, and developmentally appropriate.
Learning Activity
1. Have participants go to www.pdesas.org to retrieve the five items mentioned above from the SAS Portal /Homeroom learning portal. Also make sure they have the completed SLO Process Template (Template #4) found in SLO/Build/Templates.
Slide 3 / Process Components
Concept
This slide reviews the parts associated with each phase of the SLO Process. This Module will focus on the Review Phase of the SLOs.
Key Points for Trainers
1. It will be important for each educator to have all the completed templates from their work with Module 1: Design and Module 2: Build. That would include the following templates:
a. Template #1-Goal Statement (found in SLO/Design/Templates)
b. Template #2-Targeted Content Standards (found in SLO/Design/Templates)
c. Template #3-SLO Blueprint (found in SLO/Design/Templates)
d. Template #4-SLO Process Template (found in SLO/Build/Templates)
e. Template #5-Performance Task Framework (found in SLO/Build/Templates)
2. A key point for teachers is that the Review phase is iterative, meaning that quality assurance criteria are designed to inform corrective action rather than simply assign a score. Likewise, the performance measure quality requires at least one full administration to determine basic psychometric characteristics of newly developed assessments. Validity evidence builds upon successful iterations of the assessment “lifecycle.”
Learning Activity
Slide 4 / Session Objectives
Concept
The three-tier quality assurance review process of each SLO includes the following:
• Completeness
• Using the SLO Procedural Cheat Sheet (Handout #4) and the Quality Assurance Checklist (Handout #5), educators will determine the completeness of the SLO Process Template (Template #4).
• Comprehensiveness
• Using the Performance Measure Rubric (Template #6) and the SLO Procedural Cheat Sheet (Handout #4), educators will evaluate the quality of each teacher-made performance measure.
• Coherency
• Using the SLO Coherency Rubric (Template #7) and the SLO Procedural Cheat Sheet (Handout #4), educators will evaluate the entire SLO Process.
Key Points for Trainers
1. Emphasize that all performance measures are evaluated in terms of high quality and rigor. Template #6 - Performance Measure Rubric (found in SLO/Review/Templates) will guide educators through each phase of the build-out; Design, Build, and Review.
2. Reinforce the 3 C’s concepts (Completeness, Comprehensiveness, Coherency) so the participants understand the reason each aspect (“C”) must be considered before statements regarding “high quality” SLO can be made and supported.
Learning Activity
1. Have the participants spend 5-10 minutes reviewing and discussing in their small groups the different tools and how they are used in evaluating an SLO. After this exploration, have the members select a model and “test drive” Handout #5 - Quality Assurance Checklist.
Slide 5 / Task Structure
Concept
The first step in the Review Process is to check for the completeness of the SLO Process Template. This step ensures that the SLO Process Template is complete. While reviewing, any incomplete, incorrect, or missing element will be highlighted and potential corrective actions will be identified. The “Help Desk” (found in SLO/Build/Templates) provides detailed information on filling in the template.
The second step in the Review Process is to check the comprehensiveness of the performance measures selected and/or developed. Educators will be guided through a process to review each aspect of the Performance Measure: Design, Build, and Review (checking for quality). The Performance Measure Rubric (Template #6) and Quality Assurance Screening Checklist are helpful tools. For vendor-made assessments, a more detailed rubric is available.
The final step in the Review Process is to check the coherency of the SLO Process. Coherency is about examining the “parts” of a teacher’s SLO and verifying the alignment to the “Big Idea”. The SLO Coherency Rubric (found in SLO/Review/Templates) focuses on examining how the components fit into a cogent process for applying evidence of student achievement to teacher evaluation.
Key Points for Trainers
1. Clarify that when evaluating the entire SLO Process (coherency) educators will be checking to ensure that everything is aligned with the Goal Statement and Targeted Content Standards. Handout 4-SLO Procedural Cheat Sheet and the SLO Coherency Rubric (Template #7) will aid educators through the process and identify any issues that will need to be addressed.
2. Emphasis that completing the form correctly is a relatively straight forward task and the evaluation of a performance measure’s technical data is much more complex and time consuming. It (Comprehensiveness) should be viewed as a two-stage process for both the SLO and the performance measures within the SLO: before administration and after administration.
Learning Activity
Slide 6 / SLO Design Coherency
Concept
This figure is designed to help visualize how the “parts” fit into the “whole.”
Key Points for Trainers
1. It is essential for the audience to fully understand how only a coherent SLO approach should be used within the greater Educator Effectiveness System.
a. Goal Statements are the “Big Idea” upon which the SLO is based. These goals must be based upon PA standards and provide a rationale for choosing a particular standard which addresses the important student learning.
b. Performance Measures are the various assessments which will be used to measure student achievement toward a specific goal.
c. Performance Indicators are the expected levels of achievement for students in the SLO population based on the performance measures.
d. Ratings are determined by the overall results given the expectation ranges established at the beginning of the year.
Learning Activity
1. Have participants go to the SAS portal and review the Curriculum Framework. “The Curriculum Framework specifies what is to be taught for each subject in the curriculum. In Pennsylvania, Curriculum Frameworks include Big Ideas, Concepts, Competencies, and Essential Questions aligned to Standards and Assessment Anchors and, where appropriate, Eligible Content.” (SAS Portal)
Slide 7 / Helpful Tools
Concept
As noted previously, many resources are available for educators to use during the Review process. They include the following.
1. Handout #4 – SLO Procedural Cheat Sheet (found in SLO/Review/Resources)
2. Handout #5 – Quality Assurance Checklist (found in SLO/Review/Templates)
3. Template #6 – Performance Measure Rubric (found in SLO/Review/Templates)
4. Template #7 – SLO Coherency Rubric (found in SLO/Review/Templates)
5. Refinement Control Checklist (on-demand Performance Measures only; found in SLO/Build/Resources)
All training documents and materials can be found on the SAS Portal and RIA’s Homeroom learning portal. Go to http://www.pdesas.org and you will see a link to RIA’s Homeroom portal as well.
Key Points for Trainers
Learning Activity
1. Have participants retrieve and review the Handouts and Templates associated with the Review Process.
Slide 8 / Final Deliverables
Upon completion of Step 3: Review; educators will have a completed SLO Process Template that identifies the rigorous, high-quality performance measures as well as any area that was identified through the process needing improvement.
Key Points for Trainers
1. Articulate the expected deliverables for teacher-made performance measures:
a. Purpose Statement
b. Targeted Standards
c. Specifications and Blueprints
d. Scoring Keys and Rubrics
e. Operational Forms (tests/tasks)
f. Administrative Guidelines
2. Expected deliverable for vendor-made performance measures:
a. This deliverable will depend on the measure adopted by the school and/or district (i.e., DIBELS, AIMS Math, etc.); however, the vendor will often have written evidence and data supporting the measurement claims. Educators should simply reference this information, thus demonstrating that the selection of the measure meets the intended need of the educator (i.e., to measure the targeted content standards within the “Big Idea”).
3. Using the Performance Measure Rubric (Template #6) and other tools found within the Quick Start training, participants will be able to identify/document any shortcoming found and determine corrective actions needed.
Learning Activity
1. Probe the audience for their understanding of what is meant by a “rigorous performance measure” using the below set of questions:
a. To what degree does the performance measure align with targeted standards?
b. What is the cognitive demand of the tasks/items, both individually and in totality?
c. Are the performance measure’s reading level and vocabulary, age and grade appropriate?
d. Are the targeted skills appropriate?
e. Are the items/tasks clear, concise, and understandable?
f. Are the items/tasks free of bias?
g. Are the items/tasks sensitive to different cultures, religions, ethnic and socio-economic groups?
Slide 9 / Generic Review Workflow
Concept
This slide provides a general overview of the workflow for the Review Process.
“Work flow” within this context means the step-by-step Review process needed to ensure high-quality SLOs have been created.
Key Points for Trainers
1. Step 1: Create the SLO “packet”
a. Checking to make sure that all required documents are included in the packet for review.
i. Template #4 – SLO Process Template (found in SLO/Build/Templates)
ii. Optional -Template #5 – Performance Task Framework (for each Performance Indicator; found in SLO/Build/Templates)
iii. Performance Measures
2. Step 2: Review for Completeness – Using the SLO Procedural Cheat Sheet (Handout #4) and the Quality Assurance Checklist (Handout #5)
a. Review each section of the SLO Process Template to make sure it has been completed correctly.
b. Highlight any missing or incomplete elements.
c. Circle any incorrect element and outline the discrepancy.
d. Recommend corrective actions.
3. Step 3: Evaluate SLO for Comprehensiveness – Using the Performance Measure Rubric (Template #6), participants will evaluate the quality of each performance measure. This rubric is comprised of 18 technical aspects of quality assessment organized into three parts. Participants will rate each element and list supporting technical evidence, provide further explanations and details where needed, and make any recommended actions for shortcomings.
a. Part 1: Design – has five (5) elements
b. Part 2: Build – has six (6) elements
c. Part 3: Review – has seven (7) elements
4. Step 4: Determine SLO Coherency – Using the SLO Coherency Rubric (found in SLO/Review/Templates), participants will review the entire SLO Process. This rubric has several technical requirements divided into three (3) Parts. Through this review process, participants will check for alignment among all the elements (goal statement, targeted content standards, performance measures, performance indicators), identify any issues, and make recommendations for corrective actions.
a. Part 1: Design – has five (5) elements
b. Part 2: Technical – has six (6) elements
c. Part 3: Quality – has four (4) elements
5. Step 6: Identify and Apply Corrective Actions – There are several places where corrective actions can be identified and documented throughout this process. During this step, shortcomings are identified and then corrected, or a plan for future corrections is established.
Learning Activity
Slide 10 / Summary
Concept
During Module 3 participants reviewed, refined, and corrected the SLOs and the Performance Measures associated with each Performance Indicator. At this point, teachers should have all completed templates that were required throughout the entire SLO Process.
Key Points for Trainers
1. This professional development provided participants with a better understanding of several quality assurance practices used in the SLO Process.
2. As noted previously, the SLO process was created to meet the need of Act 82, which requires PA to have an Educator Effectiveness Rating Tool that contains measures based on teacher observation and practice as well as multiple measures of student performance. SLO is the process that develops an Elective Rating through evidence of student achievement.
3. The SLO Process also assists educators in developing/selecting fair, valid, high-quality measures to evaluate student achievement.
Learning Activity