Standard 2. Assessment System and Unit Evaluation
2a. Assessment System
Wittenberg’s teacher education program has an assessment system that directly connects to the department’s conceptual framework and reflects a commitment to character, competence, and community. The department uses these three dimensions of candidate development as the framework for organizing performance outcomes for teacher candidates (EXHIBITS OV4_PerfOutcomes, S1_4_PerfOutcomesRubric). As an ongoing and critical evaluation of the department’s assessment structure, the faculty officially adopted, through faculty vote, nine program outcomes (for the initial preparation program) during the Spring 2012 semester. The department created a crosswalk with the previous 18 outcomes to aid in the process of showing program congruency with the conceptual framework, even with this evolution (EXHIBITS OV_3_ConceptualFramework and S2_1_OutcomesCrosswalk).
The department connects key assessments to the standards found within each Specialized Professional Association (SPA) connected to our teacher education programs (EXHIBIT S1_17_ReqSPAAssess). Data from key assessments are collected by those instructors teaching courses that contain key assessments. Evaluation of data and program improvement plans are reported to the Specialized Professional Association and posted in the Accreditation Information Management System (AIMS) on the Council for the Accreditation of Educator Preparation (CAEP) website (http://caepnet.org/). As of Fall of 2014, the department offers eleven nationally-recognized programs and two state-recognized programs, as well as, state-approved endorsements at the undergraduate, post-baccalaureate, and graduate level (EXHIBIT S2_2_EPPprograms).
Wittenberg’s comprehensive and integrated assessment structure (EXHIBIT S2_3_Assessments Overview) monitors candidate performance and provides data to drive department improvement plans. Beyond key assessments connected to program standards, the department has gates at critical points within teacher education programs (EXHIBIT S2_4_StudentCycle). Gateways for the department are admit to the program, permission to student teach and program completion/recommendation for licensure (EXHIBIT S2_5_UndergradGateways). Admit to the program occurs during the semester the candidate completes course prerequisites. The candidate submits an application to the department and the department considers grade point average and outcome ratings when taking action on the application. The second gate occurs the semester prior to student teaching. The teacher candidate applies for student teaching and the department reviews grade point average, outcome ratings, and completion of required licensure and content courses prior to placement. The third gate determines successful completion of the program, by reviewing final grades and outcomes from student teaching. These gates (checkpoints) provide opportunity for the department to make informed decisions about candidate performance using multiple assessments with multiple evaluators, e.g., edTPA not scored locally, etc.). The program review system assures that the assessments being used are aligned with local, national and professional communities.
The goals for the graduate program are clearly articulated in the first two pages of the Graduate Student Handbook (EXHIBIT S1_13_GraduateGoals). Candidates in Wittenberg’s advanced programs for teachers (Master of Arts in Education) focus on learning goals that extend, expand, and enrich 1) knowledge of content, schools, and students; 2) skills to plan and assess; and 3) attitudes and values. In Wittenberg’s Master of Arts program, practitioners examine their own purposes and mission. They develop research questions that will help them serve students, the school, and the community. The program uses an ‘action research’ model to guide the graduate student to providing evidence of scholarship through the development of a thesis (EXHIBIT S1_23_ThesisGuide).
The course structure of the graduate program directs students through the action research and results in the capstone thesis project (EXHIBIT S2_6_GradGatewayAssessments). Candidates defend their work in an oral defense to the thesis committee (consisting of three members). In addition, candidates share their research at a Department Research Forum where the study can inform a larger audience. The public presentation as well as the oral defense before a committee serve to eliminate bias and promote fair, accurate, and consistent departmental decisions.
2b. Data Collection, Analysis, and Evaluation
Multiple assessments from internal and external sources provide the data the department uses to assess the teacher preparation programs (EXHIBIT S2_7_ AssessReviewCycle). Data used to make decisions about the program include:
1) Evidence of Effective Teaching and edTPA Projects (EXHIBITS S2_8_EET_Data, S2_9_edTPA_Data).
2) Student teaching evaluations completed by cooperating teachers and university supervisors based on the final student teaching evaluation form (EXHIBIT S2_10_FinalSTEval), and includes rubric addendums for Middle Childhood, Math, Science & Foreign Language teacher candidates (EXHIBIT S2_11_Addendums)
a. For disaggregated results by program (EXHIBIT S2_12_StuTEvals_by_Lic)
b. For aggregate results for all programs (EXHIBIT S2_13_STEvals)
3) Employer surveys completed after students exit the program and following the first year of employment (EXHIBIT S1_11_PostGradSurveys),
4) Student survey completed after completing the teacher education program (EXHIBIT S1_11_PostGradSurveys),
5) Data from licensure assessments such as Praxis (EXHIBIT S1_11_StateTests), and
6) Faculty ratings of student performance during coursework and field experience
a. For activities in relation to the department’s nine program outcomes (EXHIBIT S1_8_StuOutcomeReviews).
b. For explanations included with the program outcomes (EXHIBIT S1_4_PerfOutcRubric)
7) Ohio’s Metrics for Teacher Preparation Quality reporting compiled by the Ohio Board of Regents (EXHIBIT S2_14_MetricsReport2013)
The department maintains a regular and systematic data collection process (EXHIBIT S2_3_AssessmentsOverview). Course-related data are collected and archived by the course instructor. Additionally, upper-level courses contain key assessments which we score, collect and store through Taskstream, a cloud-based software. Students have access to their performance data through their Taskstream account, as well as direct feedback from the instructor.
Final student teaching evaluation data are collected and archived using Survey Monkey then downloaded to Microsoft Excel for department analysis and display. Student Teachers are provided with a final student teaching evaluation from both their cooperating teacher and university supervisor (EXHIBIT S2_15_Final_STRptSample). The Academic Department Assistant acts as the Taskstream platform coordinator, maintaining data and supporting the department chair, the director of licensure, and the director of student teaching & field placement. The assistant archives teaching licensure data, establishes and maintains student placement files, manages assessment data, and coordinates data for state and national reports. The Administrative Assistant manages and maintains department files for students admitted to the program which include instructor outcome ratings (EXHIBIT S2_16_DeptStructure) and archives essential information through Wittenberg’s DataTel/Colleague database system.
2c. Use of Data for Program Improvement
The department regularly and systematically uses data to evaluate the programs and clinical experiences. During the 2013-14 academic year, the department held 26 meetings (EXHIBIT S2_17_DeptMtgTopics). The percentage of meeting agendas containing items that focused on data and program improvement by topic included
1) Specialized Professional Association reporting requirements – 54%
2) Review of teacher candidates – 35%
3) EdTPA – 15%
4) Data Collection – 38%
Faculty have access to candidate assessment data through a common (firewall protected) share-point within Wittenberg’s MyWitt system. Candidate assessment data are regularly shared with candidates as evidenced by faculty use of an online open-source learning platform (Moodle). Through this program, and in varying ways, instructors make course grades and progress available to students.
The course instructor, cooperating teacher, and teacher candidate complete field experience evaluations, which are used by instructors as formative assessment of student progress. Aggregated data will also be used to help inform program improvement.
The university supervisor, cooperating teacher, and student teacher conduct mid-term student-teaching evaluations as a collaborative effort. This provides an opportunity to reach consensus on areas of strength and areas for improvement prior to the final evaluation.
Education faculty retreat semi-annually to discuss issues of importance, as well as to consider data-analysis in the interest of informing program improvement. Since the recognition of our SPA reports, the unit has implemented the edTPA project and this data has been added to the retreat agenda.
CONTINUOUS IMPROVEMENT
In recent years, based on faculty, student, and partner input, we have refined and reworked our program outcomes to be more succinct and clear. As a result of this refinement, our previous 18 program outcomes have been condensed to nine.
Realizing that technology allows improved gathering and tracking of data, a gift from a generous Witt alumna enabled us to enter into a contract in 2011 for services from Taskstream, a cloud-based software company. This transaction helps us collect and monitor candidate performance data and assist candidates with the edTPA, a standardized performance assessment.
As we made use of Taskstream's ability to support the edTPA, it came to light that students were having difficulty navigating the process, so faculty have been integrating elements of it earlier in our program coursework, to help students become more familiar with terminology, etc.
Based on feedback from faculty, students and SPAs, we are realigning our curriculum in the Early Childhood and Early Childhood/Intervention Specialist programs so that there is more flexibility in student movement through the program, the addition of four practica at the 200 and 300 level replacing field experience in courses at that level, and a full year in the clinical practice classroom during the final two semesters of study. Additionally, new courses have been created to more fully integrate differentiation, content, and models of co-teaching for these programs. These "block" courses will be cotaught by faculty. Descriptors of student learning in our program are year one - knowledge; year two - comprehension; year three - application; and year four - analysis and synthesis. Similarly the four aspects of the revised programs include:
· providing a placement exclusive to preschool for our Early Childhood program
· offering content methods coursework earlier in the program
· separating coursework and field experience through the use of four practica, each with a specific focus on teacher and student development
· adding a senior seminar to provide more faculty support for deepening candidates' understanding of teaching as a reflective practice. Candidate's will also be provided support for edTPA, requirements for licensure, Ohio's Resident Educator program, etc., as well as the opportunity to complete program evaluations.
Realigning the administrative staff (summer of 2015) will allow us to provide a full-time focus for the Coordinator of Accreditation and Program Improvement (CAPI) position. The primary purpose of this position is to establish and maintain databases on all education program outcomes related to our required program accreditation reports (CAEP) and Specialized Professional Associations (SPA) reports. Maintain the establishment of Taskstream as our designated program for the collection of signature assignment data, as well as introduce and train faculty and students in the use of this program. Develop data summaries and recommendations related to the education programs and report the findings and recommendations to the Department Chair, Director of Graduate Studies, and Chair of the Department Program Committee. Based on the data, findings, and subsequent changes (if any), compile documents and write CAEP and SPA reports for discussion, review and editing by faculty members.
Through updating and realigning the roles of department staff personnel, we hope to accomplish:
· Routine and Systematic review of department notes to validate the use of data to drive decisionmaking.
· Improved use of Taskstream in an effort to better utilize its capabilities with regard to collecting, storing and analyzing data collected.
o One example of accelerated use is recording field experience outcome ratings and using that as a measure of growth from start of student teaching to the end of student teaching
o Assurance that data are collected at the course level.
· In order to maintain a focus on continual improvement, we have outlined timetables for Program Improvement and Operations (see EXHIBITS S2_18_OperImprovCycle and S2_19_ProgrImprovCycle), as well as revising our Assessment Timetable that the CAPI will use when monitoring compliance with needed data collection (see EXHIBIT S2_20_RvAssessmentsOverview).
EXHIBITS
S2_1_OutcomesCrosswalk
S2_2_EPPprograms
S2_3_AssessmentsOverview
S2_4_StudentCycle
S2_5_UndergradGateways
S2_6_GradGatewayAssessments
S2_7_AssessReviewCycle
S2_8_EETData
S2_9_edTPAData
S2_10_FinalSTEval
S2_11_Addendums
S2_12_STEvalsByLic
S2_13_STEvals
S2_14_MetricRept2013
S2_15_FinalSTRptSample
S2_16_DeptStructure
S2_17_DeptMtgTopics
S2_18_OperImprovCycle
S2_19_ProgrImprovCycle
S2_20_RvAssessmentsOverview