APPENDIX G: DETAILED FACULTY DISCUSSION OF RESULTS
Discussion and Plan: Claim 1
Claim 1: NMU teacher education candidates demonstrate proficiency of the Entry-Level Standards for Michigan Teachers (ELSMT)
Narrative
Upon reviewing the data collected for Claim 1, we found differences among elementary, secondary, and special education. Elementary students rated themselves more highly, secondary lower, and special education students varied from high to low ratings. There was greater consistency among the cooperating teachers and university supervisors in terms of high ratings. Cooperating and University Supervisors generally rated themselves higher than students rated themselves. We do question University Supervisor ratings in special education as they might have been done by the same University Supervisor for all students. The greatest consistency of ratings comes from the elementary table. The elementary are consistent among categories and raters with little variability in the range. The secondary and special education ratings had a higher rate of variability. We felt this meant the difference in rating between elementary and secondary might be based on relative judgments. As the developmental level of the students increases so might the relative perception of the ratings. We also felt there might be some value in comparing data sets at mid-term and final evaluation due to the eight week placements that occur. This is especially true for special education who must do an eight week placement in a content area and a special education placement. It would also be interesting to see if there is a difference in final evaluations between eight week and sixteen week placements. Based upon our data and our understandings of the data we believe the program needs to address the following ideas in the future:
· Management of students’ behavior because it is the one area consistently low in all reports
· Discussion of Teacher Self with candidates to help them identify who they are and how to best use that knowledge to manage learning
· Further explore with the cooperating teachers through NMU TEAC ways to analyze and address the data sets
· Conduct focus groups with teacher candidates on outlier areas; possibly try to do this at the final seminar or at the 5:00 Teacher Graduate reception and conduct a 4:00 focus group
· Define the purpose of UN100 experiences as it relates to student understanding of program expectations and knowledge.
· Discuss the various categories we have for students and the value of them.
Table 1.23: Claim 1 Data Sources Faculty Review Table
What? / So What? / Now What?Student Teaching Evaluation Data: Elementary / 1. B2 Management of Students’ Behavior 80% P by st tchr (93% by CT & 96% by US)
2. B3 management of learning progress 88.3% P by st. tchr / 1. Lowest of all indicators by all parties; we have been aware of this for 20 yrs. As a #1 concern; how much might be related to student teacher need to adhere to management according to cooperating teacher; Can this be taught out of a student teaching experience? How can we indirectly recognize this in our field experiences?
2. How closely are B3 and B2 related?
Student Teaching Evaluation Data: Secondary / 1. B2 and B3 at 79.9% and 79.5% respectively. Note that the CT and US percentages are closer with CT ratings are 87.7% and 88.2% respectively. US ratings probably higher due to the low number of visits (4) by US.
2. The whole category B Instructional Competence is rated below 91% in all categories (range 79.5% - 90.5%)
3. Content areas rated at 89.4% and 90.3% in Gen Knowledge.
4. D1 Professional Ethics is the ONLY category where students rank themselves higher than el ed students 5. C1 Assessments at 81% / 1. Volume of sec ed students managed for classroom and learning progress could be as high 6 times that of an elementary teacher candidate; see el ed comments above; is there an expectation that sec students are able to self-manage?
2. This category seems to relate to those questions most asked in an interview for a job? Do they really see themselves at a C+ in this category?
3. Is this different than el ed #s because the curriculum is more challenging and more likely to be challenged by the students at the sec level?
4. Question on the validity of this
5. Do we need more formal connections to use of assessments in methods or develop an assessment course?
Student Teaching Evaluation Data: Special Education / 1. Category B – Instructional Comp70.8% in B2 to 100% in B3; biggest gap in B2 data and B4 and B5 have a roughly 25 % between ST and CT.
2. All rankings done by same supervisor whereas others have multiple supervisors / 1. Nature of special needs might account for the behavior. 100% on managing learning progress might be related to IEP knowledge and involvement? Why is there a big dip then to 75% in B4 (management of contingencies) and B5 (Unit Lesson Planning)? Most variable data, has some of the lowest and highest probably due to small numbers (is all of this based on an eight week placement?)
Application to Teacher Education Data / Code 01, 02, 07 codes are all not admitted (74 total) and not in a promising position to do well academically.
10 – 50 codes tell us how many people make it and how many don’t make it. / Are UN100 students more likely to advance? Is advancement specific to a ED UN100 or a general UN 100? Should we have faculty teaching UN100s to help answer the questions for students and expose them to expectations? Are all faculty and staff clear on goal and purpose of a UN100?
Discussion and Plan:
Claim 2
Narrative
Upon reviewing data for Claim 2: “NMU teacher education candidates integrate technology into their teaching and learning;” we find ratings that are generally just below to just above the 90% mark. In many of these cases, the number of NC returns are significant, and suggest that a more accurate formulation would eliminate the NC category from statistical consideration and use only P, I, and B categories for percentile calculation.
In general, we find that students typically rate their abilities lower than the cooperating teachers rate them, who in turn rate the student abilities lower than university instructors. Further, we find commonalities of scores among elementary, secondary, and special education reports which suggest that that either instructional issues need to be addressed for our students, or students did not have the opportunity or equipment to effectively implement the strategy. The latter is supported by the number of NC responses in certain categories.
For example, for category C.2.3, (Uses technology to organize, manage, evaluate and communicate information about student performance), secondary students rank themselves as 87.4% proficient and cooperating instructors rank the student proficiency at 89.6%, and university supervisors at 84.7%. In all cases, had the NC responses not been considered all values would have been over 90%. The greatest discrepancy was with the university supervisors, 14.5% of whom supplied NC responses. In the case of this particular item, the student also has limited decision-making ability, since the methods of reporting/recording evaluations and parental contacts may be dictated by the principal or cooperating teacher. Validity of the statement is questionable, and includes many variables. It is possible to use technology to evaluate and communicate, but not to organize and manage. Do I then rank myself at 50%? This item should be broken down to better distinguish areas that use and don’t use technology, and whether the cooperating teacher uses technology for a given task.
Items B.7.4 (Helps students access and use information technology and other resources to become independent learners and problem solvers.) was low scoring in all reports, accounting for the lowest score for secondary students across the board and for elementary student self-ratings. Special education rating, while not as severe, were still of concern. Again however, high percentages of NC scores skew these values in some cases. As in the previous paragraph, the question arises: Is a low score here due to student performance or limited opportunities for in-school access? Students cannot help students access and use technology if the technology is not readily available as part of the regular in-class experience. Is the reason for the high numbers of NC responses due to the fact that the instructors don’t have the opportunity to see the student in action?
Eighty-five percent of School of Education undergraduate courses report utilization of technology in some fashion (17 of 20 courses). In two of these, the use of technology is primarily in delivery (they are “online” courses), but in all others there is some significant component that requires students to adapt, construct, or in some other fashion actually use digital technology. The School of Education has one class dedicated to technology use, ED 483, Educational Media and Technology. ED 483 is based on the National Educational Technology Standards for Teachers (NETS-T) and focuses on bringing student skills, awareness, and capabilities to national standardized levels. ED 483 is a project-based course. As such, student grades tend to be higher than traditional courses with content-area examinations.
Summary and Suggestions
Values for Claim 2 are slightly lower than optimal, with most hovering a few points below 90% proficient. The values rise when NC comments are removed from the calculation, in some cases significantly. Cooperating teacher and University supervisor ratings tend to be higher than student self-evaluations.
Cooperating teacher and University supervisor comments would indicate that, while students have knowledge about instructional technology, they do not have as high a degree of use of that technology in the classroom, and they do not as readily assist students in learning how to use the technology. Student teachers also do not use the technology to manage and evaluate students or communicate information about student performance. The numbers associated with this claim are suspect, however, due to access variances, multiple variables in the statement, and lack of independence to address the methodology used by the school or teacher for these issues.
Use of technology in NMU School of Education coursework is widespread and broadly based. Technology is addressed in the vast majority of courses as an integral component. Additionally, students take a class that is focused on the use of technology in the classroom as identified by a national organization.
The following points are identified as either questions for further review or suggestions for next directions regarding Claim 2.
· If student technological instruction focuses on meeting the NETS-T standards, do the line items of Claim 2 match the NETS-T standards? If not, one or the other needs to change.
· Students are not utilizing what they know to the desired degree. This is either due to unwillingness on their part, which would suggest a lack of understanding of the pedagogy; discomfort with the tools, which would suggest a lack of familiarity; or lack of access or availability in the school, for which we have no immediate solution.
o Student instruction needs to reinforce that certain technologies are better suited for instruction than others at certain times. Learning multiple ways of doing the same thing over the course of a student’s education provides multiple tools for students that can be injected under different circumstances.
o While familiarity can breed contempt, it also can develop worthwhile skills that can be incorporated in a pinch. Students need to use technological tools not just on special occasions, but as an integral part of their education.
· It might be advantageous to have different instructors focus on a particular technological tool (one that is appropriate to a given course and well-understood by the instructor) so that students get more hands-on experience with many different tools. This would not require a great shift in teaching, merely an identification of what goes on now and a gap analysis to see where we need to add additional focus.
Table 1.24: Claim 2 Data Sources Faculty Review Table
WHAT? / SO WHAT? / NOW WHAT?Student Teaching Evaluation Indicators: Elementary Elementary Cooperating Teacher values all =99.1% / B.7.4, C.2.3, ranked low by student; C.2.3 ranked low by cooperating teacher; B.7.4, C.2.3 ranked very low by university instructor / B.7.4 and C.2.3 are areas of greatest concern, based on the numbers, but low response rates from cooperating and university instructors are significant.
Student Teaching Evaluation Indicators: Secondary / B.5.5, B.6.4, B.7.3, B.7.4, C.2.3 ranked low by student; B.5.5, B.6.4, B.7.4, C.2.3 ranked low by cooperating teacher; B.7.4, C.2.3 ranked low by university instructor. B.5.5 and B.6.4 (identically worded) are the biggest concern for cooperating teachers. / B.7.4 is the lowest score, but low response rates from cooperating instructors are significant. C.2.3 also suffers from a large number of NC responses, particularly from Univ. Sup.
Student Teaching Evaluation Indicators: Special Education / B.5.5, B.7.4, C.2.3 ranked low by student; B.7.4 ranked low by university instructor / B.7.4 is area of greatest concern
ED 483 Grades / Mean = A / Discrepancy between this grade and student self-report in a number of areas
Samples of Faculty & Student Use of Technology / References are broad-based, varied, and use current technology
Discussion and Plan: Claim 3
Claim 3: NMU teacher education candidates graduate with a foundation in the liberal arts and subject-matter knowledge.