Burke Co. Schools Project RISEUp Year-end Evaluation
George H Olson, Ph. D., Evaluator
19 August 2010
The tables on the following pages, summarize the attendance results for the Burke County Schools’ Project RISEUP dropout prevention program for the school year 2009-2010. There were two primary intended outcomes for this program, stated as SMART Outcomes:
SMART Outcome No. 1: By the end of the 2009-2010 school year, 90% of the 6th and 9th grade students who have poor attendance (15 days or more absent) during this school year (2009-2010) will have evidenced improved attendance.
SMART Outcome No. 2: By the end of the 2009-2010 school year, 75% of the 6th and 9th grade students who receive tutoring during this school year (2009-2010) will be promoted to the next school year.
Two interventions were implemented to achieve these objectives. For SMART Outcome No. 1, Attendance Intervention Team (AIT) members counseled students with poor attendance. For SMART Outcome No. 2, after-school tutoring sessions were established to provide additional instruction in Reading, Mathematics, Language Arts, and Social Studies in the county’s middle schools and high schools.
There is an inherent inconsistency in the statement for SMART Outcome No. 1. It states that students with poor attendance in 2009-2010 will evidence improved attendance in 2009-2010. The problem with this statement is that the criterion for poor attendance is defined as 15 days or more absent. It is reasonable to expect that students would be more likely to reach this criterion later in the school year, where, according to the intended outcome, they should be showing improved attendance. Unfortunately, this inconsistency was not noticed when the SMART Outcomes were established.
One proposal for dealing with this inconsistency was to examine student absenteeism for the fall semester of the 2009-2010 school year and identify as meeting the criteria any student who was absent more than seven days during that semester. Unfortunately, due to procedures the district’s Offices of Technology uses to collect attendance data does not permit this. Attendance data are dynamic; they change daily. Furthermore, there is no attempt during a school year to reconcile students’ statistical data for those students who change schools during the year (these students have multiple records). It is not until well after the close of the school year, when the attendance data are reconciled by DPI, that stable attendance statistics become available.
Additionally, the AITs did not record absenteeism data. However, even had they done so, the problem of higher likelihoods of poorer attendance as the year progressed would still exist.
While there is no perfect solution, SMART Outcome No. 1 can be assessed in two ways. First, it can be modified to state that students with poor attendance in 2008-2009 (15 days or more absent) will show improved attendance in 2009-2010. Second, since the AIT teams were established, in the first place, to deal with students with poor attendance issues, it may be reasonable to assume that these students at least met the spirit of the poor attendance criterion.
SMART Outcome No. 2 poses no such problem. The intended outcome states, simply, that 75% of the students participating in tutoring would be promoted at the end of the 2009-2010 school year. The district’s reconciled student databases provide the necessary data.
Summary statistics pertaining to the two SMART Outcomes (and more) are presented in the tables given on the following pages. Since SMART Outcome No. 2 is the most straight-forward, it is considered first.
SMART Outcome No. 2. Tables 1 and 2 provide the summary statistics pertinent to the second SMART Outcome. The final column of Table 1 shows clearly that SMART Outcome No. 1 was achieved. At Grade 6, 98.7% of the students participating in tutoring were promoted; at Grade 9, 88.1% of the students were promoted. Over both grade levels, 92.4% of the 395 students participating in the program were promoted at the end of the 2009-2010 school year.
While SMART Outcome No. 1 was clearly met, it is obvious that participating in the program was not associated with improved attendance for all students. Over both Grades 6 and 7, only 43.6% evidenced improved rates of attendance, while 40% showed a decline in attendance. Approximately 16% of the students had no change in their attendance rates.
But Table 1 includes 34 students who were also served by the district’s Attendance Intervention Teams. In Table 2 these students have been excluded. This resulted in relatively minor differences between the two tables. Among the unduplicated set, 92.5% of both grade levels combined were promoted at the end of 2009-2010. Corresponding statistics for Grade 6 and 7, respectively, were 98.6% and 88.7%.
Tables 1 and 2 also give summary statistics for those students who could be identified as havening met the poor attendance criterion by the end of the 2008-2009 school year. There it is shown that 87.9% of the duplicated counts, and 89.7% of the unduplicated counts, were promoted.
Another interesting observation, in both tables, is that relatively few of the students participating in the tutoring program, who could be identified as matriculating in Burke Co. Schools the previous year, had poor attendance in 2008-2009. Virtually all of those students were promoted.
SMART Outcome No. 1. Turning, now to the more problematical second SMART Outcome, the relevant summary statistics can be found in Tables 3 and 4. As stated earlier, the inconsistency in the wording of the intended outcome is such that there is no reasonable way to determine who had accumulated poor attendance during the year while at the same time improving their attendance during the year. Our attempt to identify students with poor attendance during the first semester of 2009-2010 was unsuccessful for reasons previously discussed.
As suggested above, one way of possibly examining the effects of AIT counseling is to compare students’ attendance the previous year with that of 2009-2010. This is done in the left-hand side of Table 3. There it can be seen that 52 (27 in 6th Grade and 25 in 9th Grade) were absent 15 days or more in 2008-2009. Of those students, slightly less than one-half were absent 15 days or more in 2009-2010. Hence, by this criterion, only 67.3% met the goal specified in SMART Outcome No. 1.
Students, however, were not identified for counseling on the basis of their previous year’s attendance record. Instead, during 2009-2010, students were referred to the Attendance Intervention Teams because they were perceived to have serious attendance issues. Hence, the right side of Table 3 may be more useful in gauging the results of the AIT interventions.
By the end of 2009-2010, 72.2% of the AIT students had less than 15 days absent. Within grade levels, the percent of students with fewer than 15 days absent were 92.5% and 42.4% for Grades 6 and 9, respectively.
Although it was part of SMART Outcome No. 1, the percent of students in the AIT group who were promoted was impressive. Overall, 92.4% were promoted (98.7% at Grade 6, and 88.1% at Grade 9)/
As to whether or not we can state that SMART Outcome No. 1 was attained it would seem that the bulk of the evidence suggests that it was not. However, there is evidence that the Attendance Intervention Teams were reasonably successful in helping students improve their attendance—at least at Grade 6.
There was an expressed interest in the number and types of contacts made by the Attendance Intervention Teams. A summary is provided in Table 4. To compute the results shown in Table 4 it was necessary to scan through the written comments given by the AIT members. To accomplish this keyword search was conducted. Although a careful attempt was made to identify as many critical keywords as possible, it is also possible that some keywords were overlooked. Even so, the data provided in Table 4 should be considered a reasonably close estimate of the types and number of contacts made.
Demographic Data. The Demographic data requested by EdStar are provided in Tables 5, 6, and 7. These data are self-explanatory.
TABLE1: Absentee Statistics for Students Served by Tutoring in 2009-1015 Days or More Absences in 2008-09 / All Students Served
Total Served / Number / Less Than 15 Absences in 2009-10 / More Than 14 Absences in 2009-10 / Promoted / Absences in 2008-09 Greater than 2009-10 / Absences in 2008-09 Less than 2009-10 / Promoted / Less Than 15 Days Absent in 2009-2010
Grade 6 / 159 / 8 / 8 / 4 / 7 / 66 / 58 / 157 / 147
Grade 9 / 236 / 25 / 21 / 8 / 22 / 106 / 100 / 208 / 100
Total / 395 / 33 / 29 / 12 / 29 / 172 / 158 / 365 / 247
Note: The counts in this table include students also served by Attendance Intervention Teams
TABLE 2: Absentee Statistics for Students Served by Tutoring in 2009-10 (Unduplicated with AIT data)
15 Days or More Absences in 2008-09 / All Students Served
Total Served / Number / Less Than 15 Absences in 2009-10 / More Than 14 Absences in 2009-10 / Promoted / Absences in 2008-09 Greater than 2009-10 / Absences in 2008-09 Less than 2009-10 / Promoted / Less Than 15 Days Absent in 2009-2010
Grade 6 / 139 / 8 / 8 / 4 / 7 / 65 / 57 / 137 / 129
Grade 9 / 222 / 21 / 19 / 5 / 19 / 98 / 94 / 197 / 196
Total / 361 / 29 / 27 / 9 / 26 / 163 / 153 / 334 / 325
Note: The counts in this table excludestudents also served by Attendance Intervention Teams
TABLE 3: Absentee Statistics for Students Served by Attendance Intervention Teams in 2009-10
15 Days or More Absences in 2008-09 / All Students Served
Total Served / Number / Less Than 15 Absences in 2009-10 / More Than 14 Absences in 2009-10 / Promoted / Worse Attendance in 2009-2010 / Improved Attendance in 2009-2010 / Promoted / Less Than 15 Days Absent in 2009-2010
Grade 6 / 189 / 27 / 18 / 14 / 24 / 118 / 55 / 172 / 147
Grade 9 / 102 / 25 / 10 / 15 / 11 / 7 / 33 / 57 / 63
Total / 291 / 52 / 23 / 29 / 35 / 125 / 88 / 229 / 210
TABLE 4: Summary of Contacts by Attendance Intervention Teams
Tel. Calls / Review Letters / Email Messages / Face to Face Meetings / Home Visits
Grade 6 / 8 / 74 / 1 / 35 / 11
Grade 9 / 31 / 17 / 3 / 46 / 8
Total / 39 / 91 / 4 / 81 / 19
TABLE 5: Demographic Data for Students Participating in the Tutoring Program
Race/Gender / American IndianMale / American Indian Female / AsianMale / Asian Female / Black Male / Black Female / Hispanic Male / Hispanic Female / Multi-Racial Male / Multi-RacialFemale / White Male / White Female6th Grade / 0 / 0 / 0 / 4 / 7 / 6 / 5 / 6 / 0 / 0 / 80 / 61
9th Grade / 0 / 0 / 4 / 1 / 10 / 6 / 3 / 4 / 0 / 0 / 36 / 35
Note 1: These data include students who may also have participated in the Tutoring Program.
Note 2: The total number of students given in this table is five less than the number given in Table x. These five students could not be located on the district’s database. All the missing students were 6th Graders.
TABLE 6: Demographic Data for Students Participating in the Tutoring Program (Unduplicated Counts)
Race/ Gender / American Indian Male / American Indian Female / Asian Male / Asian Female / Black Male / Black Female / Hispanic Male / Hispanic Female / Multi-Racial Male / Multi-Racial Female / White Male / White Female6th Grade / 0 / 0 / 6 / 3 / 2 / 3 / 5 / 1 / 6 / 5 / 59 / 54
9th Grade / 0 / 0 / 0 / 5 / 12 / 4 / 0 / 4 / 3 / 1 / 92 / 101
Note 1: In this table, students who may also have participated in the Tutoring Program have been excluded.
Note 2: The total number of students given in this table is five less than the number given in Table x. These five students are the same five students from Table 5 who could not be located on the district’s database. All were 6th Graders.
TABLE 7: Demographic Data for Students Counseled by Attendance Intervention Teams
Race/ Gender / American Indian Male / American Indian Female / Asian Male / Asian Female / Black Male / Black Female / Hispanic Male / Hispanic Female / Multi-Racial Male / Multi-Racial Female / White Male / White Female6th Grade / 0 / 0 / 2 / 4 / 7 / 6 / 5 / 5 / 0 / 0 / 80 / 61
9th Grade / 0 / 0 / 4 / 1 / 10 / 6 / 3 / 4 / 0 / 0 / 36 / 35