OSU Academic Alert Adviser Survey
Prepared by the Academic Alert Task Force[1], March 7, 2012
The purpose of the Academic Alert Adviser Survey was to assist with evaluating the pilot of the Academic Alert System by gathering information from advisers who received alerts from instructors during the Fall 2011 term. The survey was designed to: (1) assess the level of adviser satisfaction with the system; (2) assess adviser use of the system – their follow-up with students; (3) assess adviser perceptions regarding the effectiveness of the system—whether they detected student behavior changes as a result of academic alert notices; and (4) determine whether aspects of the system should be changed for future terms. The full survey instrument is included in Appendix A.
Of the 202 Academic Advisers who received alerts from instructors during Fall 2011, 189 were still employed by OSU at the time of the survey and were invited to participate. The initial email notification was sent February 6, 2012, with a reminder notification on February 16. During the time period February 6 – 17, a total of 89 completed surveys were obtained, yielding a response rate of 47.1%.
Alert Receipt Statistics
During the period between September 7 and November 11, 2011, a total of 3525 academic alerts were submitted by instructors, involving a total of 202 Academic Advisers. More than 50% of advisers (N = 126) received fewer than 10 alerts. Just over 10% of advisers (N = 21) received more than 50 alerts. The mean number of alerts received per adviser was 17.2, and the median was 5.
College / Advisers Received Alerts / Mean Alerts Per Adviser / NotesA&S / 42 / 23.1 / 5 advisers received more than 50 alerts
CASNR / 63 / 5.7 / 11 advisers received more than 10 alerts
CEAT / 33 / 16.5 / 5 advisers received more than 50 alerts
COED / 9 / 44.0 / 2 advisers received fewer than 10 alerts
COHS / 34 / 5.5 / 4 advisers received more than 10 alerts
SSB / 13 / 39.7 / 1 adviser received 200 alerts
LASSO / 11 / 59.5 / 7 advisers received more than 50 alerts
Survey Results
Advisers were asked to report their advising role:
Advisers were asked to report their college/unit:
Advisers were asked if the Academic Alert System improved communication with students:
Advisers were asked if the Academic Alert System will improve student retention:
Advisers were asked whether the five alert categories should be retained for future terms. The vast majority of responses favored retaining all categories:
Some of the advisers (N = 5) suggested additional categories for reporting alerts. Verbatim responses are contained in Appendix B.
2 advisers would like a category to indicate if the student and instructor have already addressed the issue.
1 adviser requested a test score category.
1 adviser requested a category for online courses to indicate participation, distinguished from attendance.
1 adviser requested a category, “will not pass if work quality or quantity continues.”
Advisers were asked about their first method to make contact with a student upon receiving an Academic Alert:
Advisers were asked about their “other” first methods of contacting students in response to alerts. Verbatim responses are contained in Appendix B.
2 advisers indicated in-person contact.
2 advisers indicated that someone else was responsible for contacting their advisees.
Advisers were asked about how many of their advisees they successfully contacted regarding an alert:
Advisers were asked about how many of their alerts they responded to in SIS:
Advisers were asked about their satisfaction with the Academic Alert System:
Advisers were asked to indicate the observed level of improvement in students’ academic success after receiving Academic Alert notices.
Some advisers observed significant improvement in students’ performance following an Academic Alert. Others were not aware of students’ improvement:
Some advisers observed some improvement in students’ performance following an Academic Alert. Others were not aware of students’ improvement:
Approximately two-thirds of the advisers (N = 55) responded to Question 12: “What comments do you have regarding the Academic Alert system?” Some themes are described here. The complete set of responses is provided in Appendix B.
15 advisers made comments regarding the need for students to be held more accountable for their performance.
9 advisers made comments regarding the burden of time it takes advisers to respond to academic alerts
8 advisers made comments regarding students’ lack of response to their contacts
6 advisers made comments regarding the system increasing students who drop classes
6 advisers made comments suggesting that advisers should not be involved in the Academic Alert system
Another 4 advisers commented that their contact was redundant
5 advisers made comments indicating that some students resist acknowledging their academic problems
5 advisers made generally positive comments about the system
Approximately half of the advisers (N = 32) responded to Question 13: “What suggestions do you have to improve the Academic Alert system?” Some themes are described here. The complete set of responses is provided in Appendix B.
9 advisers made comments regarding the usability of the system (adviser entry screen, emailed excel reports, etc.)
4 advisers made comments regarding the need for accurate student contact information
4 advisers made comments regarding instructors’ misuse of the system (vague comments, positive reports, trivial issues)
Appendix A. Survey Instrument
1. My advising role can most closely be categorized as ______.
a. Academic Adviser--Staff
b. Faculty Academic Adviser—Tenured
c. Faculty Academic Adviser—Tenure Track
d. Faculty Academic Adviser—Non-Tenure Track
2. I work for ______.
a. Academic Services for Student Athletes
b. College of Agricultural Sciences and Natural Resources
c. College of Arts and Sciences
d. College of Education
e. College of Engineering, Architecture and Technology
f. College of Human Sciences
g. Honors College
h. LASSO Center
i. Spears School of Business
3. I believe the Academic Alert System has improved communication with students.
a. Strongly agree
b. Somewhat agree
c. Neither agree nor disagree
d. Somewhat disagree
e. Strongly disagree
4. I believe the Academic Alert System will affect student retention.
a. Strongly agree
b. Somewhat agree
c. Neither agree nor disagree
d. Somewhat disagree
e. Strongly disagree
5. Currently instructors may use the Academic Alert system to report concerns in the categories below. Please indicate whether you think these categories should be retained in the alert system during future terms.
a. Attendance retain delete
b. Poor Quality Work retain delete
c. Missing Work retain delete
d. Cannot Pass with Remaining Coursework retain delete
e. Instructor Comments retain delete
6. If you would like to see additional categories for reporting options, please list them here.
7. When you received an academic alert regarding a student, what was the first method of contact you most frequently used to contact the student?
a. Phone
b. Email
c. Text Message
d. Other (please describe)
8. Of all the academic alerts you received for your advisees during the Fall 2011 semester, what percentage would you estimate that you contacted regarding the alert?
a. 0%
b. 1-20%
c. 21-40%
d. 41-60%
e. 61-80%
f. 81-100%
9. Of all the academic alerts you received for your advisees during the Fall 2011 semester, what percentage would you estimate that you recorded a response in SIS (via QWS3270, also known as “black screen SIS”)?
a. 0%
b. 1-20%
c. 21-40%
d. 41-60%
e. 61-80%
f. 81-100%
10. Please indicate your level of satisfaction with the Academic Alert system.
a. Strongly satisfied
b. Somewhat satisfied
c. Neither satisfied nor dissatisfied
d. Strongly dissatisfied
e. Somewhat dissatisfied
11. In general, what level of improvement in students’ academic success did you observe in relation to their academic alert notices?
Most students / Some students / No students / UnknownSignificant improvement detected
Some improvement detected
12. What comments do you have regarding the Academic Alert system?
13. What suggestions do you have to improve the Academic Alert system?
Appendix B. Verbatim Responses to Open-Ended Questions
6. If you would like to see additional categories for reporting options, please list them here. (34 responses)
Respondent ID / Response1 / It would be nice if comments from the instructor were a mandatory accompaniment to the indicators above. The alerts can be pretty vague at times.
10 / A category indicating if an instructor has already addressed the issue with the student, or indicating what follow up they expect from the adviser.
11 / I really liked being able to discuss with students who had not done well on a first exam.
32 / test scores
48 / It will be useful if faculty can comment if the student contacted them. Some students email me back saying that they are working things out with the faculty member and I have no way of verifying this information.
52 / add online participation" to distinguish it from the mis-named "attendance" now used for online courses. students' first response when we mention attendance for an online course is "that's stupid this is an online course. right or wrong it doesn't reflect what they're doing.
55 / comments are the most useful
74 / Will not pass if work quality or quantity continues.
14. When you received an academic alert regarding a student, what was the first method of contact you most frequently used to contact the student? Other (please describe)
Respondent ID / Response
4 / Face to face
44 / in person
50 / My academic alerts were fielded by Angel Ray on the Tulsa campus
54 / one person in our office contacts the students. So, I did not.
71 / phone if available otherwise email
12. What comments do you have regarding the Academic Alert system? (97 responses)
Respondent ID / Response1 / I think in some cases, instructors sent alerts and felt their responsibilities had been met, without attempts to contact or work with the student themselves.
3 / I am a new advisor and have only received a few academic alert notifications. As a result, I have not had the opportunity to observe any major outcomes.
4 / For faculty who are teaching a full load and advising 60+ students this is pretty hard to keep up with. It is useful, but it is one more thing to do that I don't really have time to do. I think the students shoe be held more accountable, and it should not be the adviser's responsibility to track down the students.
5 / I think it's greatest benefit is alerting students to the potential need to drop a class before receieving a failing grade. I don't know that it does much for getting students to turn around their work in the class or for them to start attending, but at least many of them come in and drop before they fail.
6 / This program makes significantly more work for advisers and does not provide the student with any new information that will help make passing the class any more of a possibility. Students should know if they are not passing a class or attending a class. Student's I had interaction with still failed the course despite my contact.
9 / The most common result for my students was to drop the course. I did not see any who improved their work in the course. But that's not bad. This way they avoided getting poor grades on their transcripts.
10 / It has proved very useful for getting students to withdraw from classes that they cannot pass. Many students had already spoken with their teacher when the email was sent to them. Often my communication was redundant.
12 / Most students don't respond to my emails. I guess they either don't care...., or they are too embarassed to respond.
13 / This is a good tool for instructors to communicate with students. Some students intend to drop and forget. this is a good reminder. Some students think that they have dropped a class and this informs them otherwise. Again, I think that this is a good communcation tool between instructor and student that otherwise does not happen.
16 / My student had multiple issues over time - which we had discussed previously. I was working with a senior who had become discouraged at least partly because their skills did not match well with the major requirements.
20 / The alert system is useful in that it gives me a chance to inform a student that I am aware of academic issues that should be addressed. Some students respond immediately and provide an explanation or plan of action to address the issue. Most never respond and I do not have the time to try to contact them by phone.
21 / Every piece of information helps us in reaching out to students. It creates more work for us but provides us with valuable information we can use to reach out to students in trouble sooner.
22 / Grade of W is better than a grade of F, however, it is hard to feel enthusiastic about student dropping classes. Many academic alerts reflect situations where dropping the class is ultimately the outcome. The system is great for helping students find ways to improve if early enough in the semester or cut their lossses if later in the semester. In either event, having Academic Alert data and communications is much better than not having them, regardless of outcome.
23 / It is most beneficial when instructors provide specific comments, as a box checked with excessive absences or missing work could mean a variety of things depending on each individual’s perception. Also, some of the comments lead me to believe the instructors think we receive the info immediately, not a day (or over the weekend) delayed. I've received info saying a student must do something by that day, but I do not get the info until the next day at the earliest. By then it’s sometimes too late to provide the assistance the instructor is recommending.
24 / If there was no communication between instructor and student before, then I like that students are getting warnings, but I worry that the system is using technology to replace the simple direct contact of student with instructor. Instead of talking to a student after class about her work or lack of attendance, now instructors can handle the situation with a click of a mouse instead of an actual conversation. And I don't necessarily think having a conversation with your advisor is as good as having a conversation with the instructor of the course itself, but I feel like the system wants to get advisors involved at the expense of the first natural step, seeing one's instructor. As for the role of advisors, I also think there are more important things than student retention, such as a sense of self-responsibility. College is where you are suppose to develop that more. But have we not created a model of the hovering advisor who comes ringing you up at the first sign of trouble. I don't think these big picture concerns about student development were considered fully enough when this system was adopted. It was a rush to improve retention numbers at all costs, or so it seems to me. Take for example, attendance. Are advisors suppose to be high school truant officers now calling students up about their attendance? There should be some separation, some sense of I am now on my own and responsible for getting myself help when it comes to college life. In loco parentis, I think it is a scale and I am afraid we are in danger of going too far to one end of it with the academic alert system.