Carnegie Course Redesign Final Report, Cohort 1

College Algebra (Math 109)

University of MarylandEastern Shore

This final report describes the outcomes of the University of Maryland Eastern Shore’s (UMES) course redesign of College Algebra (Math 109). The redesign was piloted during the academic year 2011 - 2012.

PILOT FALL 2011

Two of twenty-three sections of Math 109 were deemed as pilot sections inFall 2011. Positive trends reported from both Pilot sections, such as significantly higher student evaluations, marked increases in student engagement, enhanced student confidence, and considerably higher retention rates(see table below).

Pilot Fall 2011 Indicator / Traditional Sections / Pilot Sections / Overall Math Dept.
Mean Pretest Score (Given start of Week 2; based on prerequisites from Math 101 and required Accuplacer® placement score) / 55.5% / 22.5% / --
Mean Student GPA on 4.0 scale (upon enrollment in Math 109) / 2.67 / 2.26 / --
Overall Retention Rate (number who did not withdraw/ number originally enrolled) / 78.5% / 94.5% / 68%
Overall Pass Rate () / 27% / 25.5% / 33%
Mean # hours per week students spent studying Math 109 / 3.1 / 3.4 / --
Mean # hours per week students spent going to tutors for extra help in Math 109 / 1.3 / 1.7 / --

PILOT 2 SPRING 2012

During the Spring 2012 semester, four redesign sections (17 sections offered in total) were offered. The Department is especially limited in the numbers of sections offered due to computer laboratory currently used, that is one of the central components of the Supplemental Model, has a total of 800 square feet. Additionally, the Department wished to give the project another opportunity to address the overall successful completion rate (currently at 25.5%) before exposing three (3) times more students to the project activities.

PILOT SECTIONS’ ASSESSMENTS

In both the Pilot and Pilot 2 semesters, instructors involved in the project taughtsimultaneous Math 109 classes consisting of one traditional section and one redesign section, eliminating the variable of instructor quality. A total of five instructors were involved (three tenure/tenure-track and two full-time lecturers). Data was gathered from a variety of sources, including the following:

  • Instructor’s grade reports
  • Department-Wide final exam
  • Pretest of prerequisite skills given at start of each semester
  • Instructor’s Chapter Exams and Group Projects
  • Student records (GPAs, number of times attempted Math 109 previously and number of times needed to pass after taking regular or redesign Math 109)
  • Written logs of attendance at the Assisted Learning Lab and Study Sessions
  • Student Surveys (Likert-scale, given at midterm and Final Exam)

For the purposes of this report, data from the Pilot and Pilot 2 werecombined and analyzed using mean, ANOVA, and other common statistical measures of significance. Thus we are able to make some valid assumptions regarding a comparison of regular and redesign sections of Math 109.

A. Impact on Student Learning

  1. Improved Learning

We cannot say for certain if the Math 109 Redesign would improve student learning indicators such as the pass rate, critical thinking skills, engagement, and self-confidence of all students since less than 15% of studentsparticipated in the redesign. Analysis of the Pilot and Pilot 2yielded the following conclusions.

  • During the first ten days of classes, students took a pretest of prerequisite skills. This pretest contained questions similar to a Math 101 final exam (Math 101 is the developmental course which 90% of UMES students must take due to low Accuplacer® scores- and then pass with at least a “C” prior to enrollment in Math 109.) Overall, scores showed that students were not sufficiently prepared to enroll in Math 109 after passing Math 101. Presumably all students should have attainedscores of 70% or more on the pretest because they passed Math 101 with a grade of 70% or more and it was these same skills that were required on the pretest. The lack of prerequisite skills was causing subsequent failure in Math 109.
  • The students enrolled in the 5 day/week Redesign Sections had significantly lower skill levels upon entering Math 109 than their peers in Traditional Sections (based on pretestscores and GPAs). Apparently weaker students recognized their need for extra support and self-selected the Redesign. However, in spite of initially large gaps in skill level, the Redesign’s Supplemental Model appeared to remediate deficiencies such that both kinds of sections had comparable pass rates in the end.
  • STEM majors exhibited comparable weakness to their peers in non-STEM majors (based on pretests, GPAs, and number of times attempted Math 109 previously). They enrolled in comparable percentages in both Traditional and Redesign. However, the Redesign appeared to assist lower ability STEM majors to succeed at a rate comparable to their more skilled counterparts in the Traditional sections.
  • Compared to non-STEM majors, STEM majors were slightly more successful in Math 109 regardless of whether they were enrolled in Traditional or Redesign. Nevertheless, their pass rate was less than 10% higher than the overall pass rate of all majors and was still appalling low.
  • The Redesign’s ability to close the gap between Traditional and Redesign students’ initial skill levels appeared to be due to students’ increased engagement, which in turn increased student learning. All measures reflected higher levels of student engagement with course materials, resources, and Redesign personnel.
  • Students who participated in the Redesign sections but still failed (grades of D or F) were able to pass Math 109 in fewersubsequent attempts than their peers who had been enrolled in Traditional sections. It appeared that the redesign remediated sufficient deficiencies and improved student learning to the degree that it enabled more students to be successful in a shorter period of time. Transcripts of students in the Fall 2011 and Spring 2012 Pilot and Regular sections who will enroll in upper level math courses during the next year are currently being analyzed to see if further course taking success is affected.

Indicator of Student Learning / Traditional Sections / Redesign Sections / Conclusion
Mean Pretest Score (Given start of Week 2; based on prerequisites from Math 101 and required Accuplacer® placement score) / 55.5% / 22.5% / Significant difference: Students enrolled in Redesign have fewer prerequisite skills than peers in Regular
Mean Student GPA on 4.0 scale (upon enrollment in Math 109) / 2.67 / 2.16 / Significant difference: Students enrolled in Redesign have lower GPA than peers in Regular
% of students who attempted Math 109 before the current enrollment / 58% / 59% / No significant difference: Students have same rate of previous failure
% of students who failed Math 109 and required more than one more subsequent semester topass / 38% / 23% / Significant difference: Redesign students have higher pass rate in subsequent retakes of course
Mean # hours per week students spent studying Math 109 / 3.3 h / 7.9 h / Significant difference: Redesign students spend more hours studying than peers in Regular
Mean # hours per week students spent going to tutors for extra help in Math 109 / 1.4 h / 3.1 h / Significant difference: Redesign studentsmotivated to get more help from tutors than peers in Regular
Mean % of questions attempted on Department-Wide Final Exam (problems not left blank) / 62% / 91% / Significant difference: Redesign studentshave confidence to attempt more questions on Final Exam at end of course than peers in Regular
% STEM majors enrolled (remaining non-STEM) / 53.1% / 51.8% / No significant difference: Both Sections have same proportion of enrolled STEM/non-STEM majors
Overall Pass Rate () / 30.4% / 34.3% / Minor difference: Both Sections have close to same pass rate
STEM-majorPass rate () / 39.8% / 41% / (1) No significant difference between sections for STEM-major pass rate; (2) Significant difference between STEM & Overall pass rate
  1. Improved Retention

Indicator / Traditional Sections / Redesign Sections / Conclusion
Overall Retention Rate (number who did not withdraw/ number originally enrolled) / 71.5% / 89.8% / Significant difference: Students in Redesign persevere to end of course more often than in Regular
STEM Major Retention (number who did not withdraw/ number originally enrolled) / 84.6% / 89.7% / Slight difference: Redesign has slightly higher proportion of retained STEM majors

Retention rate is often influenced by a student’s major discipline. Characteristically, students in non-STEM majors tend to haveacademic difficulty in Math 109 and admit defeat more frequently and easily. As a resultthey comprise a greater relative proportion of withdrawals from the course.STEM majors, on the other hand, tend to have higher achievement in mathematics. They typically persevere until the end of the semester because Math 109 isthe foundation course for subsequent higher level mathematics classes they are also required to take. Data yielded these findings:

As the data above illustrates, the redesign improved retention rates when compared to the traditional course for all students, regardless of major. Even a separate analysis of the retention of STEM majors shows that the redesign had a positive impact on our most capable students.

  1. Other Impacts on Students

Students were given the opportunity to give their opinions about Math 109, the redesign efforts, and other relevant information in two anonymous paper-and-pencil surveys using a 5-point Likert-scale (given at midterm and the Final Exam).Both the student surveys indicated greater satisfaction with the Redesign sections of the course over the traditional sections taught by the same instructor. Thus, any satisfaction could be attributed to the redesign curriculum and strategies rather than personal preference for an instructor’s style of teaching, personality, or other individual characteristics. Written comments tended to focus on the following topics:

  • Satisfaction with services provided in the Assisted Learning Lab (ALL). Studentsreported feeling supported, empowered, and safe to learn in the lab environment. As one student said, “it’s OK to make mistakes in the ALL. No one makes fun of you ‘cause we’re all learning down there.”Many others commented on the professionalism and patience of our redesign tutors, or Undergraduate Learning Assistants (ULAs).(It should be noted that Redesign ULAs went through intensive paid training sessions at the start of each semester to help them hone their tutoring skills and teach them to tutor in ways consistent with Teaching Standards put forth by the National Council of Teachers of Mathematics.)
  • Students generally preferred the use of the redesign PowerPoint slides for lectures rather than regular instructors’ practice of writing on the chalkboard. They cited a major advantage being the fact that these lectures could be accessed after class whenever desired via the WebAssign® course site.
  • Almost unanimously, students stated a preference for having peer learning groups over professors’ lectures. Some even suggested omitting professor’s lectures altogether and allowing them to learn in groups full time!
  • Some students cited the use of the Redesign’s Math Notebook (purchased at the bookstore for under $10) as critical to organizing their assignments and making it possible for them to study more effectively and pass the course.
  • Students felt the example videos that redesign instructors imbedded in their eBooks were beneficial. With each homework assignment, students were required to watch two videos and take notes on them in a designated section of their Math Notebook. Viewing other instructor’s solutions to problems gave them an expanded perspective and suggested alternative solution strategies.
  • Written comments showed an increased interest in mathematics and appreciation for the use of math as a tool in the real-world gained from doing the application projects. Many commented that these assignments were the first time they ever understood the value or meaning of mathematical formulas that they were always required to learn in math. Some felt they had learned more mathematics in one semester than they had learned in their entire lives up to that point. Over 80% felt the application projects were a better evaluation of what they knew and understood in Math 109 than what could be determined by any traditional assessment like a test. Negative comments focused on the immaturity or lack of commitment of other members of their group, but not on the assignment itself.
  • Students also consistently stated that tutors in Assisted Learning Lab were more capable than those provided in “Access & Success” – the general tutoring lab provided by UMES. The higher quality of tutoring was most likely the result of (1) the training ULAs receive, and (2) theULAsworkingconsistently with students during the regular Tuesday/Thursday class times. Thus, ULAs were familiar with current topics and course requirements, whereas generally qualified tutors work from their own personal knowledge base irrespective of current curriculum topics.

Data collected from the surveys yielded these results:

Indicator / Traditional Sections / Redesign Sections / Conclusion
Mean # hours per week students spent going to tutors for extra help in Math 109 / 1.4 h / 3.1 h / Significant difference: Redesign students motivated to get more help from ULAsthan peers in Regular sections using regular tutors

B. Impact on Cost Savings

Cost reduction should have been realized by increasing the section size from 25 to 45 students, which did occur in Pilot and Pilot 2. The fewer number of attempts that students in the Redesign needed to retake the course to pass should have also lessened faculty load and in turn realized a cost savings.

C. Lessons Learned

  1. Pedagogical Improvement Techniques

The following pedagogical improvements were made in the Math 109 Redesign:

  • Implementation of theALL. The ALL wasa critical component of NCAT’s Supplemental Model and was put into operation as support forPrinciple #1: Redesign the whole course. The ALL focusedon strategies proven effective with minority students, such as collaborative group workand a supported lab experience using specially trained full-time LA and part-time ULAs (tutors) to foster self-confidence. Also, the ALL was instrumental in realizing Principle #2: Encourage active learning because it utilized regular peer learning groups (4-5 students) and required presentations of problems on the board by peer groups (5 times, graded by ALL staff). The ALL also assured that learning quality improved as outlined in Principle #3: Provide students with individualized assistance.We actively taught students to take full advantage of online resources (WebAssign®) and staffed the ALL with trained personnel. Our staff offered students individualized intervention as needed and served as a liaison between faculty and students, provided insight into student achievement, supervised online quizzes, andgraded Math Notebooks and problems done on the board.
  • The use of theinstructional strategies like PowerPoint lecture slides, a Math Notebook, and videos imbedded in the course eBook. These pedagogical improvements supported NCAT’s Principle #2: Encourage active learningby teaching students to be responsible for their own learning and then empowering them to be so. Using a Math Notebook for taking notes, watching videos of example problems, and showing all work on homework problems encouraged students to organize and understand their assignments, as well as helping them to develop good study habits for Math 109 orany other course.
  • Full implementation of the WebAssign® online site accompanying our textbook. This pedagogical technique was an improvement that supported Principle #4: Build in ongoing assessment and prompt (automatic) feedback. Students were assigned frequent homework problems, weekly quizzes, and remedial modules and quizzes through WebAssign®. The website provided an instant feedback mechanism. Automated feedback also quickly identified students who scored lower than 75 so these individuals were promptly assisted by the LA and ULAs. (Scores of 75% or higher were required to move on to the next task.) Math notebooks and student peer groups presenting solutions to problems on the board also provided both ongoing assessment and prompt feedback.
  1. Cost Reduction Techniques

Cost reduction should have been realized by increasing the section size from 25 to 45 students. The fewer number of attempts that students in the Redesign needed to retake the course to pass should have also have been a cost reduction strategy.

  1. Implementation Issues:

The implementation issues that we consider to have had the greatest impact on the UMES Math 109 Course Redesign:

  • Administrative and Faculty Support. Since the submission of the proposal to redesign MATH 109, UMES administration has experienced significant turnover.In our case, once the redesign grant had been accepted, some faculty in the Math Department came forward to state they felt that they hadn’t had the opportunity to fully express their opinions. In particular, there were concerns regarding the classroom space needed to provide an ALL sufficiently large to accommodate every student enrolled in Math 109 (not just those in Redesign sections) and the financial expense needed to staff the ALL with qualified LAs and ULAs. Negative feedback often centered on potential resources or the lack thereof. There were also valid concerns with the lack of improvement in pass rate.
  • Financial Considerations. In addition to the above issues, any university needs to answer somelooming monetary questions. Although UMES initially matched funds provided by the USM Carnegie grant (in fact, they exceeded their commitment by setting up a new computer lab in the Math Dept.), when outside funding is dissipated the university and department needs to carefully examine their long-term responsibility.

E. Sustainability

The Supplemental Model is used as a basis for our continuing MATH Redesign Efforts, especially in terms of resources, and learning strategies like 5-day per week classes, computer assisted learning, the Math Notebook, and the Assisted Learning Lab. The Mathematics and Computer Science Department Chairperson is currently directing redesign efforts that involve teams composed of the majorityoffull-time faculty. Two workshops were conducted during the summer of 2012 involving many full-time and adjunct faculty to address the concerns of the three mathematics General Education courses offered at UMES (Math 101, Math 102, and Math 109).These efforts are responsive to the lessons learned in the Redesign Pilots. Ideas from the summer workshops have been combined with new faculty innovations and some of the universally accepted strategies used in the Math 109 Redesign. The UMES Math Department continues to redesign Math 109 in a more inclusive, cooperative manner.