May 2016 Student Aportfolio Assessment Results
Readers for this year’s R_C Student Portfolio Assessment scored Aportfolios for the second year. The Aportfolio platform is still in the pilot stage although now there are many more sections to randomly select from, about thirty in all. The readers chosen by the Director of R_C were Miles Britton, Karen Ginn, and Julie Karaus. Sherry Alusow Hart conducted the meetings for developing a rubric (April 14) and training the readers in scoring holistically (May 10). The actual reading took place over the weeks between May 10 and May 27 independently and online using the Digication assessment platform.
The rubric was designed to evaluate students’ reflective practices, widely recognized across the curriculum in pedagogical literature as an essential skill for learning, self-awareness, and self-examination.
Generally the findings are not unexpected. The value added between students from R_C 1000 and R_C 2001 is evident in most categories. The categories where value was lost—Self-Assessment and Making Connections--are somewhat puzzling since the scores for Awareness (of Reflection), Shows (with evidence), and Plans (for the future) have each risen. Scores for the same artifact were often divergent by more than a point, sometimes even three points, which suggests that the norming session needed to be more robust, perhaps split into two days using more than four e-portfolios as practice. The Assessment Sub-Committee of the Rhetoric and Composition Advisory Committee will need to look more closely at this for next year’s reading preparations and especially for further faculty development.
Value Added, Maintained, or LostSelf-Assess / Awareness / Connects / Shows / Plans / Clarity / Average Total
R_C 1000 / 1.75 / 2.06 / 2.19 / 1.75 / 1.25 / 2.16 / 11.16
R_C 2001 / 1.62 / 2.29 / 2.1 / 1.81 / 1.89 / 2.19 / 11.27
While progress in helping students see the value in reflective practice can be demonstrated with this data, faculty still need to involve students regularly in reflective practices, both for learning and for self-evaluation. A reader suggested that reflection be used both as a term and identifiable practice so that students will become more overtly aware of its importance and benefits.
One reader commented on the increased opportunities for reflection which students’ made use of in their eportfolios. In the past, with paper portfolios or those turned in on AsULearn as one document, students treated reflection as an assignment, such as in the traditional reflective letter opening the collection of material. With the layout of Aportfolio, stronger students introduced each selection with a brief review of their response to the assignment, including valuable reflective comments. All readers remarked on the evidence of sustained instruction in reflective practice, especially in “[t]hose portfolios that included multiple opportunities for reflective work in different contexts and mediums . . . .”
Another reader was concerned with the uniformity of the design of the majority of e-portfolios, likely due to the lack of both instructors’ and students’ experience.
As for the interface with Digication for scoring students’ e-portfolios, all agreed that the platform was “quick, straightforward, and pretty easy to use.” Being able to see each rubric element and the portfolio at the same time allowed greater accuracy as well.The readers suggested gathering the artifacts later in the semester to allow a greater variety of sections from which the e-portfolios were drawn. Once the pilot is completed and all instructors in R_C sections required to use Aportfolio, this issue should disappear.They suggested that Aportfolio also be introduced earlier in the semester to “have it well integrated into the course so it does not become a repository.”
The readers have made plans to use this year’s rubric to give their students a chance to see how reflection may be evaluated, including with lower stakes writing. As instructors, they hope to learn more about the platform in creative ways to encourage students to take control of their e-portfolios and each develop a “voice” that expresses his or her uniqueness. We will recommend further faculty training for these purposes.
Sherry Alusow Hart
Chair, Assessment Sub-Committee, R_C Advisory Committee
Director of Assessment for WAC
June 30, 2016
Attachments include the rubric and the complete scoring data.