to:Jenny Keil
from:Leondra Hanson
subject:AssessmentDirector Progress report fy12 and FY13
date:September 16, 2013
Brief History: The Committee on Learning Outcomes Assessment was formed in 2007-08 in response to the Hamline University Strategic Planning process and the concerns of the 2007 Higher Learning Commission Site Visit Team. The teammandated that assessment become a priority at Hamline, and it recommended Hamline’s participation in the HLC Assessment Academy. From 2007-2012, Hamline participated in the Academy with the ambitious goal of building and systemizing assessment of learning outcomes at the program level. The work began at the ground level and required the development of administrative structures to support Learning Outcomes Assessment, frameworks to achieve it, and most importantly significant development and education of faculty and staff about the work. Beginning in September 2011, the Assessment Director reported to the newly created Associate Provost position and shared the CLOA committee chair position with the IR Director-Assessment.
Position Description: The Assessment Director leads the work of the Committee on Learning Outcomes Assessment to determine and set University goals and priorities for Assessment and to facilitate faculty development. In addition to working with the committee, the Assessment Director acts as the primary point of contact, internally and externally, for all issues related the framework, structure, and success of Learning Outcomes Assessment activities of Hamline University.
Specific Goals 2011-2013: In September, 2011, based on feedback from the outgoing assessment director, the HLC Academy mentor, and the incoming Associate Provost, the primary goals for the incoming Assessment Director and the Committee on Learning Outcomes Assessmentfor FY12 and FY13 were:
1)Successfully complete the HLC Assessment Academy;
2)Continue long-term efforts to bring Blackboard Outcomes to campus as technology to assist faculty and staff in data collection and distribution;
3)Improve the participation rate in assessment work and the “culture” of assessment among faculty and staff and capitalize on newly created administrative structures with the Provost’s office and CTL to systemize training and development around assessment issues;
4)Continue leading and systemizing the annual review and feedback efforts for program level learning outcomes assessment and report annual progress to Department Chairs, Deans, and the Provost.
5)Strategize and build frameworks to begin Phase II of Hamline University Outcomes Assessment – General Education Assessment;
Report on 2011-2013 Goals:
1)Successfully completed the HLC Assessment Academy in November, 2012.
In FY 12 the Assessment Academy required online reports to the Mentor reporting Hamline’s progress around a number of specific questions, and dialogue with the Academy mentor as well as preparation and registration to complete the Assessment Academy in Fall 2012 at the end of the term. With input from the Academy Team that included the Assessment Director, Associate Provost, Former Assessment Director, and IR Director/Assessment the final Impact Report was submitted, reviewed, and approved by the Assessment Academy Mentor. Because of the extensive efforts of CLOA throughout Hamline’s several years in the Academy, Hamline University was asked to present its work to other institutions at the Academy’s Final Roundtable. Accordingly, we prepared and presented Hamline’s work at the Final Roundtable: Framework and Flexibility: Building a Culture of Assessment at Hamline University
Hamline University successfully completed the HLC Academy final roundtable and the response to Hamline’s Impact Report from the Academy mentor was overwhelmingly positive:
While not every assessment need identified in the Academy Application has been fully addressed, and not every initial goal met, assessment of student learning has been established as a pervasive, significant, systematic, and ongoing initiative at the university. Nearly every unit has developed meaningful learning outcomes and is engaged in the process of assessing student achievement of those outcomes, and the institution has identified concrete and necessary steps to extend assessment efforts beyond individual programs to general education. –Impact Report Reviewer, emphasis added
Next Steps: The HLC Academy was a successful and positive experience for Hamline University. The project helped HU put in place sustainable frameworks for assessment activity and build a long-term approach that is true to the teaching and learning culture at Hamline. Between now and the next accreditation visit in 2017, it will be important for HU to deepen the assessment efforts at the program level – to see the good work started in most departments extrapolated across program learning outcomes and systemized into routine collection and review of information. While frameworks and culture were an imperative piece of part 1, the challenge over the next years is to see and use the data from this work in a visible and routine way.
2)Completed purchase of Blackboard Outcomes and began implementation efforts.
Negotiations and strategy to get Blackboard Outcomes to campus consumed most of FY12. Hamline University successfully completed its work on bringing the technology to campus in Summer 2012. CLOA and the Director of Assessment worked in connection with Blackboard Outcomes to bring technical and educational consultants to campus for multiple days in Fall 2012 to train faculty and staff about how to use the software and to begin pilot projects.
Following the purchase of Blackboard Outcomes, a Blackboard Outcomes Implementation Team was created including the Director of Assessment and the IR Director-Assessment, along with representatives from ITS and the CTL. The work of that committee included continuing the collaboration with Blackboard to create the underlying structures necessary to deploy Blackboard Outcomes on campus, and to develop training for faculty and staff that would allow faculty to use Outcomes to collect program level learning outcomes data.
Next Steps: Technology for rubric based data collection and distribution is imperative to continuing the efforts with respect to Number 1, above. Blackboard Outcomes will need to continue to be used and explained to faculty in order for it to be an effective tool for full cycle assessment needs at Hamline. Challenges include delegating responsibility for creating program level assessment projects within the software in a way that ensures success.
3)Routine and thorough assessment training has been embedded into faculty development sessions, and Assessment participation and enthusiasm on campus has increased as a result.
With the connection with CTL that occurred in FY12, training and development with respect to assessment issues became a routine part of Faculty Development Days, which occur three times annually. In Ay 11-12 and 12-13, the following training and development sessions were offered by CLOA in conjunction with the CTL.
University Wide Development:
January 2012, Faculty Development Day:
Collaborating in Groups and Using Clickers in the Classroom
May 2012, Faculty Development Day:
How to Create Meaningful Learning Experiences (Using Assessment Frameworks) – Leondra Hanson
Designing Writing Assignments for Writing Intensive Courses– Jules Thompson
So you are Teaching a Class with an ‘O’… How does that fit with Learning Outcomes-Patricia Palmerton
Writing Across the Curriculum: Faculty Responding to and Assessing Student Writing - Alice Moorhead
August 2012, Faculty Development Day:
Teaching, Learning, and Accountability – Leondra Hanson
Your Syllabus Through Your Students Eyes – Melissa Embser-Herbert
Syllabus Workshop
January 2013, Faculty Development Day
Assessment Overview and Models and Planning Templates – Stacie Bosley
Deans Report on Assessment – Deans Matachek, McCarthy, and Sorenson
Breakouts: Refreshing the Basics of Assessment (Amy Sheehan/Suda Ishida); Rubric Maximization (Kristina Deffenbacher/Caroline Hilk); Closing the Loop (Leondra Hanson and Tracy Williams)
In addition to University Wide assessment training sessions, sessions were presented by the Assessment Director annually at New Faculty Orientation to explain the Hamline approach to Learning Outcomes Assessment. And at least once per academic year, the Assessment Director was invited to and presented on assessment strategies and reporting during at least one, and sometimes multiple,full faculty meetings of the College of Liberal Arts, the School of Education, and the School of Business. The Assessment Director also worked with and presented to the Teaching and Learning Committee at the School of Law, specifically on whether to review and revise Learning Outcomes for the School of Law.
Assessment training at Hamline also took the form of individual meetings with Departments with respect to specific guidance and advice on learning outcomes, mapping, rubric creation, and building assessments. For example, one on one advising occurred with the Political Science, Music, Studio Arts, andEast Asian Studies programs, among others, and with small cohorts from Departments to work on specific projects. In addition to the one on one training by the Assessment Director, in FY 12 CLOA piloted relationships with Learning Assessment Fellows, consultants from within the faculty and staff paid small stipends to share their own assessment work and provide individual assistance and expertise with assessment issues.
Next Steps: With the faculty development calendar and the work of CTL being expanded to included issues like assessment and pedagogy, the training for assessment has really grown and become a natural and ingrained piece of annual discussion and development. While many resources are available on the CLOA website, and still more are available from the CTL, one project that began in FY13 was to streamline and provide a constant, consistent, and easily locatable home for resources and templates related to program assessment. CLOA had some discussion about creating some online or fixed training sessions – particularly for new faculty – in the form of online modules or other easy to access presentations of learning outcomes assessment issues. Another identified need with respect to development and training is the incorporation of adjunct faculty into the assessment culture. Many Departments have expressed interest in ways to engage, incorporate, and train numerous part-time facultyin the methods and approach to assessment.
4) Annual review and feedback efforts for program level learning outcomes assessment and report annual progress has continued and expanded.
CLOA has continued to create annual report forms for faculty and staff to report assessment work at the Department/program level. In FY 12 and 13 CLOA created and communicated annual assessment goals, report forms for those goals, and feedback forms. The Assessment Director and individual members of CLOA worked with many Departments to create an assessment system at the department level in order to move even more Departments into alignment with CLOA and HLC expectations for outcomes assessment. CLOA, as a committee, reviewed, and provided comprehensive feedback in narrative and rubric form to individual Departments. IR Director-Assessment, organized the data that came in and, where possible given the constraints of available technology, created some comprehensive reports on the results of assessment work. By FY 13 over 90% of Departments and Programsat the University had created learning outcomes, including most Student Affairs offices and programs. 80% or more of those programs had created assessment tools for at least one outcome, and were meeting CLOA’s annual expectations - completingassessment work and review of at least one learning outcome per AY.
In FY13, CLOA moved beyond seeking reports from Programs and Departments about individual goals within the assessment framework and created a template for a Department Assessment Plan. Plans were collected from Departments in June, 2013. The goal of the Assessment Plan template as opposed to a yearly report was to have Departments define their own plan for routine systematic assessment now that most Departments and Programs understand and have defined their program level outcomes and approaches. CLOA intended at the distribution of these plans to use a peer review process to help faculty and staff work together to highlight practices that work well. That process will take place in Fall 2013 and builds on models obtained from other institutions at the HLC Assessment Academy
Next Steps: Move Departments to systemized data reporting on all Learning Outcomes with larger, more comprehensive requirements for assessment work that recognize systems, data, and data response more than defining, mapping, and initiation. Continue strategizing ways to peer review the processes to grow the culture of assessment. While the first several years at Hamline have built an important and sustainable foundation in a culture that expects and anticipates learning outcomes, the next task of program level assessment involves focusing the University on the routine collection, review, and response to the data being generated.
5)Strategize and build frameworks to begin Phase II of Hamline University Outcomes Assessment – General Education Assessment
In FY 12, with the HLC reaccreditation visit pending in 2017, the new Strategic Plan calling for assessment of the Hamline Plan over the next several years, and a revision, or refresh, process underway to examine possible changes to Hamline’s undergraduate general education model (“Hamline Plan” curriculum), CLOA started to turn its attention not only to growing program level assessment, but also on beginning to establish the necessary structures for assessment of the undergraduate curriculum. In FY12, CLOA worked closely with the Undergraduate Curriculum Committee (UCC) and the Associate Provost to incorporate strategies and best practices from assessment into the revision approach. Because of that work, all revised and new Hamline Plan letters included a clear articulation of student learning outcomes with input gathered from all faculty through survey, forums, and task forces. During the first year of the refresh process, FY12, UCC and directors of the Hamline Plan clarified learning outcomes for existing letters and worked in concert to ensure that any newly proposed pieces of the curriculum clearly articulated student learning outcomes. Leondra Hanson, as Assessment Director, and Ken Fox, in his role on the UCC worked together to create a calendar for the first year of revision work and a process for UCC and CLOA to work together to ensure learning outcomes were integral to the revision process. Throughout the work on establishing a process, the focus was on making sure CLOA had a clear voice and input into helping shape a curriculum model that included a focus on assessment, while leaving the authority and responsibility for devising and defining the curriculum with the UCC and faculty.
As a result of the focused and determined efforts of UCC with the assistance and guidance of CLOA, the HP refresh process allowed for a curricular model that put renewed and transparent focus on student learning outcomes.
In FY13, the Assessment Director worked with the Director of Writing Across the Curriculum to create a model of assessment planning that could be extrapolated across each letter in order to create a routine assessment system. The writing project will be piloted in FY14. That assessment plan, which is similar in form to the templates and plans created at the program level will, hopefully, serve as a model for assessment planning across all the letters of the Hamline Plan.
Next Steps: Each letter of the Hamline Plan will need to have an Assessment Plan it can articulate with respect to measuring student progress. In FY14, CLOA will continue to collaborate with UCC on how to move forwarded with building structures and collecting data about multiple learning outcomes on a routine schedule.
Conclusion: Changes to University structure supporting CLOA and the growth of the committee itself in 11-12 and 12-13 allowed the committee to achieve significant positive results. CLOA successfully led Hamline through the HLC Assessment Academy, increased the participation and positive engagement by faculty and staff in program assessment endeavors,and built the foundation for an ambitious general education assessment strategy.
1