University Learning Outcome Disciplinary Lens Assessment Plan and Reporting Guidelines

University Learning Outcome Disciplinary Lens Assessment Plan and Reporting Guidelines

Table of Contents

1. Overview of the assessment plan
2. Choosing and scoring data for reporting
3. Submitting data through the survey link

Overview of the Assessment Plan

Why are we conducting regular assessment of the University Learning Outcomes (ULOs) in Disciplinary Lens classes? The ULOs are the “glue” that academic and social learning together in the undergraduate curriculum. Assessment at the course level focuses our attention on the outcomes and lets us answer the essential ULO questions:

• To what extent are our students learning the skills, knowledge, and habits of mind that make up our outcomes?”
• What do we do to support that learning – and what might we do to continually work to improve learning?
• What supports do we need to enhance learning?

Phases of the Assessment Plan

A faculty-driven continuous improvement cycle is at the heart of the University Learning Outcome (ULO) Assessment Plan. In this cyclical assessment model, the initial collection and evaluation of evidence is just the first phase of a more comprehensive, four-year process. The four-phase assessment cycle begins with evidence collection but also includes making plans for change, supporting faculty and staff development, and reviewing progress.

Phase 1: Active Assessment

Year one. Faculty collect and evaluate evidence based on student learning in their courses. Faculty report on their assessment activities through the assessment reporting survey sent to their Boise State email address each semester during the reporting year.

Phase 2: Distribution and Development

Year two. The results of Phase 1 are collated and shared with the university community. Departmental reports are given to the academic departments. Departments are asked to review reports and develop plans for changes. If needed, faculty development plans are created and support for development is requested.

Phase 3: Implementation

Year three. Departments and programs implement changes and review the progress they have made. Additional adjustments are made including providing faculty development resources if needed.

Phase 4: Formative Assessment

Faculty maintain changes made in phases two and three making revisions as needed.

During the four-year cycle pictured above, two to three of the eleven University Learning Outcomes (ULOs) will be in phase one each year, and all eleven ULOs will be assessed in the course of a full four-year assessment cycle.

ULO Assessment Reporting Schedule

The ULOs are divided across four years of the cycle so that reporting is manageable. Table 1 illustrates the grouping of ULOs and the target courses for assessment in each year.

Table 1 ULO Assessment Reporting Schedule

Communication in the Disciplines (CID) and Finishing Foundations (FF) courses will be assessed as part of the periodic review of academic programs.

How to Approach Phase One ULO Assessment Information Gathering

• Faculty are encouraged to use student work that originates in the course such as tests, papers, speeches, posters, quizzes, projects, e-portfolios, etc.
• Faculty may choose whether to average all grades on an assignment or use a sampling approach.Averaging assignment grades might work best with multiple choice assessment approaches, while sampling works better with papers, projects, speeches, etc. Average of final course grades is NOT a good method for assessment. Course grades may include extra credit, participation, and other points that are not related to the ULO.

The following section provides guidelines for selecting and gathering assessment material.

Choosing Data for Assessment Reporting

Faculty decisions about the evidence to use for assessment depends upon the types of pedagogies used in the course.

Essays and Projects

If the course includes essays and projects, it is possible that re-reading a group of papers in light of the ULO criteria will help you determine the level of performance on that outcome. This type of review is much faster than grading as you are not marking a paper for the same criteria. Not all of the projects need to be assessed! Use the sampling table below to determine the number of randomly drawn projects you should review for reporting.

Table 2

Graded Tests

If the course includes graded tests that can be connected to ULOs, whether in their entirety or by identifying particular questions, test scores may help determine ULO performance. If tests are not currently mapped to ULOs, assistance on creating useful test questions for outcomes assessment is available from the Center for Teaching and Learning.

1. Faculty who use test or quiz scores for assessment have provided the following suggestions for using these scores for assessment.
2. On a spreadsheet, connect problem numbers, problem set (or entire quiz/exam) to ULO criteria across the header row.
3. Students are listed in the first column.
4. Scores for the identified items are filled in through the semester (see figure 1)
5. At the end of the semester calculate an average score for each of the items associated with a ULO criteria (see figure 2)
6. Relate the level of the cores to a ULO proficiency (1-4) based on the questions asked.

Figure 1: Graded Work Mapped to ULO Criteria

Table 3

Ungraded Classwork

One minute papers or ungraded journals can provide a rich source of information for ULO assessment. Faculty can collect these items throughout the semester and quickly sort responses into categories to answer ULO assessment questions. See below for more on strategies that are useful with classroom assessment techniques such as this.

Example

Professor uses “minute papers” 4 times during the semester to assess students’ ability to communicate mathematical ideas (ULO 7.2). She divides the stack into exemplary (4), good (3), developing (2), and unsatisfactory (1) based on the mathematics rubric criteria. She keeps in mind that 4 is the level we expect for graduating students.

She averages the scores over the 4 times she uses this method.

The combined average scores for the class is a “3” which she enters for the “non-graded work” assessment on the reporting form.

E-Portfolios

E-portfolios are created by Boise State students in their first year (for those who enroll in UF 100 and/or First-Year Writing). Asking students to add a section to their e-portfolio for your class is an easy way to gather assessment evidence and to help them connect their learning in your course to their overall learning at the University. The IDEA shop can assist you in building e-portfolio assessment into your class and using it for reporting.

Reporting Assessment Data through the Assessment Survey Link

At the end of the semester, a link will be sent to your email address for each course you teach. Multiple sections of a single course can be combined onto one submission.

Follow the instructions on the assessment survey to input the course information into the survey.

Following the Phase One instructions above for reporting data or sampling, complete the table asking for your assessment information (see figure 3, below).

The survey should be completed with the assessment information you have collected. If you are missing information for a particular ULO criteria, please leave it blank. You will make note of missing data in the next section.

Rubrics for the ULOs are available online through the Foundational Studies ULO Rubric page.

After entering the numerical scores for your assessment data, you will respond to three open-ended questions. The first asks for a summary of the assessment activities or assignments. The section asks, “what conclusions have you drawn from our assessment data about the students’ learning?” The third asks for conclusions or implications for future planning based on the assessment review.

Follow up from Assessment Reports

Assessment reports are summarized by the Foundational Studies program and returned to departments for their review. Reviewing assessment reports can be as simple as reading the findings and making small adjustments in the courses or in the reporting. Other actions may include curricular revision or pedagogical training. The Center for Teaching and Learning and the IDEA Shop are excellent and confidential resources available for departments to use for minor to major changes or shifts.

Phases two and three ask faculty to collaborate with other faculty to review assessment reports or other curricular or pedagogical questions raised during the assessment process. These steps comprise phases two and three. If changes are minimal they will be completed in phase two, but if they are more complex, using the time for both phases two and three may be important.