When, How, and How Much?
A Guide to Designing and Implementing a Departmental Assessment Plan(Based on Barbara Walvoord’sAssessment Clear and Simple)
Contextual Note
On April 19, Western’s Accreditation and Assessment Advisory Committee (AAAC) is bringing Barbara Walvoord to campus to present an assessment workshop for departments. An assessment specialist known for keeping assessment “Clear and Simple,” Walvoord’s workshop will help departments create simple, sustainable plans “…useful to the department for its own goals, and consonant with regional and professional accreditation guidelines.”
Some departments believe it will be simpler to complete their assessment plans during winter term. To this end we present the following guidelines and standards, drawn from Dr. Walvoord’s book Assessment Clear and Simple.
Working Definitions
Accreditation is a process of peer-review by which university practices are evaluated against a set of published standards.
Assessment is the systematic collection of performance data in order to inform decisions and make improvements. Academic assessment involves three key steps: goals, information, and action.
Direct Assessment Measures evaluate student performance of a learning objective, rather than reports or perceptions of such achievement. A common direct measure would be a capstone paper or project.
Indirect Assessment Measures evaluate reports or perceptions of student achievement of a learning objective. A common indirect measure is a survey.
Steps in the Assessment Cycle
Walvoord breaks assessment into three simple steps:
Step 1: Goals (Also called “outcomes”)
Departments decide what they want their students to learn.
Step 2: Information
Departments research whether their graduates have successfully learned what they wanted them to learn. Walvoord recommends that departments use both direct and indirect assessment measures when gathering information.
Step 3: Action (Also called “closing the loop”)
Departments use their research findings to make changes and decisions that improve student learning.
NOTE: This guide addresses Step 2, providing guidance and minimum standards for researching whether students have successfully learned what a department wants them to learn. This assumes that the department has previously established its goals, or program learning outcomes.
Orienting questions: When, How and How Much?
When?
A department can collect information on student learning in classes while students are learning, or at the end of the program in a capstone course, concluding exam, or practicum. While it is ideal to collect information both during and at the conclusion of an academic major, concluding assessments are the simplest way of demonstrating whether graduating students demonstrate that they have met the program learning outcomes.
How?
To oversimplify the matter, there are two principle ways to directly assess whether students have learned what a department wanted them to learn: standardized licensing or professional exams, and classroom work (papers, portfolios, exams).
Standardized licensing or professional exams.
Departments can use licensing or professional exams to assess outcomes that are directly measurable via such exams.
Classroom Work: Papers, Portfolios and Exams.
Papers and Portfolios.
A capstone or senior seminar provides a particularly rich opportunity for direct program assessment. Papers permit the assessment of many outcomes; capstone papers place that assessment at the end of a program, permitting faculty to determine whether graduates ultimately did or did not learn the knowledge and abilities they wanted them to learn.
Walvoord suggests that a committee of faculty evaluate a sample of capstone papers, portfolios or assignments against a rubric describing departmental standards for each outcome, or against anchor papers that provide examples of work that is “at departmental standard” for particular outcomes.
Faculty findings are summarized and shared with the department, which discusses where students have and have not met the program learning outcomes. Based on this information, the department plans program improvements.
Exams
Alternatively, faculty can look for final exam assignments and questions that permit assessment of departmental outcomes. These can be evaluated against rubrics or anchor answers which indicate whether a departmental outcome has been achieved. Faculty volunteers can collect the results of this assessment to share with the Department, which discusses where students have and have not met program outcomes and what steps can be made to make improvements to improve learning.
How Much?
Walvoord states that one direct assessment activity, such as those described above, which involves a fair representation of department members and which covers all departmental outcomes at the conclusion of a program, is largely sufficient for accreditation standards. Combining this concluding assessment with an indirect assessment, such as a satisfaction survey, will typically meet all regional requirements for program assessment.