Designing Rubrics
AMTA 2012 Massage Therapy Schools Summit
Chicago, IL
February 17, 2012
This is a 2 hour course for instructors on how to create and implement rubrics. Although they can be applied to written essay questions, research papers, projects and oral exams, our focus will be on rubrics for practical exams in Massage Therapy.
Learning Outcomes
At the end of this presentation, participants should be able to:
1. List three uses for a rubric in a Massage Therapy Program
2. Identify the parts of a rubric grid
3. Determine the hierarchy of skill description
4. Create a rating scale for a rubric
5. Construct a rubric for a written, oral, and practical exam
Presented by:
Kathy Paholsky, PhD, NCTMB
Owner, Optimum Health Center, LLC.
Massage Therapy Program Coordinator
Schoolcraft College, Radcliff
Garden City, MI 48135
Ó 2012 Optimum Health Center LLC; Kathleen M Paholsky
This material is solely designed for seminar attendees. It may not be reproduced or distributed without written permission of the presenter. Note: the rubric samples provided in this handout can be used, modified and shared without my written permission.
Designing Rubrics
February 2012
Page 2 of 11
Rubrics
Rubrics are scoring tools for assessments.
Rubrics are standardized scoring tools. The word rubric means red and originally referred to portions of text that were printed in red ink to highlight information. When applied in the educational setting, the rubric is a scoring tool for assessments.
The simplest rubrics have ___pass/fail implications______.
Either the student performs the task or does not.
Rubrics can increase in complexity based on quality, quantity and skill of the work performed; they identify levels of potential achievement for each criterion.
At no time should the rubric complicate your assessment. In all cases it should add to the understanding of what needs to be done by both the student and the instructor.
Rubrics are typically given to students ___before the exam______.
Use them as working guides. It doesn’t serve the student or instructor if the exact requirements of an assessment are not clear.
Rubrics
· let the instructor identify the _____skills and performance______necessary
· let __students______know what is expected and how to be successful; they can prepare accordingly
· allow the assessment to be more ______objective______
· evaluate student on the full range of criteria instead of a single score
· are authentic; they measure according to real-life criteria
· vary expectations based on skill level: ex: beginners vs. advanced
Who?
What?
When?
Where?
Why?
· Benefits of using rubrics
· Challenges of using rubrics
The Value of Rubrics
Rubrics standardize assessment grading.
Once the criteria are identified, rubrics allow for:
· consistency in _____assessment______
· alignment with ____learning objectives______
· less test bias: reflection before reaction
· evaluate subjective material in the most objective and measurable way
· improved student motivation______
· repeatability with multiple evaluators
· objective information to provide for students continued progress
· evaluation of how the item was taught; course corrections needed
Rubrics are usually expressed in __a grid______that contains measurable/observable criteria in incremental designations. It also contains the values for each of those criteria. This allows for consistency and repeatability in assessments by more than one evaluator. A quality rubric will replace vague criteria with specifics.
Although it can be applied to a simple pass/fail exam, a rubric allows for expansion and explanations.
Rubrics:
o show students their _strengths and weaknesses______in each assignment.
o show instructors and course designers the specific areas of strength and weakness in student understanding and performance.
o provide a start for ______communication______when examining these areas.
Designing Rubrics
Rubrics are recyclable!
Rubrics need to be __useful______and ____totally meaningful______at many levels for assessment.
Rubrics align the evaluation criteria to the curriculum and learning outcomes.
Rubrics initially take a while to create, yet they are one of the best investments of instructor time. Include them in your planning and preparation!
Rubrics ___organize information______that allows for checks-and-balances not only of student skills but of the course itself. Although they can be created by one person, using other faculty ensures a better tool. Discussions with peers when creating rubrics bring to light differences in how we each identify the individual processes, the value we see in each item, and how a subject is taught. In some cases, students can provide valuable input to the design of a scoring rubric.
When designing a rubric, start simple then identify any important factors that may be missing. They can be specific or broad, detailed or generalized.
The value assigned to any of the elements allows for prioritization of each element. _You_____ determine the values assigned.
Here are some steps that can help when Designing a Rubric
1. Identify the skill (or item) to be evaluated.
2. Brainstorm all qualities and components; make a checklist
3. Find 3 to 5 main criteria for each skill; categorize and create descriptors
4. Create a rating scale (Q). It may be letters, numbers, or terms of value
5. Create a chart. Criteria on left side and rating scale along top
6. Describe the levels, from excellent to poor
Criterion 1 / Criterion 2 / Criterion 35 points
3 points
1 point
Helpful Hints:
Describe the ideal or model response first
Use absolute language: be specific, don’t say ‘always, seldom, good, appropriate
Review them with students before the test.
Designing Rubrics
February 2012
Page 2 of 11
SAMPLE RUBRICS
Rubric for Lubricant Use
Here is a simple rubric that provides different points for skill with the lubricant.
Simple Rubric for lubricant use
3 / Student uses sufficient amount to apply strokes with ease and no excessive slippage2 / Student needed to reapply or remove lubricant in more than one area
1 / Student frequently uses excessive or insufficient amounts of lubricant
Notice that the number of points is in a small range. If the use of lubricant is an important factor in your assessments, the numbers can be increased and additional criteria added.
Rubric for Draping
Since each of us may have a different idea of what is modest and what is comfortable, a rubric would provide clarity for students who are having difficulty performing one or both of these tasks consistently during an exam.
Example of designing a rubric for draping.
What are students expected to perform?
· Draping for modesty
· Draping for warmth
How are they expected to perform?
At beginning or advanced student skill level, or just draping?
· Draping is tentative and therapist hesitant when client is changing positions
· Student drapes for modesty and warmth at least 70% of time
· Student does not expose client at any time (rewrite as a positive)
Designing a rubric for draping would include draping for both modesty and comfort:
Points / Modestly draped / Points / Comfort draping / Points / Skill level5 / 100% / 5 / 100% / 5 / In command of materials
3 / 75% / 3 / 75% / 3 / Hesitant
1 / 60% / 1 / 60% / 1 / Very tentative
0 / ¯60% / 0 / ¯60% / 0 / Unskilled
What do you think?
· Where would you add “exposed client once during session”
· Where would you add “client expressed concern about exposure”
In addition to a grid, a few comment lines can be made available below the grid or on the student exam form for any specifics to individual student performance.
Example: Let’s say a student keeps the client well covered most of the time yet gets the sheets twisted up or lets them slide around when the client rolls over. In addition to the points given, provide feedback for corrective measures.
Comments: Student drapes for modesty and warmth at least 75% of time. Needed reminder from client to cover exposed leg.
Draping is awkward and therapist hesitant when client is changing positions
More columns could be added that address the student’s ability to use the draping material as a barrier for hygienic purpose, to add additional draping materials for a client in side-lying.
Rubric for Draping Material
Are the students graded on the draping materials themselves? If so, that can be separate from the rubric, possibly as a check list.
Draping materials are:
· Clean
· Fragrance free
· Opaque
· Natural materials (or hypo-allergenic)
· Size appropriate for client and table
· Sufficient draping material for warmth and modesty (extra towels, linens, etc)
Items can be added or removed as needed
What do you think?
Would you create a grid for this item?
Rubrics for Technique Assessment
During a practical exam, students are assessed on multiple techniques. Imagine that each of the strokes/manipulations has a value of 5 points toward the total.
This is an example of a simple rubric designed to assess massage strokes. Additional criteria could be added for hand placement and use as well as the comfort to the client.
Value / Quality / Criteria #1 / Criteria #25 / In command of stroke / Effectively accesses and mobilizes tissues, identifies areas of restriction / Performs with skill and ability to change pressure levels and speed
3 / Some hesitation / Effectively accesses tissues with some mobilization; unsure of areas of restriction / Performs mechanically
1 / Tentative / Accesses tissue with no attempt to move or mobilize / Unable to perform without coaching
0 / Unskilled / Did not perform; does not know technique / Did not perform; does not know technique
Here is the same rubric designed to assess petrissage specifically.
Value / Quality / Criteria #1 / Criteria #25 / In command of stroke / Lifts and mobilizes tissues, identifies areas of restriction / Performs with skill and ability to change pressure levels and speed
3 / Some hesitation / Lifts tissues with some mobilization; unsure of areas of restriction / Performs mechanically
1 / Tentative / Pushes or rolls tissue with no attempt to lift or mobilize / Unable to perform without coaching
0 / Unskilled / Did not perform; does not know technique / Did not perform; does not know technique
Designing Rubrics
February 2012
Page 2 of 11
Bibliography
BOOKS:
Burke, Kay. From Standards to Rubrics in Six Steps/ 3rd Edition. Corwin Press: October 2010
ISBN: 9781412987011
Gross Davis, Barbara. Tools for Teaching/2nd Edition. Wiley, John & Sons: February 2009 ISBN: 9780787965679
McKeachie, Wilbert. McKeachie’s Teaching Tips: 13th Edition. Cengage Learning: January 2010
ISBN: 9780495809296
WEBSITES:
The following Websites were accessed multiple times in December 2011 and January 2012.
How to Create a Rubric http://www.ehow.com/how_2083034_create-rubric.html
Kolar, Rachel. How to Design an Assessment Rubric. July 09, 2011 http://www.ehow.com/how_8710785_design-assessment-rubric.html
Langley, Michelle. How to Design Scoring Rubrics for your Classroom
http://www.ehow.com/how_6571847_design-scoring-rubrics-classroom.html
Discovery Education “Kathy Schrock’s Guide for Educators”
http://school.discoveryeducation.com/schrockguide/assess.html
Designing Rubrics
February 2012
Page 2 of 11