Cross, West & Teisl (2009)

Lesson 8

Resilience Project (RP) Observational Rating Scale:

Mentor Adherence

Rating Form and Coding Manual

Wendi Cross, Jennifer West, & Michael Teisl

University of Rochester Medical Center

Resilience Project (RP) Mentor Adherence Rating Scale: Lesson 8

I. Adherence toStructure: Mentors are rated on adherence to the training manual on 6 structural domains, including check in, review, introduction, teaching, active learning, review/generalization. The specific scoring criteria for each of these domains varies consistent with variation in thelesson, as outlined in the Child Component manual (Cross & Wyman, 2004). Structural domains are composed of one or more scoring items. Each “item” is individually scored using a 0-2 point scale and the scores for items are totaled within domain for an overall score. The duration of lessons also is rated, with a consistent target of 22 to 28 minutes across all lessons. Lesson duration is scored dichotomously (see scoring criteria for Time below). Overall Adherence to Structure is expressed as raw scores for each domain and as a total across domains for the lesson. Percent adherence also may be calculated in order to compare lessons with different raw score totals. See the Scoring Descriptions and Tables for specific information regarding the unique scoring criteria for each lesson.

**NOTE: The Resilience Project strives to link concepts and skills to each child’s individual experience. As such, a common scoring criterion is that Mentors “contextualize” lesson material for individual children. In the context of coding adherence, it is not always possible to discern if the teaching examples or scenarios Mentors use are specifically tied to a child’s context. Thus, if an example or situation is deemed to represent a common scenario for children in general, the coder can assume that the Mentor has “contextualized” the material.

II. Child Challenge Score: An additional constructis rated on the adherence rating sheet, reflecting the fact that the mentoring lesson is a transaction between child factors and Mentor expertise. When scoring the ‘child factor’ scale, it is important to keep in mind that Mentors are trained in effective limit setting and engagement strategies. The failure to use these techniques should not be the basis of these ratings.

Child Challenge is scored globally on a 3 -point scale.

1 = The child is generally compliant and responds to limits.

2 = The child presents some challenge to the Mentor. For example, the child may be inattentive and physically active in a way that is disruptive and/or may be verbally rude/challenging at times. Also, the child may be challenging by wanting and asking for things or trying to take things or may be impulsive verbally and interrupt throughout the lesson.

3 = The child’s behavior is difficult to manage. For example, the child may go under the table, push objects off the table, swear, be oppositional/defiant, or physically destructive.

The rating scale and coding system for Lesson 8 is below:

1

University of Rochester Medical Center © 2009

Cross, West & Teisl (2009)

Child ID #: ______Lesson ID #: ______Mentor ID#: ______

Lesson Date: ______Rater ID#: ______Date rated: ______

Lesson 8: Adherence Item / Structure / Rating Scale
0 (not observed)
1 (partially obs)
2 (full obs ) / Score / Consensus
1 / Check in / Check-in sheet/feelings poster is used to reflect on the Mentor /child’s feeling / CI=
2 / Review / Feeling 2 ways is reviewed (can be in vivo) / R =
3 / Intro. focus/topic / A story is used to illustrate the concept of Dimensions of Feelings / INTRO=
4 / Teaching Activity / Thermometer is used to sort feeling labels from 2 or more feeling dimensions into intensities and to elicit the child’s experiences with different intensities / TA =
5 / Highlights that high intensity feelings are the most difficult to manage and that it is best to stop before going over the top by using Mental Muscles
6 / Active Learning/ practice/ skill development / Examples from the child’s book/folder are used to practice identifying feeling intensities and discussing what level of feelings would be ok in different situations / AL =
7 / Thermometer (or other concrete illustrations of the concept) is used to actively engage the child in identifying and sorting feelings intensities
8 / Folder is used to highlight relevant information/relationships, etc.
9 / Review /Generalization / Lesson content (feeling words for different intensities/levels, helps to know what causes “over the top” feelings ) is reviewed / R/G =
10 / Lesson content (can try to stop strong feelings, Mental Muscles, hot zone) is reviewed
11 / Child is encouraged to notice when feelings increase/decrease in intensity (or MM) over the next week
12 / Time / 22-28 minutes (goal: 25 minutes) / 0 1 / T =
Sequence: 0 1 (not included in total score) / Total: (range: 0–23) / Total =
Child challenge
1 / 2 / 3 / Consensus
Not behaviorally challenging / Somewhat behaviorally challenging / Very behaviorally challenging

1

University of Rochester Medical Center © 2009

Adherence Coding System for Lesson 8

Check In (Item 1; range of score 0-2)

0 = Check in is not completed within the first 10 minutes, or is completed, but does not include the use of either the poster or the activity sheet.

1 = Check in is completed within the first 10 minutes and the Mentor uses the poster, but not the check in sheet,OR the Mentor fails to connect with or reflect upon the feeling the child indicates, whetherthe check in sheet is used or not.

2 = Check in is completed within the first 10 minutes and the Mentor uses a check in sheet. The Mentor reflects with the child about what is contributing to the feeling in a way that highlights feeling facts, expands vocabulary, and/or reinforces previous learning.

Review (Item 2; range of score 0-2)

0 = The Mentor does not conduct a review or reflect with the child on content or learning from the previous lessonwithin the first 10 minutes.

1= The Mentor conducts a review within the first 10 minutes, but either fails to connect with or reflect upon the child’s experience this particular week (e.g., “So did you notice if you felt2 ways this week? No? Ok.”), or mentions feeling two ways, but does not elaborate. The review may focus on feelings experienced ‘in vivo’rather than reflecting on the past.

2 = The Mentor conducts a thorough review within the first 10 minutes, reflecting with the child about their experience of feeling two ways either over the past week or ‘in vivo’, with two clearly identified feeling states. The review must enhance the child’s feeling vocabulary and/or build upon or reinforce previous learning. In cases where the child does not offer any content/memory, the Mentor uses his/her own experience/story to review feeling two ways.

Introduction (Item 3; range of scores 0-2)

0 = The Mentor never presents the introductory material or tells an introductory story, or does, but only includes one point on one feeling dimension.

1 = The Mentor introduces the concept of levels of feelingwith an engaging story abouttheir own experience. The Mentor only mentions two feeling points increasing on one dimension and/or doesnot include specific labels for feeling levels beyond “a little” and “a lot”. The Mentor receives a score of 1 for a good introductory story wherein they illustrate decreasing levels of feeling (e.g., furious  annoyed) rather than increasingor if the Introduction does not precede the Teaching Activity.

2 = The Mentor introduces the concept of increasing levels of feelingwith an engaging story about their own experience. The story includes the labeling of3 points of increasing intensity on a single feeling dimension, using descriptive vocabulary (e.g., annoyed  angry  furious, content  happy  thrilled).

Teaching Activity (Items 4-5; range of score 0-4)

**NOTE: The total Teaching Activity Score cannot be greater than 2 if Item #4 is administered out of order (i.e., The Mentor engages the child in active learning without teaching the concept first, although the content is eventually laid out by the Mentor).

Item 4: Feeling Labels

0 = The Mentor never leads the child in sorting labels for feelings of different intensity.

1= The Mentor leads the child in sorting labels for oneor more feeling dimension (or uses cards to identifylabels as part of a story), but fails to elicit the child’s experience with the feelings.

2 = The Mentor leads the child in clearly sorting labels for two or more distinct feeling dimensions and elicits or connects to the child’s experience with the feelings. The child may participate in the activity (e.g., putting tape on the labels), but the Mentor must teach the child how to sort before allowing the child to take the lead.

Item 5: Feeling Intensity

0 = The Mentor fails to highlightthat increased intensity on a feeling dimension (e.g., anger) is likely to lead to a problem.

1= The Mentor illustrates increased intensity on a feeling dimension that is likely to lead to problems (e.g., anger) and notes that there is a danger area (hot zone/hot spot/over the top). However, the Mentor does not introduce Mental Muscles, fails to explain that Mental Muscles are important for stopping or controlling feelings before they get into the ‘hot zone’, or does not complete this item within the first 10 minutes of the lesson.

2 =The Mentor illustrates, within the first 10 minutes, increased intensity on a feeling dimension that is likely to lead to problems (e.g., anger), notes that there is a danger area (hot zone/hot spot/over the top), and introduces Mental Muscles as a strategy to stop or control feelings before they get into the ‘hot zone’.

Active Learning (Items 6-8, range of score 0-6)

Item 6: Contextualizing

0 = The Mentor fails to practice labeling feeling intensities with the child.

1= The Mentor guides the child in the practice of labeling feeling intensities, but does not contextualize the activity for the child.

2 = The Mentor uses examples from the child’s folder/life experiences to practice labeling feeling intensities and the child is actively engaged (even if participating in the discussion). If no specific examples are available, the Mentor contextualizes by discussing a plausible alternative example based on common child experiences.

Item 7:Child Activity

0 = The Mentor does not use a visual aid to actively engage the child in a labeling activity.

1 = A thermometer or another visual aid/concrete illustration is used in an activity to label feelings of intensity, but either the child doesnot play an active role (e.g., places the words/pictures) or the child does not receive guidance in applying labels of varying intensity.

2 = The Mentor guides the child in using a thermometer or another visual aid/concrete illustration in an activity to label feelings of intensity. The child must be actively engaged (e.g., places the words/pictures) in the task of identifying labels for varying intensities of feeling.

Item 8: Folder

0 = The Mentor does not add any relevant information to the folder.

1 = The Mentor adds materials (e.g., thermometer, a lesson ‘worksheet’, single contextualized feeling word) to the folder, but does not clearly link the materials to the lesson or contextualize the additions for the child.

2 = The Mentor adds materials (e.g., thermometer with labels relevant to the child’s problem feelings, contextualized activity sheet, labels for intensities of a contextualized feeling dimension) to the folder that include contextualized written information that is relevant to the lesson.

**NOTE: The Mentor does not have to state that the material is being added to the folder. If a material is created, it is assumed to be added to the folder, even if it does not occur on camera.

ReviewGeneralization (Items 9-11,range of score 0-6)

Item 9: Review I

0 = No review is conducted.

1 = The review is conducted and focuses on only one key element of the conceptof feeling intensity (e.g., feelings can grow in intensity, feelings can get over the top, or in the “hot zone”), or focuses on both elements, but is not contextualized for the child.

2 = The review is conducted and the Mentor links the summary of both key elements of the concept of feeling intensity (e.g., feelings can grow in intensity, feelings can get over the top, or in the “hot zone”) to what occurred in the lessonand/or the child’s context. The review may occur ‘in vivo’ (e.g. utilizing the house).

Item 10: Review II

0 = The Mentor does not emphasize the importance of stopping strong feelings.

1= The Mentor indicates that feelings can reach a level of intensity where they may need to be stopped, but does not use a visual illustration, refer to the ‘hot zone’, or note that a child may ‘lose control’ at that intensity.

2 = The Mentor uses a visual illustration to highlight that feelings can reach a level of intensity where they may need to be stopped, labels this level the ‘hot zone’, and notes that it is important to stop feelings from growing in order to maintain good control.

Item 11: Generalization

0 = The Mentor does not encourage the child to notice intensities of feelings over the next week.

1= The Mentor encourages the child to notice the intensity of feelings or times to use Mental Muscles, but does not specifically contextualize to the child’s situation.

2 = The Mentor encourages the child to notice the intensity of feelings or times to use Mental Muscles, and contextualizes specifically to the child’s situation (e.g., “use your Mental Muscles in the classroom or in gym class”).

**NOTE: Summary statements may occur throughout the lesson, but the review/generalization must occur after all active learning segments have been completed. If the Mentor “piles on” a learning activity after conducting a good review, it should not detract from the overall review score, but should factor into the score for Pacing/Focus under the Competence rating.

Time(Item 12, range of score 0-1)

0 = The Mentor does not complete the lesson within the prescribed time frame of 22-28 minutes.

1 = The Mentor completes the lesson within the prescribed time frame of 22-28 minutes.

Sequence (range of score 0-1)

**NOTE: The score for sequence is not included in the total score for lesson adherence.

0 = Lesson items are not completed in the prescribed sequence.

1 = Lesson items are completed in the prescribed sequence.

1

University of Rochester Medical Center © 2009