Schools
A Performance/Competence Model
Of Observation Assessment
Carol E. Westby
University of New Mexico Medical School, Albuquerque, NM
Wichita State University, Wichita, KS
Meave StevensDominguez
Patti Oetter
University of New Mexico Medical School, Albuquerque, NM
1
A
wide variety of developmental, speech/ language and motor assessment instruments and procedures are available for evaluation of children and adults. The purposes and types of assessments depend on who is requesting the assessment and what they want to know. Assessment may serve several different purposes: (a) screening to identify children at risk, (b) diagnosing to determine the extent of a problem, (c) gathering information to assist in making decisions regarding the best educational placement, (d) planning intervention programs, and (e) determining the person’s rate of progress and effectiveness of intervention (Bailey & Wolery, 1989). The specific assessment selected is determined by the questions that are to be answered. Table 1 lists the possible assessment questions and the types of assessment procedures that would be used to answer each question.
TYPES OF QUANTITATIVE ASSESSMENTS
State and federal agencies require assessments to determine if children qualify for services. The focus of these assessments is to determine how the child compares to others of similar age or experiences. Children are eligible for services if their performance falls below some preset value. Most states require the use of standardized norm-referenced tests for this purpose. For preschool children, some states permit the use of developmentally referenced assessment to determine eligibility. The issue in this case, is the location of a child’s skills or developmental accomplishments within a predetermined hierarchy that is usually based on the developmental patterns of mainstream Caucasian children.
For example, a child may qualify for services if a speech-language pathologist states that based on an analysis of a language sample, a child is 2 years or more below expectations for his or her age or cognitive level. For school-age children, parents and teachers may wish to know the specific curriculum knowledge the student has or has not acquired. For this purpose, criterion- or curriculum-referenced assessments are used.
These types of assessments tend to be discrete point assessments that tell us what questions a child can answer or what tasks a child can or cannot do. The information yielded is quantitative rather than qualitative. Such assessments seldom provide information concerning how the children arrived at the answers or the strategies they used to complete the activities. More importantly, such assessments provide little useful information to guide intervention.
TYPES OF QUALITATIVE ASSESSMENTS
If the purpose of an assessment is primarily to establish effective programming, then one needs to consider some combination of judgement-based, ecological-based, and dynamic assessment. These assessments rely on observation of the child’s performance in structured and unstructured settings. As ecological assessment considers the effect of the physical, social, and psychological context on a child’s performance (Bronfenbrenner, 1997). Ecological assessments often include judgement-based elements, that is, they include the insights, knowledge, and impressions of professionals and parents who work with the children. Ecological, judgment-based assessments may also be dynamic. In a dynamic assessment, the evaluator presents a task or series of tasks, teaches the tasks in one or several ways (often using language to mediate the learning), and
Table 1. Assessment questions and procedures.
How does the child’s level of performance compare with age or grade peers? / norm-referenced
Where are the child’s skills or developmental accomplishments with in a predetermined hierarchy? / Developmentally referenced
What specific knowledge base has the child acquired or not acquired? / criterion- or curriculum-referenced
What perceptions of the child’s status and progress are held by parents and professionals? / judgement-based
What social and physical qualities of the child’s developmental context (at home and school) affect performance? / ecological-based
How responsive is the child to intervention? What repertoire of problem-solving processes does the child employ or not employ? By what means is change I in the child’s performance best effected? / dynamic assessment
how the child observes which methods are most effective for the child, learns, and the strategies the child uses in learning (Lidz, 1991).
Ecological, judgement-based, dynamic assessment involves systematic observation of ongoing behavior. The observations may occur in naturalistic settings or in simulated or staged situations where children are asked to show how they would behave in particular circumstance. The observation should include a description of the physical environment such as the size of the room, pieces of equipment, type and amount of materials, temperature in the room, noise level, the activities and the adults and other children present and what they are doing (particularly the manner in which they are interacting with the children and effects of their interactions on the child).
Ecological, judgement-based, dynamic assessments are particularly useful in documenting factors that are not easily measured by traditional instruments, such as attention, motivation, goal directedness, reactivity, self-control, self-esteem, temperament, postural control, social orientation, play style, communication, problem-solving strategies, and so forth (Fleischer, Bledgredan, Bagnato, & Ogonosky, 1990; Neisworth & Bagnato, 1988).
An advantage of observational assessments is that they can be conducted in natural, familiar settings and can include persons familiar to the child. Bailey and Wolery (1989) maintain that “assessments conducted by strangers, using irrelevant tasks and in isolated settings will be limited in usefulness” (p.16). They cite Bronfenbrenner (1977), who suggested that:
The understanding of human development demands going beyond the direct observation of behavior on the part of one or two persons in the same place; it requires examination of multiperson systems of interaction not limited to a single setting and must take into account aspects of the environment. (514)
Ideally, and observation assessment should consider multiple domains (speech/language, sensory, motor, cognitive, social/emotional, temperament, self-regulation, etc.) across multiple settings (home, school, clinic) with multiple persons (parents and other family members, teachers, peers); use the judgements of multiple persons (parents, grandparents, teachers, therapists); and focus on the competencies necessary to meet current and expected environmental demands (Bagnato & Neisworth, 1991). Such a comprehensive assessment enhances the validity and usefulness of the assessment.
OBSERVATIONAL SNAPSHOTS
Silliman and Wilkinson (1991) suggested that observational assessments be conducted as a series of snapshots through different lenses (microcloseup, close-up, regular, and wide angle) that show different views of the child’s behavior. The microcloseup lens focuses on a particular skill the child performs or does not perform. That might include a child jumping over a rope, repeating words, drawing a diamond, or retelling a story. Through the microcloseup lens, one notes what the child does or does not do-establishes joint attention, explores objects, hops over the rope, repeats three but not four words, uses immature pencil grasp, or omits key elements in the story.
The closeup lens focuses on the social circumstances or interactions that affect children’s performance in any skill area. With whom do the children show their best performance? What types of interactions appear to help the children-sitting quietly beside the child and modeling or physically guiding an activity, verbally explaining how to do an activity, slowing down an activity, or physically holding or stabilizing the child during the activity?
The regular lens focuses on the nature of the activity. Does the activity require primarily visual, motor, or language skills, or does it require simultaneous integration of all of these abilities? How complex is the activity? How many steps are required to complete the task? How familiar or unfamiliar is the task? Is the task concrete or abstract?
The wide angle lens focuses on context in which the activity takes place-a classroom, therapy room, living room, or playground. Is the context hot, cold, noisy, quiet, cluttered, crowded, brightly lighted, etc.? How does this context affect the child’s performance? Does noise or do other children distract the child from the tasks? Does the child like to be in the center of the action or does the child seek to seclude him or herself?
Different disciplines tend to use different lenses when observing children. Speech-language pathologists and diagnosticians tend to use the microcloseup lens, looking at breakdowns in specific skills and processes. In recent year, speech-language pathologists are increasingly using the close-up-lens by considering interaction children have with other children and adults. They recognize that much of how children learn to communicate and what they communicate about is dependent on their interactions with others. Occupational therapists tend to use a regular lens, looking at the complexity of activities in terms of their tasks requirements, including sensory processing, organization of information, and planning and execution of oral, visual, fine, and gross motor planning and execution of oral, visual, fine, and gross motor planning skills. Families see their children through the wide-angle lens. They see how their children function in a variety of environmental contexts.
TEAM ASSESSMENTS
Public Laws 99-457 and 94-142 require that assessment be conducted by teams of professionals so that children’s skills in a variety of domains are identified. Child evaluations have generally included the assessment of multiple developmental domains, however evaluators have not necessarily considered how the domains interact, the effect of persons who interact with the child, or overall, what factors support or compromise performance.
Team assessment is critical because no single person can possess all the knowledge or skills necessary to assess an individual and develop an intervention plan that will address all of the child’s needs. Team assessments, with their judgements by multiple persons concerning multiple domains, should reduce erroneous placement decisions (Pfeiffer, 1992) and should result in more appropriate goals and intervention strategies (Allen, Holm, & Schiefebusch, 1978).
Traditionally, teams have been classified as multidisciplinary, interdisciplinary, or transdidisciplinary (Fordyce, 1981).
· Multidisciplinary teams contain members from multiple disciplines, but the members conduct their own evaluations, write their reports independently, and are little influenced by one another.
· Interdisciplinary teams involve greater interactions among team members, with each other member using information and suggestions from other members in interpreting their data. The evaluation report and intervention plan we written collaboratively.
· In transdisciplinary teams, multiple disciplines work together in the initial assessment, but the provision of services may be conducted by one or tow team members. Individual team members may relinquish their roles and train others to perform their responsibilities (Lyon & Lyon. 1980)
If families and professionals are to understand how the child’s various developmental domains interact and the factors that influence a child’s performance, interdisciplinary or transdisciplinary team assessment and intervention are essential.
An assessment that will positively impact the effectiveness of intervention for children should consider the factors that facilitate or compromise their performance in multiple domains, in multiple settings, and with multiple persons. The goal of intervention should ultimately be to improve a child’s quality of life by enabling them to be participating members in their families and communities and to experience a sense of personal competence. This can only be accomplished by a dynamic assessment approach that is conducted in an ecologically valid manner by individuals who are knowledgeable about both their disciplines and child development.
Such an assessment provides information that speech-language pathologists, occupational and physical therapists, or teachers can use to develop interventions or educational programs that are within the child’s zone of proximal development (ZPD). The child’s ZPD is the difference between the child’s level of performance when functioning in collaboration with a more knowledgeable partner. By working within a child’s ZPD, one can maximize development of the child’s competence and confidence (Hedegaard, 1990; Newman, Griffin, & Cole, 1989: Tharp & Gillimore, 1988).
THE NEW TeamS ASSESSMENT MODEL
NEW TeamS, a federally-Funded model demonstration project, is developing a team-oriented, ecological, judgement-bases assessment that seeks to identify what supports and compromises a child’s performance. The model is being developed to provide better answers to the questions of parents, therapists, and teachers and to link assessment with intervention. The NEW TeamS acronym stands for neurological, ecological, wholistic team system. This model maintains that adequate assessment of children requires:
· attention to neurological underpinnings of performance,
· an assessment conducted in ecologically valid/naturalistic settings,
· consideration of the whole child and how all aspects of the child’s abilities interact, and
· an assessment conducted by a team working in an interdisciplinary or transdisiplinary manner.
The model assumes a family-centered philosophy in assessment and intervention. Consequently, family input is interpreting the information gained from the assessment, and designing intervention strategies.
NEW TeamS Framework
Although the NEW TeamS model was developed primarily for infants, toddlers, and preschool children, the principles apply to individuals to any age. The model emphasizes an individual’s performance and competence. Performance refers to the manner in which persons act or are able to express themselves or respond in different situations. Personal competence refers to performing in a way that the individual feels good about. Specific competence is the ability to perform a task or activity in a way that meets some specific standards.
Because the goal of a NEW TeamS assessment is to identify what supports and/or compromises a child’s competence and performance, the orientation of the assessment is to understand what intervention strategies will contribute to the child’s development. To determine this, children are observed in a variety of activities with a variety of people. The goal is not to determine the child’s development level (although this could be done), where the child is in a developmental hierarchy, or if the child qualifies for services. Consequently, if these other goals are important, other types of assessment must also be conducted.