Needs Analysis
KIHd System Design Team
2004-2005
Yaa Gyasi
Akua Gyasi-Koduah
Wayne Roland
Tracy Soular
Table of Contents
1.0 Goal …………………………………………………………………...………….3
2.0 Target Population………….…………………………………………………….3
3.0 Discrete Trial……………………………………………………………………..3
4.0 Strategy…………………………………………………………………………...4
5.0 Formal Data Gathering………………………………………………………….4
6.0 Actuals and Optimals………………………………………...………………...10
7.0 Solution System.………………….……………………………………………..10
8.0 Goals………………………….…………………………...……………………..11
9.0 Conclusions………………………………………………..…………………….12
10.0 KIHd System User Diagram…….……...……...... ………….……………....13
11.0 Design Team Phase 2………………………………..………………………….14
12.0 Appendix……………………………………..………………………………….15
Needs Analysis
KIHd System Design Team
1.0 Goal:
Building on our informal data gathering from the initial analysis of Dr. Berhmann and the design team performance analysis, this document will:
· Detail the ongoing design team plan for formally gathering data
· Identify user actuals and optimals
· Present findings from a review of both data collections
· Propose a performance recommendation/solution system
The goal of the design team is to design and develop the KIHd System as a powerful and flexible tool for discrete trial data collection and analysis by teachers, parents, and school administrators, among others, who are concerned with the communication, social/emotional, cognitive and motor development of special needs children. During our extant data gathering from Dr, Behrmann’s proposal for funding, we learned that the current practice in discrete data collection is to make handwritten session notes. During our needs analysis, we will obtain insight into the current methods of educators collecting data in discrete trials, by observing and interviewing practitioners. We will also develop human resources who will be allies to the project, and sources of information for our ongoing research and development of the KIHd System.
2.0 Target User Population
Future KIHd System users will be drawn from people who collect and analyze discrete trial data, including teachers, parents school administrators and therapists. These people are collectively the target population for the KIHd System. <insert statistics>
3.0 Discrete Trials
Discrete Trial Training (DTT) is a method of addressing the learning challenges and knowledge deficits of special needs children. A discrete trial is a single instance of stimulus-response-reinforcement training that is repeated many times and across many domains to address deficits and difficulties in learner cognitive, communication, motor and social/emotional development. Typically an educator employing discrete trial training will ask a child to do something, such as “pick up the ball”(stimulus), to which the child will do something or nothing (response), and the teacher will either praise the correct response or correct the incorrect response (stimulus reinforcement).
4.0 Strategy
We will contact human resources through the Fairfax County School District, local special education teachers, and other educators working with special needs students with autism. We will develop a self-selecting sample group by issuing:
· An electronic mailing to these entities that introduces us and our interest in discrete trial data collection and analysis
· A questionnaire on discrete trial data collection and analysis
· A phone interview with discrete trial training practitioners
· A focus group of discrete trial training practitioners
5.0 Formal Data Gathering
<insert description
5.1 Data Gathering Method
We will use questions posed to discrete data collectors to test design team hypotheses as well as to gather new information. We will:
· Develop a standard set of questions for interview subjects which can be customized according to the subject’s expertise (parent, teacher, school administrator, etc.)
· Evolve questions over time as hypotheses are successively validated or invalidated and as new hypotheses are generated.
· Divide subjects among team members for interviewing
· Pose authentic example-based questions (i.e. "straw" and "ideal") to elicit active responses and to identify optimals.
5.2 Data Gathering Sources
The KIHd design team used the following sources for detailed information on discrete trial data collection:
§ We conducted several interviews with Subject Matter Expert (SME) on autism, Heidi Graff. This provided significant information and understanding of the spectrum of autism in children, as well as how children with autism are trained and evaluated.
§ We held a meeting with high-level administrators in the Fairfax County School District. We demonstrated the original KIHd prototype and they gave us feedback about the possibilities of the project.
§ We visited a local Preschool for Autistic Children (PAC) class to observe teachers and paraprofessionals collecting discrete trial data from their sessions with children with autism.
§ We conducted interviews with current users of discrete data collection and analysis methods.
§ We conducted an online survey of graduate students involved in discrete trial collection.
5.3 Telephone Interviews
We contacted ten users of discrete data for telephone interviews Out of those, we carried out interviews with:
· Subject Matter Expert in Autism
· Inclusion Specialist
· PAC (Pre School Autism Class) assistant
· Student of Applied Behavioral Analysis
· Instructional Therapist
Below are questions and answers to our phone interviews. Questions asked are in black. The different responses we got from the respondents are below the questions in color coded by respondent:
Section 1 – Data Collection
- What type of data collection system are you currently using?
- Probe data
- Data recorded at intervals or at conclusion of session by self.
- Probe data
- I used an ABA system, it was paper/pencil
- How do you feel about the current method of data collection?
- It was what worked best for her because she wrote down informal notes and came back to formalize it. However, it gave her twice as much work
- It is demanding on memory. Note taking muddies instructional context.
- I haven’t been collecting data very long and I don’t have strong feelings about it either way.
- It was a very inefficient system. We had lots of notebooks for clients. It was very inefficient.
- What is your intent for data collected?
- Communication with subject’s other therapist and parents
- To see what skills students have mastered - “So we can specifically see what is going on with students and mold their education.”
- “To give us the best picture of the client. It was also to track the child so we would know when to move the child to another level.”
- Are you able to benefit from the data you collect?
- Yes, she benefited form it. It gave her a good idea of seeing specific things in each student
- Yes, in terms of effective communication with therapist and parents
- Would you like to see the current system changed?
- Yes – It is very hard to collect data. It took her years to perfect the art of speedy data collection. She even got frustrated and stopped after 6 weeks.
- Yes, in terms of reducing memory and time demands
- It would be a good idea.
- I think it would be a good idea
Section 2 - Data Analysis
- What kind of data analysis do you currently use?
- A lot of anecdotal and graphs for behaviors
- None
- We just use a general overview
- “We would get together in a group and discuss what was happening with a child.”
- What are the steps that you currently use to analyze the data you collect?
- N/A
- None
- None
- “We would discuss the results in a meeting.”
- What was important for you to get from the data?
- N/A
- N/A
- N/A
- “Where to take the child next.”
- How long does your current method of data analysis take?
- N/A
- N/A
- About 30 minutes
- N/A
- What are the most important analysis outputs?
- N/A
- N/A
- N/A
- How often are you able to analyze your current data?
- N/A
- N/A
- At the end of every day
- What kind of data analysis would you like to use?
- N/A
- N/A
- N/A
Section 3 - Improvementsto Current Methods
- What kind of data system would you like?
- Something that could be done quickly
- Fast, useful data collection; audio note taking
- “Something easy to use. Not a lot of abbreviations. Something that allows for a standard set of definitions.”
- “Something to automate and coordinate the data.”
- Would you like to see some kind of electronic system?
- If that would be better – Yes
- Yes, if is simpler and faster and not fault-prone
- That’s a nice idea.
- “Yes, something to track the progress of students and to look at behaviors across people.”
- How do you ultimately see a better method of data collection and analysis helping you and your students?
- N/A
- Better record of session data
- “It would be very helpful. Just taking in all the data is hard. Having to work with all the data on all the students can be mind boggling.”
-“We could see across people and across behaviors and make decisions.”
- Are there any features you would like to see in an improved method of data collection?
- Not a lot of time, not cumbersome
- Not take a lot of space
- Not draw a lot of attention to itself
- A checklist of some kind
- Audio recording, immediate data notation
- Big buttons for different types of things you have to write down
- The ability to categorize
- A place to put anecdotal notes
- “Definitely a place for anecdotes and behaviors. Something in chart form. Something with a graph that would allow students progress to be shown visually over time.”
- What security issues exist for the district concerning student data?
Yes – Has laptop from school but cannot take IEP’s home. It has to be saved on diskettes and left at school
-None. Data is shared only with other therapist and parents
-None
- Would you be available to answer follow-up questions?
Fine to contact again
- Yes
- Yes, by phone
- How would you like us to contact you?
- Phone
- Email or phone
- By phone
5.4 Online Survey
We conducted an online survey at http://www.surveymonkey.com/s.asp?u=13111666706 of university students with knowledge of current data collection methods. This survey was inconclusive due to a lack of respondents.
5.5 Analysis of Interviews
Some of the major themes that came across through the interview include:
· Current data collection methods are currently paper and pencil methods.
· Discrete trial data gathering methods are cumbersome and time consuming.
· Data analysis methods require more time than most professionals working in the field have.
· Data collection methods are subject to errors due to heavy reliance on human memory and recollection
· Data analysis is time consuming difficult and inconclusive due to inappropriate collection methods used.
5.6 Meeting with Fairfax County Public School Administrators
On September 16th, 2004 as part of our data gathering we met with a group of Administrators from the Fairfax County Public School (FCPS) at the Belle Willard Center in Fairfax, VA. The purpose of our visit was to introduce this group to the KIHd System project and to explore the possibility of partnering with FCPS in the pilot test of the alpha KIHd System. The people we met included the FCPS Preschool Autism Class (PAC) Coordinator, the FCPS Autism Director, and the FCPC Discreet Trial Coordinator.
The KIHd team, together with Heidi Graff, Subject Matter Expert (SME) on Autism, presented an overview of the KIHd project and proposed to pilot test the KIHd system in one of their schools. This meeting was very productive for all parties. The administrators were very interested in considering a functional beta prototype for classroom testing. The design team learned the criteria and procedure for proposing the KIHd System for consideration by the FCPC for classroom testing. We learned the high priority placed on student data security by the FCPC group and were much encouraged by their enthusiasm for getting the KIHd System approved for testing by late spring, well ahead of the schedule we anticipate for the readiness of a beta device. The FCPC administrators promised to offer us any assistance they could and they suggested additional people who can also be of assistance when we are ready to submit a proposal for classroom testing of the KIHd System.
5.7 Observation of Franklin Sherman Elementary School PAC Class
On October 7th, 2004 the KIHd team visited the Franklin Sherman Elementary School in McLean, Virginia. The purpose of the visit was to observe the current method of data collection in a special education class with autistic children. FCPS uses a variety of data collection methods to measure progress including probe data, time sampling recording procedures, discrete trial, baseline and trial by trial data probes. During our visit we observed Special Education teachers and assistants who used discrete trial and baseline to collect data on children with autism. Our two and a half hour visit provided us with first hand information on the current method of data collection. This helped the KIHD team to understand the information we had received in the classroom and validated our notion of the need for the current method to be improved.
<insert details of what was learned>
5.8 Background
Following the mandates of the No Child Left Behind (NCLB) Act and the Individuals with Disabilities Education Act (IDEA), children with disabilities are required by law to have free appropriate public education. For children with autism this means that services must be designed to meet their unique needs and prepare them for employment and independent living. As part of this process of addressing the special educational needs of children with autism, data collection and analysis is required to measure student progress, evaluate the effectiveness of the student's individual program, and to identify skills and/or behaviors that need to be taught. There are a variety of data collection methods such as continuous data procedures and time sampling procedures that yield reliable, accurate, and valid data.
6.0 Actuals and Optimals
Based on our data gathering and analysis, this section identifies both the actual situation that exists for teachers and other educators collecting discrete trial data , as well as the optimal situation desired by this target population.
Actuals
· Collecting data by paper and pencil methods in a multi-step process