LEADS: INSTRUCTOR QUALITY AFFECTING EMERGENCY MEDICAL TECHNICIAN (EMT) PREPAREDNESS
Darlene F. Russ-Eft
OregonStateUniversity
Philip D. Dickison
National Registry of EMTs
Roger Levine
American Institutes for Research
Abstract
This represents one of a series of studies of the Longitudinal Emergency Medical Technician Attribute and Demographic Study (LEADS) being undertaken by the National Registry of Emergency Medical Technicians and the National Highway Traffic Safety Administration. This secondary analysis of the LEADS database, which provides a representative sampling of EMTs throughout the United States, examines the effects of instructor quality on the level of preparedness of emergency medical technicians (EMTs). Results showed significant differences, based on instructor quality, in the ratings on 10 dimensions of EMT preparedness for both EMT Basics and EMT Paramedics. Implications for HRD practitioners and adult educators and for researchers are discussed.
Key words: emergency medical technicians (EMTs), instructor quality
Introduction
Emergency Medical Technicians (EMTs) comprise a critical segment of the medical care delivery system in countries throughout the world (e.g., Carney, 1999; Chapleau, 2001; Hay, 2000; Suserud, Wallman, and Haljamae, 1998; Weller, 2000). In the United States in particular, there are nearly 17,000 Emergency Medical Services (EMS) agencies, approximately 506,000 EMTs, and more than 99 million annual emergency room visits (ACEP, 1998), and this number is growing. Cady and Scott (1995) reported that in 1995 approximately 28 million patients were treated and transported by EMTs. In most accidents and sudden illnesses, these EMTs are the first medical professionals providing medical treatment, and the quality of the care given at this initial stage will have an impact on the patient’s ultimate medical outcome (Cydulka, 1989).
EMTs comprise a large segment of the U.S. workforce, and successful performance of their health care duties requires adequate training, motivation and supervision. Accordingly, factors related to the training of these EMTs (for initial licensure, re-licensure, certification, and re-certification) yield a direct impact on the quality of critical pre-hospital medical care received by millions of Americans. As a result, various aspects of EMT training have been examined. For example, Cannon, Menegazzi, and Margolis (1998) investigated the relationship between training hours for paramedics and their performance on the National Registry of Emergency Medical Technician’s Paramedic (NREMT-P) examination. Janing (2001) studied the link between teaching approaches and learning styles. More recently, studies focused on the use of simulations games and multimedia (Stolk, Alexandrian, Gros, & Paggio, 2001), the effects of video case studies (Janing & Sime, 2001), and computer-assisted learning (Jones & Cookson, 2002). Nevertheless, few studies have examined factors associated with the quality of this training or the quality of the instructors and the resulting preparedness of the EMTs to perform their work.
Theoretical and empirical research focused on trainee populations has pointed to a variety of factors affecting such transfer of training (e.g., Baldwin & Ford, 1988; Holton, Bates, & Ruona, 2000; Holton, Bates, Seyler, & Carvalho, 1997; Russ-Eft, 2002). Many of these factors focused on the environment surrounding training, such as the organizational climate and supervisory support, or on issues related to training design. Indeed, the Rouillier and Goldstein’s study (1993) resulted in much of the recent emphasis on organizational issues related to the transfer climate. However, as stated by Stiehl and Bessey (1993: 33) “training and managing high performance is a learner-centered function. At a time when organizations are more customer driven, employees must be perceived as internal customers whose learning needs must be met by trainers and managers.” Thus, the quality of the instructor affects the quality of the instruction and the quality of the materials being used, which in turn, can affect both the learning and the performance.
An examination of the theoretical and empirical literature on trainee reactions to the instructor and the training shows mixed results. Kirkpatrick (1959a, b; 1960a, b; 1994) suggested that positive reactions lead to positive results in learning and behavior. Clement (1982) found a positive relationship between reactions and learning. In contrast, Bretz and Thompsett (1992) Dixon (1994), and Warr and Bunce (1995) reported that such relationships did not hold. A meta-analytic study by Alliger Tannenbaum, Bennett, Traver, and Shotland (1997) found that a positive relationship between utility reactions and performance but no significant relationships between affective reactions and learning or performance. More recently, however, Tan, Hall, and Boyce (in press) found that negative reactions to training resulted in higher learning and performance.
This study, then contributes to the needed literature on the role of training and instructional quality on EMT preparedness. Furthermore, it adds to the literature on factors affecting training transfer by examining the role of instructional quality on such transfer, defined as EMT preparedness.
Research Problem
The National Registry of Emergency Medical Technicians (NREMT) and the U.S. Department of Transportation, National Highway Traffic Safety Administration (USDOT/NHTSA) have been concerned about a number of issues related to the work of EMTs, with one of these being the quality of the education and training being received. The NHTSA has been identified as an agency within the federal government to work with the State EMS lead agencies to create national standardized education programs for EMTs. These two agencies have collaborated on issues related to EMT education and training (e.g., Brown, et al, 1999). Together the two agencies identified the need to research the EMS workforce as an aid to better understand the impact of future decisions from a regulatory and educational perspective.
The present study focused on issues related to the licensure, training, and certification received by EMTs. The major questions revolved around issues related to the quality of the instruction. We examined three specific variables: the quality of the instructors, the quality of the materials, and EMT preparedness. The quality of the instructors can have an effect on the quality of the materials (or at least the perception of that quality by the trainees) as well as the preparedness of the trainees. For statistical testing purposes, the null hypothesis was that there would be no differences in the level of the quality of the materials and in the level of EMT preparedness based on the quality of the instructor.
Methods
Background
This Longitudinal EMT Attribute Demographic Study (LEADS) is a joint venture between NREMT and NHTSA. The project is lead by a team of researchers who have experience as State EMS Directors, State EMS Training Coordinators, EMS System Managers, Emergency Physicians, EMS Educators, and survey researchers. It also includes staff of the NREMT and NHTSA. More about the LEADS effort can be found in Brown, Dickison, Misselbeck, and Levine (2002). The current study on education and training represents a secondary analysis of this larger study.
Sample
The sample for the LEADS studies, including the present study, comes from the NREMT database. This database consists of nearly 185,000 EMTs who are nationally registered in the U.S.. The sampling frame for the study is EMT-Basics and EMT-Paramedics who were currently registered at these levels by the National Registry of Emergency Medical Technicians, as of 15 September 1999. Separate frameswere established for EMT-Basics and EMT-Paramedics. These frames were stratified by duration of continuous registration at each level (“new”, referring to those registered at the level for less than one year versus “old”, referring to those registered for one year or longer) and by race (“white”, referring to those who self-identified as white, other, or did not self-identify versus “minority”, referring to those who self-identified as Asian, Black, Hispanic, or Native American.). Sample sizes were intended to maximize the efficiency of the sample for comparing different types of EMTs as well as for producing estimates of population parameters. The sizes of the strata and return rates appear in Table 1.
Case weights were calculated for respondents in each stratum, reflecting the individual’s probability of selection. These case weights were adjusted, within strata, for non-response.
Instrument development
Since this is a longitudinal study, the survey instrument was designed to consist of two sections. The first section, the “core,” consisted of 78 items and will continue to be readministered in each annual survey. The second section, the “snapshot,” has a different focus in each mailing. The first snapshot survey consisted of 41 items, focusing on EMS education and training. This first snapshot will comprise the basis for much of the current study.
Draft survey instruments were reviewed and modified, leading to the production of a pilot test instrument. This version was pilot tested on 42 EMT-Basics and EMT-Paramedics at 8 different locations. After respondents completed the instrument, a debriefing protocol was administered. The protocol elicited extensive feedback about specific survey items as well general feedback about the instrument and the proposed cooperation elicitation procedures. As a result of the pilot test, additional response categories were developed for several items, several items underwent minor revisions, and several items were deleted to reduce respondent burden.
Table 1
Frames and Sample Characteristics
Frame Size / Sample Size / Number of Respondents / Response RateEMT-Basics
White, New / 23,432 / 1,050 / 354 / 33.7%
White, Old / 45,804 / 1,050 / 296 / 28.2%
Minority, New / 2,093 / 551 / 129 / 23.4%
Minority, Old / 2,799 / 551 / 101 / 18.3%
Total / 74,128 / 3,202 / 880 / 27.5%
EMT-Paramedics
White, New / 6,376 / 840 / 302 / 36.0%
White, Old / 27,828 / 840 / 341 / 40.6%
Minority, New / 646 / 442 / 137 / 31.0%
Minority, Old / 1,075 / 440 / 130 / 29.5%
Total / 35,925 / 2,562 / 910 / 35.5%
Analysis items and categorizations
The items examined in the current study focused on the quality of the instructors, the quality of the materials, and the EMT’s self-assessed level of preparedness in various EMT tasks. Items on instructor quality asked about (1) technical knowledge, (2) practical knowledge, (3) teaching ability, (4) enthusiasm, (5) availability outside of class, and (6) professionalism. EMTs were asked to respond on a scale of “excellent” = 4, “good” = 3, “fair” = 2, and “poor” =1. “High-quality instructors” were defined as those receiving no “fair” or “poor” ratings, while “Low-quality instructors” were defined as those receiving at least one “fair” or “poor” rating. Items on the quality of the materials included (1) the textbook, (2) audiovisual materials (videos, slides), and (3) course equipment (mannequins, splints, etc.). These three items used the same rating scale as that used for rating instructor quality. As for preparedness, EMTs were asked to rate the following: (1) trauma patient assessment, (2) medical patient assessment, (3) cardiac arrest management, (4) airway management, (5) spinal immobilization, (6) fracture management, (7) hemorrhage control, (8) childbirth, (9) pediatric patient management, and (10) patient transport. EMTs rated their preparedness as “very well prepared” = 3, “adequately prepared” = 2, or “poorly prepared” =1.
Data collection procedures
Surveys were mailed out on September 17, 1999to the selected EMT-Basics and EMT-Paramedics. Included with the survey was a postage-paid return envelope and a letter that outlined the goals of the project and provided assurance of confidentiality. To motivate response, two round trip airline tickets to any location within the continental United States were offered to one randomly selected participant. After surveys were returned, they were scanned by an optical reader and used to create an analytic data file.
Non-respondent survey
Procedures.An abbreviated version of the survey, containing demographic, attitudinal, income, and educational items from the regular survey, was mailed out to 500 EMT-Basics and 500 EMT-Paramedics survey non-respondents in March 2000. Surveys were received from 154 EMT-Basics (30.8 percent) and 207 EMT-Paramedics (41.4 percent), optically scanned, and used to create analytic data files. This enabled comparisons of respondents and non-respondents.
Results. The responses of EMT-Basics and EMT-Paramedics who completed and returned the non-respondent survey were compared with the responses of demographically similar regular survey respondents. Discriminant analysis indicated differences in the response patterns of three types of EMT-Paramedics respondents and non-respondents (minority, new; minority old; white, new). These three types of EMT-Paramedics comprised less than a quarter (22.5 percent) of the total number of EMT-Paramedics. Differences were associated with responses to an item about EMS income in the past 12 months: Non-respondents reported significantly higher incomes than respondents. When this earning item was excluded, there was little evidence that non-respondents differed from respondents on demographic, attitudinal, or education items (Brown, Dickison, Misselbeck, and Levine, 2002).
Results – Demographic Data
Brown, Dickinson, Misselbeck, and Levine (2002) provided an initial reporting of some of the demographic characteristics of the sample. This section will present a brief overview of some of the results.
Age
The average age of both the NREMT-Basics and the NREMT Paramedics was 35 years.
Race/ethnicity
The racial/ethnic composition of the EMT-Basic and EMT-Paramedic workforces was similar. The vast majority of both EMT-Basics (90 percent) and EMT-Paramedics (92 percent) reported themselves to be White.
Gender
EMTs are predominantly males. Approximately 71 percent of the EMT-Basics were male, and 69 percent of the EMT-Paramedics were male.
Level of education
The Associate’s Degree was the highest level of education attained by 22 percent of the NREMT-Basics and 30 percent of the NREMT-Paramedics. About 21 percent of both EMT Basics and EMT Paramedics reported their highest level of education attainment was a Bachelor’s degree. About 5 percent of EMT-Basics and 6 percent of EMT Paramedics reported having received a graduate degree.
Level of practice
The majority of EMT-Basics (81 percent) and EMT-Paramedics (96 percent) reported practicing at their level of national registration.
Income
Basic EMT’s reported a median income from all sources of $23,350 compared to EMT-Paramedics reporting $37,282. When asked to report income from EMS related jobs, the EMT-Basic median income was $3,607 compared to the EMT-Paramedic reporting $32,460.
EMT and other work experience
Randomly selected EMT-Basics from the general population of the NREMT database reported working a median of about 2 years as an EMT compared to about 9 years reported by the EMT-Paramedics. Similar proportions of EMT-Basics (14 percent) and EMT-Paramedics (15 percent) reported having served as an EMS provider in more than one state during their careers.
Communities served
EMT-Basics (85 percent) and EMT-Paramedics (67 percent) do most of their EMS work in communities with populations under 150,000. EMT-Paramedics are more likely than EMT-Basics to be doing most of their work in larger communities. The percentage of EMT-Paramedics (11 percent) who reported doing most of their work in cities with populations of 500,000 or more was nearly twice as large as the percentage of EMT-Basics (6 percent). The percentage of EMT-Paramedics (12.5 percent) who reported doing most of their work in mid-sized cities was twice as large as the percentage of EMT-Basics (5.5 percent).
Workload
The median number of calls NREMT-Basics responds to in a typical week was reported to be 4, while the EMT-Paramedic reported responding to 18 calls during a typical week. The median number of hours NREMT-Basics perform the duties of an EMT during a typical week was reported to be 9, while the EMT-Paramedic reported performing the duties of an EMT 45 hours during a typical week.
Results – EMT Education and Training Data
Continuing Education
The maintenance of licensure or certification typically requires continuing education. The median number of hours of continuing education completed by the NREMT-Basics in the past 12 months was almost 26 hours, while the EMT-Paramedic reported completing almost 48 hours during the same time period. When asked about the usefulness of this continuing education, 77 percent of EMT-Basics reported that it was useful or very useful and almost none (0.2 percent) of them reported that the continuing education was useless. The EMT-Paramedics indicated similar opinions relative to continuing education with 83 percent reporting that the continuing education was useful or very useful and only 1 percent reporting that continuing education to be useless.
Instructor Quality
The results were examined separately for EMT Basics and EMT Paramedics, and the frequencies reported here represent the unweighted data. Among the EMT Basics, 616 rated all areas of instructor quality as “good” or “excellent” while 180 rated some aspect as “fair” or “poor.” Among the EMT Paramedics, 629 rated all aspects of instructor quality as “good” or “excellent,” and 262 rated some area of instructor quality as “fair” or “poor.”
To examine the issue of the effects of instructor quality on the quality of the materials, a series of t-tests were undertaken. The results appear in Table 2 for the EMT Basics and in Table 3 for the EMT Paramedics. In this case, the null hypothesis was rejected; significant differences appeared for all aspects of material quality. Thus, EMTs reporting a high quality instructor also indicated that the materials were of high quality.
Table 2
t-Tests on Quality of Materials as rated by EMT Basics
based on Instructor Quality
Material Quality / Difference / High Quality / Low Quality / t value / ProbabilityTextbook / .183 / 3.42 / 3.24 / 3.15 / <.002
Audiovisual materials / .403 / 3.13 / 2.73 / 4.98 / <.0001
Course Equipment / .396 / 3.32 / 2.92 / 5.53 / <.0001
Note: Material Quality is rated as “excellent” = 4, “good” = 3, “fair” = 2, and “poor = 1.
Table 3
t-Tests on Quality of Materials as Rated by EMT Paramedics
based on Instructor Quality
Material Quality / Difference / High Quality / Low Quality / t value / ProbabilityTextbook / .314 / 3.26 / 2.95 / 4.32 / <.0001
Audiovisual materials / .630 / 3.12 / 2.49 / 8.50 / <.0001
Course equipment / .531 / 3.20 / 2.67 / 7.41 / <.0001
Note: Material Quality is rated as “excellent” = 4, “good” = 3, “fair” = 2, and “poor = 1.