MAKING PROGRESS IN ADULT BASIC EDUCATION:

HOW LONG DOES IT TAKE?

Dr. Mary Ziegler

Olga Ebert

Center for Literacy Studies

University of Tennessee, Knoxville

600 Henley St., Suite 312

Knoxville, TN 37996-4135

Acknowledgments

This study was conducted by the Center for Literacy Studies with the funds from the Tennessee Department of Human Services with the goal to further improve the educational component of the state Families First program. Dr. Mary Ziegler and Olga Ebert were project researchers. Large portion of data was collected by Michael Burford, a graduate student in Adult Education at the University of Tennessee. Data from several programs were collected by program staff not named here for the reasons of confidentiality; however, their assistance is greatly appreciated.

The reviewer for this project was Dr. Thomas Sticht of Applied and Behavioral Sciences in San Diego who offered us valuable comments and suggestions about the study design. Cary Springer, a consultant from the University of Tennessee Computing and Academic Services, provided assistance with the statistical data analysis.

We also appreciate the support of our colleagues at the Center for Literacy Studies who helped us by discussing, reviewing, and editing of this report. Dr. Donna Brian did the final review and editing. All these people made possible this exciting, although at times difficult, study.

TABLE OF CONTENTS

INTRODUCTION AND EXECUTIVE SUMMARY......

Summary of Findings...... 3

REVIEW AND ANALYSIS OF EXISTING STUDIES......

Overview of Reports on Multiple Studies and Comments from Test Developers......

Review of Existing Studies that Examined Learning Gains......

STUDY OF STUDENT RECORDS FROM 16 TENNESSEE PROGRAMS......

Research Questions......

Sample and Data Collection......

Limitations...... 14

Data Analysis......

Findings......

Days between levels......

Advancement according to participant group...... 17

Entrance levels of Families First and traditional ABE......

Rate of passing the GED tests......

Going the distance - Level 1to GED......

Summary of Findings......

CONCLUSIONS AND IMPLICATIONS......

Questions for Further Research...... 28

REFERENCES...... 30

APPENDIX...... 31

INTRODUCTION AND EXECUTIVE SUMMARY

Families First, Tennessee’s welfare reform initiative, provides cash grants, education, job training, child care, employment assistance, and transitional benefits to eligible adults in order to prepare them for work and a life without welfare. In June 1999, 10.4% of adults in Families First were enrolled in adult basic education classes.[1] Adult basic education (ABE) is a key activity for these individuals. Relatively little research-based information is available on the length of time it takes for adults in basic education classes to move from level to level or to achieve a GED. At the request of the Tennessee Department of Human Services, the Center for Literacy Studies (CLS) at The University of Tennessee conducted a study to examine the length of time needed for adults in basic skills classes to achieve learning outcomes in an ABE class.

This report focuses on providing policy makers and administrators in the Tennessee Department of Human Services with information about the length of time it takes to make learning gains in an adult basic education class. The report has five parts: (1) introduction, and executive summary, (2) review and analysis of existing studies, (3) study of a sample of records collected from adult basic education programs in Tennessee including methodology, (4) summary and comparison of findings from different data sources, and (5) implications for further research.

Tcommissioned The original plan for the study included “identifying performance levels that policy makers can reasonably expect from adult basic education programs with regard to achieving learning gains and obtaining a GED.” A critical part of identifying reasonable performance levels is knowing how long it takes to advance from one level to another or to achieve a GED. Because of the short time frame of this study (five months), we planned to review and analyze only existing studies that were conducted in adult basic education programs across the country. However, the review and analysis of these studies provided very little new information. Because of this, we decided to collect a sample of records from adult basic education classes in Tennessee to see what kind of information these might yield. The comparison of the data from existing studies across the country and the data from sample records from Tennessee produced both findings and questions for further research.

Summary of Findings

This study resulted in several important findings that may have implications for basic skills education in Tennessee. Two different types of findings emerged from this study. The first type was extrapolated from the literature on existing studies from the literature that examined the outcomes of literacy programs. The second set was a result of the examination of participant records in sixteen adult basic education programs in Tennessee.

Findings from existing studies

  • Participation in adult basic education results in learning gains that can be measured by standardized achievement tests.
  • The average length of time it takes to advance one grade level is approximately 90 instructional hours.
  • A positive relationship exists between the number of hours of instruction and achievement of a learning gain.

These findings leave many unanswered questions about learning gains in general and, specifically, about learning gains achieved in Tennessee programs.

Findings from the study of records from Tennessee programs

The study of 594 participant records from sixteen adult basic education programs in Tennessee revealed the length of time traditional ABE (383 cases) and Families First participants (211 cases) need to advance a level and pass the GED examination. Levels are defined by the federal government and used by state and local adult education programs. Although we would have preferred to collect the number of instructional hours it took to make a learning gain, instructional hours were not available. Therefore, length of time is captured in calendar “days” rather than instructional “hours.”

  • Participants in adult basic education need a median of 105 days to advance from Level 1 to Level 2. This time is longer for Families First participants (156 days) than for traditional ABE participants (84 days).
  • Participants in adult basic education who entered at Level 1 need a median of 202 days to pass the GED examination/ Families First participants need more time (285 days) than traditional ABE participants (139 days).
  • There are no significant differences between Families First and traditional ABE participants in terms of the median length of time needed to advance from Level 2 to Level 3 (77 days); from Level 2 to GED (97 days); and from Level 3 to GED (52 days).
  • More Families First participants enter adult basic education at Level 1 (49%) than traditional ABE participants (27%).
  • More participants who entered adult basic education in 1996-1998 at Level 2 and 3 passed the GED examination in 1999 (than those who entered at Level 1 (39%).

These findings may be affected by the characteristics of a participant sample from one of the programs that had numerous dislocated workers who entered at Level 1 and moved rapidly through the basic education levels and passed the GED. If this group did affect the findings, then traditional ABE participants may take slightly longer to achieve gains or pass the GED. Even considering this, these findings show that there is a significant difference between the Families First population and the traditional ABE population. Differences are primarily found in the number of participants who enter at Level 1 and the length of time it takes these individuals to make learning gains.

The following sections describe the project’s methodology, findings, implications and questions for further research.

REVIEW AND ANALYSIS OF EXISTING STUDIES

The purpose of this review was to determine the average amount of time needed to make learning gains that are equivalent to one grade level. Traditionally, studies on the outcomes of literacy programs are difficult to conduct (Beder, 1999). As Beder points out, most of the outcome studies are flawed to some extent, and because of realities present in the field, flaws are almost impossible to avoid. The characteristics of adult literacy education make research difficult (Beder, 1999). Open enrollment, sporadic attendance, high dropout rate, lack of staff experienced in testing and data collection, and the lack of resources needed to collect data are among the main problems present in literacy education studies. Mikulecky (1986) considers the variety of attendance patterns to be one of the major barriers to conducting research in literacy education. When researchers from the Center for Literacy Studies were collecting data from participant records, they found the same kinds of difficulties as those described by Beder and Mikulecky. Even though data are difficult to obtain and may be flawed, studies have been conducted on the learning gains achieved by participants in adult basic education and a percentage of these studies focuses on the length of time it takes to make progress.

Overview of Reports on Multiple Studies and Comments from Test Developers

In his comprehensive review of 23 studies of the outcomes and impacts of adult literacy education, Beder concludes that “as measured by tests, the evidence is insufficient to determine whether or not participants in adult literacy education gain in basic skills” (p. 113). Other studies, however, show that participation in ABE does lead to learning gains. Tracy-Mumford (1995), in her review of studies of programs funded by the Adult Education Act, found that “all twenty studies reported academic learning gains; the ten studies that relied on self-reported skill improvements cited higher perceived skills” (p. 6). According to Mikulecky (1986), “large scale evaluation studies suggest 100 hours of instruction per grade level gain is typical.”

Solorzano (1989a & b) found reading and writing gains in California students who were enrolled in California Literacy Campaign programs for three, four, and five months. These gains could not be translated to grade level equivalents so they cannot be compared to other studies that use grade level gain; however, there was learning gain as a result of participation.

The length of time it takes to make progress depends on what educational level an individual has achieved prior to entrance in an ABE program. Carnevale and Desrosches (1999) studied “unpublished data from Educational Testing Service on the skill gains of participants in prose, document, and quantitative literacy” (p. 118). They conclude that “moving the least skilled [welfare recipients] with the minimal skill level [that of a typical high school drop-out] up to the basic skill level [that of a below average high school graduate] could require up to 900 hours of additional education or training” (p. 118).

Sticht (1987) examined whether there is a link between the number of hours of weekly instruction and learning gains in his article, “Solving the Basic Skills Crisis.” He found that those who made the greatest learning gain per 100 hours were in programs that provided the fewest instructional hours. In Sticht’s and other similar studies, the gain rate is commonly expressed as grade level gain per hundred hours of instruction extrapolated from the actual hours of instruction and actual gain. For example, in one of the programs, the learning gain of one grade level after two hours of instruction would amount to a 50 grade level gain after 100 hours. This gain rate index assumes that the rate of learning would be the same regardless of the length of instruction. With learning gains studies, Sticht cautions, one must bear in mind that “the gain rate index can be absurdly high when test-retest artifacts, such as regression to the mean, or warm-up effects produce post-test scores that are higher than pre-test scores after only a few hours of instruction” (p. 9).

In addition to reviewing existing studies, we contacted the organizations responsible for developing two of the major tests used in this country to measure adults’ basic skills (TABE and GED). When asked, “How much time is needed to achieve tested learning gains?” project personnel could not offer even an approximate answer to that question. A long-time project director in the Product Planning and Management Department at CTB/McGraw-Hill (TABE publisher) said that CTB has not published anything about the length of time it takes for adults to achieve learning gains. Personal experience led the project director to the conclusion that “the time it takes will vary, depending upon the student and the quality of the instruction” (electronic mail communication, May 1999). A research analyst at the national GED testing office offered a similar comment saying that the length of preparation for the GED does not necessarily correlate with the GED score. The research analyst said that many of those who pass GED on the first attempt often do not attend ABE classes at all, or attend for very short time (personal communication, May 1999).

To further examine the question of the length of time needed to make learning gains, we identified 28 of the relevant studies that measured tested learning gains. A bibliography of the studies is included in the Appendix.

Review of Existing Studies that Examined Learning Gains

Twenty-eight studies measured reading gains (for the complete list of the studies, see the Appendix). The studies were identified through the ERIC database, and from several reports that synthesized research focusing on measuring outcomes in Adult Basic Education (Darkenwald, 1986; Mikulecky, 1986; Sticht & Armstrong, 1994; Beder, 1999; and Medina, in press).

For this project, we analyzed the studies that reported learning gains measured by several different tests that are commonly used to determine whether or not individuals have made progress in their learning. Across the 28 studies, TABE (Test of Adult Basic Education) was used 13 times, CASAS (Comprehensive Adult Student Assessment System) was used seven times, SORT (Slosson Oral Reading Test) – three times, Woodcock Reading Master Test – two times, and ABLE (Adult Basic Learning Exam), Stanford Achievement Test, NALS-based (National Adult Literacy Survey) test, Botel Word Recognition Test, and Gates Reading Survey were each used once. Two studies used both TABE and CASAS. Since CASAS does not use grade level to show gain, we assumed that five CASAS points are equal to one grade level[2]. The term grade level refers to the kinds of skills a person would have at a particular grade level in the kindergarten through high school system.

Sticht (1987) reminds researchers and practitioners that it may not be appropriate to use grade levels with adults, but it is the only “common denominator” that makes the review of studies utilizing different tests possible at all. Even though different tests use grade levels, the way grade level is defined may not be the same. For example, one standardized achievement test may place a person at grade 5 and another test may place the same person at grade 7. This makes comparing the skills a person achieves difficult since no test defines skills in exactly the same way as another test. However, most programs, including those in Tennessee, do not differentiate between the different instruments used to determine a person’s grade level. They equate the different tests for reporting purposes. Even within the same test (for example, TABE), different results would be obtained if a different test version were used. Unless administration of the appropriate version of the test is rigorously monitored on all occasions with each learner, it is not possible to be sure that learning gains are measured correctly.

When trying to compare one study with another, an additional barrier to drawing conclusions is that the number of subjects in each study varies widely. Therefore, there are limitations in data reported on test scores and learning gains.

Even though the limitations of the 28 studies may be significant, there was no other way to compare the results of these studies in order to address the research question. We decided that the quantity of information found in the studies compensated somewhat for the imperfect quality.

The following table presents averages of instructional hours and tested learning gains from the 28 reviewed studies. The first eleven studies examine and summarize learning gains in multiple programs, while the other 17 focus on one program each. Four of the studies reported two sets of results for different subgroups that brought the analysis to 32 points of data on instructional hours and tested learning gains. In some cases, particularly with studies of multiple programs, instructional hours reported in this review are, in fact, averages of the range of instructional hours reported. For example, Sticht (1982) uses the range of 80-120 hours of instruction to report a gain of one grade level; for the present review, an average number of 100 instructional hours was used. Such an approximation was used solely in order to have a homogeneous set of numbers and make statistical analysis possible. For more precise information about findings from the reviewed studies, the reader should turn to the “Bibliography” section of this report.