Aspects of Text ComplexityProject Why ComplexTextMatters David Liben
The AmericanCollegeTestingService, in its influential study“Reading Between the Lines” (ACT 2006), determined a benchmarkscore ontheirreading test;51%of students scored above this benchmark.These students weremore likely to:
Enroll in college.
Earna gradeof B orhigher in first‐year U.S. historyandpsychologyclasses.
EarnaGPA of 3.0 or higher.
Returnfora second year atthe same institution.
It wasalsofound that 47% ofstudents whometthereading test benchmarkmet the science testbenchmark as well, whereasonly5%ofstudents who did not meet the readingbenchmark met thescience test benchmark. This is a particularly interesting findingin light ofrecenteffortstoboost K‐12 science learning. The51% figureof test takersmeeting the benchmarkwasthe lowest in overa decade.
Student responses were analyzed withthegoalof determining what patternsmightdistinguishstudents scoringabovethe benchmarkfromthose below. The major findingsfollow:
1.Literal vs. inferential questiontypefailedtodifferentiatestudentsscoringabove the benchmarkfrom those scoring below (p. 13).
2.Questions focusing on textual elements—main idea/author’s purpose,supporting details, relationships, meaning of words, and generalizations andconclusions—also failedtodifferentiate
students scoring above from those scoring below (p. 14).
3.The clearest difference of performance between the twogroups wasdegree oftextcomplexity, in the passagesthatacted as “sorters” withinthe ACT. Thisfinding held trueforboth males and females,all
racial groupsand was steady regardlessof familyincomelevels(p. 16).
This is a stunning finding. The textual elements described above and inferentialquestions in general constitutemanyoftheessentialelementsof what we usually think of as“criticalthinking.”Developing these skills instudentshas been a major focus ofeducationaleffortsin all disciplines for decades.Yetthe ACT study shows that, atleast forthis group ofnearlya halfmillion high school students, critical thinking does not distinguishthosewho arecollegeandcareerreadyfrom those who are not; facilitywithreading complextextdoes.
Textcomplexity onACT’sReadingtests (the ACT, PLAN,andEXPLORE,coveringgrades12,10and8 respectively)wasdivided intothree levels of complexity: uncomplicated, more challenging, and complex (p. 14). In looking atscores basedonthiscomplexitygradient thefollowingwas found:
1.Students scoring belowthebenchmark (49%of the568,000takingthe test) scorednobetterthan chance on multiple‐choice items associatedwithcomplex text,the mostchallengingofthethree levels.
2.Only studentswho obtainednearly perfect scores (35 outof 36) did as well on complextext as they didonthe lesschallenging text,indicating thatasignificant number of studentswho metthe
benchmark still scoredrelatively poorlyoncomplextext.
Fourhundredandsixty eight thousand studentstook the 2006 ACT exam. All were applying or considering applying to some formof postsecondary education and thereforewere likely to engageseriously with this test. Despite this, 49% , nearly a quarter ofa million students,performednobetteronthe more complex readingpassagesthanif thesepassages were written inSanskrit.
Howdid we arrive atasituationwhere so many of our studentsfail to understandcomplextext? We will address this question, as well asthe consequences this problem has generated, boththose alreadypresent and those likely to emergeor become more widespread over time.We begin withthe causes.
1.School BooksandReading DemandsK12HaveBecomeEasier
Chall et al. (1977) found a 13‐year decrease from 1963–1975 inthe difficultyof 11th‐gradetextbooks in all subjects;thiscorresponded with concurrent declines in SAT scores. She foundasimilar pattern for6th‐gradetexts butnot as clear‐cutasfor older students. Similarly, declines in first‐grade basal readerscorresponded withdeclining SATscores10 yearslater.
Hayes, Wolfer, and Wolfe(1996)foundmore:between1963–1991, average lengthofsentencesin reading textbooksK–8(basals)wasshorter thanin bookspublishedbetween1946–62;in7thand 8th
grade readers (usually anthologies, very widely used), the meanlength of sentences decreased from
20 to 14 words. Vocabulary also declined: the vocabulary levelof8th‐gradebasalreaders after 1963 was equivalentto 5th‐gradereadersbefore1963;12th‐gradeliteraryanthologiesafter1963were equivalent to 7th‐gradereaders before 1963.
Hayesalso foundthatthoughthe vocabulary level of words in basalreadersforgrades1–7increased each year, high school literature booksdid not increasein vocabularydifficulty for each yearanddid
not differgreatlyfromgrades 7–8 literature books.
Hayesalso foundthatthoughscience books were more difficult than literature books,only booksin
AP classes had vocabulary levels comparable to even newspapers of the time.
The spanof yearsHayes’ work coverscorresponded with SAT declinesin thesameperiod.Hayes addresses the question ofwhether declining SAT scoresreflected demographic changesin students
takingthetest. He pointsout thatthe yearsforthe decline do not match up with the years for the
demographicshift; more pointedly he notes that the number of students scoringin the highest ranges
(600‐800)decreasedbothrelativelyandabsolutely.
Data since 1962 (Williamson, 2004) showa305L (Lexile) gap between end ofhighschooland collegetexts, equivalent to 1.5 standard deviations, ormore thanthelexile difference between the4th
grade NAEP and the 8thgrade NAEP.
Although data after 1992 are not asthorough, it should be notedthatthe SATwas re‐centered in the mid‐90s,thus essentiallyaddingabout80pointstothe verbal scores (Adams, in press).
These data do not include analysis of elements of text cohesion,which mightgiveadifferent picture (McNamara, in press). That being said, while no measure oftextdifficultyisperfect,what is relevant in these numbers isthe steadydecline over time,across grades,in sophistication and difficulty of text,andthe resultingcorrespondence with droppingSAT scores.
Sothe texts studentsread, or certainly many of thetextsstudentsreadK–12,becameeasierafter1962.What abouttextsstudentswere asked toreadin collegeoverthatperiodand into our current period?
2.CollegeBooksandCollege Reading HaveNot GottenEasier
Lexile scoresofcollegetextbooks have notdecreasedinany blockof timesince1962 and in facthave increased (Stenner, in press).
Hayes(1996)foundthatvocabulary difficultyof newspapershadremainedstable overthe periodof his study.
Hayes (1992) found thatword difficulty of every scientific journaland magazine he examined between1930–1990hadincreased.
Relatedtotheabove,aCollege Boardresearchreport (2005)showsthat college professorsassign more reading fromperiodicalsthan do high schoolteachers.
3.CurriculumandPedagogy May HaveExacerbated theProblemofDeclining K12 TextComplexity
RelativetoCollegeDemands
Students in high schoolarenot only readingtexts significantly less demandingthanstudents in college, but instructionwithanytextsthey doread isheavilyscaffoldedcompared tocollege, where studentsare routinely expected to read more independently (National Governors Association& Council of Chief StateSchool Officers, 2009).
Students incollege are heldmoreaccountable forwhatthey readthan studentsinhigh school.
College instructorsassign readings,notnecessarilyexplicated in class, for whichstudentsmightbe held accountable throughexams, papers,presentations, orclassdiscussions.Students in high school
arerarelyheldaccountableforwhat theyhavereadindependently(HellerGreenleaf,2007).The
jarring exception is when college‐bound studentssit forthe collegeentranceexams.
Note: Weare not recommendingherethat teachers stopsupporting studentsin their reading, only that this support taperoffandthat onregularoccasionsstudentsbe held accountable andassessedon textsthey have not seen before and for which they have had nodirectpreparationfrom teachersprior to reading.As pointed out above, for most students, the only timein their K‐12 experience this takes place is on standardized tests.
Students havemoredifficulty reading expositorytexts than narrative(Bowen,1999;Duke,1998; Heller & Greenleaf, 2007; Shanahan & Shanahan, 2008; Snow, 2002),yet this material currently constitutesonly 7% to 15%ofinstructionaltext inelementary and middle school (Hoffman etal.,
1994; Moss & Newton, 2002; Yopp &Yopp,2006).In college, most,and for many studentsnearlyall, reading is expository (Achieve, 2007).
Theabovedata takeongreaterrelevancewithrecent findingsfromMcNamara andGraesser
(personalcommunication–Active Ingredients work) that narrativity is “the most prominent componentofreading ease.”In other words, the greater the portionofastudent’s totalreadingis narrative, thegreater the ease. Given the time constraints inevitably encountered inschool,the more narrative textread,the lessopportunity there is of encountering text thatiscomplex.
Expositorytext fromsocialstudies and science presents students with a different mix of rhetorical andsemanticchallengesrelativeto narrative (McNamara, GraesserLouwerse,2004). If students only engage in even successful reading ofnarrative, they willbe denied the opportunity to develop the abilities to overcome the challenges presented by expository texts.These genre challenges however,arerelatedtoeachother (McNamara, in press), thus eachgenre’ssetofchallenges will overlap tosome degree, and failure to learn fromone genre willlikely weakentheabilitytolearn fromthe others.
Successful learning from text and theconsequent developmentofcomprehension skills require the employmentof both strategies andknowledge tobuildamentalorsituation modelfromthegiven textbase. A high standard for coherence(ademandforthetext to make sense) then drives comprehension monitoring.This recruitsmanyof the same strategies that are called uponwhen comprehension breaks down (Perfetti, Landi, & Oakhill, 2004; Van den Broek, Risden, & Husebye‐ Hartman,1995;VandenBroeketal.,2001).If studentsengage in this process frequently, the use of strategiesbecomesmoreautomatic and habitual, and the strategies become skills (Afflerbach, Pearson, & Paris, 2008). If students do notemploy this process whenreadingexpositorytextthenthe resultant learning is superficialandshortlived (Kintsch,1998; Kintsch, in Tobias and Duffy, 2009).
Shallow reading from complex expositorytexts—skimmingforanswers, focusingonly on details,and failing to make inferencesinordertointegratedifferentpartsof thetext, toconnect to background
knowledge, and therefore forma rich situation model—will domore thanimpedestudents’ability to
read complex text. Itwill likely cause reading ability to deteriorate. Years of reading expositorytext
in this superficial way gives studentsthemessagethat expositorytextitself isshallow, thus reading it is an inevitably shallowand unrewarding exercise. The messenger, inthiscase, hasbeen slain.
In sum,the texts students are provided inschool to readK‐12are not of sufficientcomplexityto prepare themforcollegeorcareerreadiness. In addition, expositorytext, the overwhelmingly dominant form of careerand college reading, constitutesa minute portionof whatstudents areasked to read in pre‐ collegiate education.When it is read, itisover scaffolded byteachers,and taught superficially (read these pages, and find the answers). Fartoo many students are not only ill prepared cognitively forthe
demandsthistypeof textpresents; but are unawarethere iseven a problem, asidefrom how boringtheir
informationaltexts seem to be.Those quarter million studentswhoscoredat levels no better thanchance on the ACT likely had noidea how poorlythey did.About to leave high school,theywere blind‐sided by tasksthey could notperform ontextpassagesthey had never been equipped to encounter.
Given all of this, it is notsurprising that Heller and Greenleaf(2007), in findings that paralleled the ACT Between the Lines study, found that advanced literacyacross content areas(reading ofexpository, subject focused text), isthebest availablepredictorof students’abilityto succeed inintroductory college courses. Nor surprising that in a synthesisof nationalandinternational reportson adolescentliteracy prepared for the Vermont Principals Association (Liben unpublished Power Point, 2007), we found that all nine called forenhancements in content area reading.
What are Some Consequencesof soManyStudentsLeaving High SchoolUnableto ReadComplex
Text?
In addition tothe findings noted in theACT study:
20% of college freshman required remedial reading courses (NCES, 2004b). This is especially significant in lightof the fact that11stateshavealreadypassedlaws “preventing or discouraging” enrollment inthese classes in publicfour‐year institutions(Jenkins & Boswell, 2002).In fact, students who enroll in thesecourses are 41% more likely to dropout thanotherstudents(NCES,
2004A).
Only 30% ofstudents enrolled inany remedial reading course wenton toreceive a degree or certificate(NCES,2004).
Differences between students in top bracketsandall others, on measures suchas NAEPtest scores andAPcourses successfullycompleted,haveincreased,(National Pipeline Data, 2005).
Over 75% of surveyed students whodropped out indicatedthat difficulty withreadingwasamajor contributing factor(Lyon,2001).
According to the NationalAssessment of Adult Literacy(2003),15% ofadultsscored as proficient in
1992andonly13% in2003,a statisticallysignificantdifference in a decade.
The NationalEndowment forthe Arts, inReading at Risk (NEA, 2004), reportsthefollowing:
The percentage ofU.S. adults reading literaturedropped from54.0 in 1992 to 46.7 in 2002, a decrease of7.3 percentin a decade.
The percentage of adultsreading anybooklikewise dropped by 7 percent in the same period.
The rate ofdecline was inall demographic groups—women and men; whites, African Americans, and
Hispanics; all education levels; and all age groups.
Thoughallage groupsare readingless, the steepest decline by far is in the 18–24 and 25–34 age groups: 28% and 23%, respectively. In other words, the problemisnot onlygettingworse but doing
so at an accelerating rate.
The NEA study cites declines in readingbeginning in 1982 with 18‐ to 24‐year‐olds. Hayescites adecline in difficulty of textbeginning in 1962. Itis tempting to linkthese findings, as 18‐ to 24‐year‐olds in 1982 beganschoolfrom1969–1975andthe Hayes study cites text difficulty decreasing beginning in 1962.
Conclusion
Being able toreadcomplextextcritically with understanding and insight is essential forhigh achievementin collegeandtheworkplace(Achieve, 2007, ACT, 2006). Moreover, if students cannot read challenging texts with understanding, they will read less in general, extending the societal effectsthe Reading at Risk reportalready documented. Ifstudentscannot read complex expositorytext, they will likely turn to sources such as tweets, videos, podcasts,andsimilar media for information. Thesesources, while not without value, cannotcapturethenuances,subtlety,depth, or breadth ofideasdeveloped throughcomplex text. Consequently, these practicesarelikelyto lead toageneralimpoverishment of
knowledge, which in turnwill accelerate the decline in ability tocomprehend challengingtexts,leadingto still further declines. This patternhasadditionalseriousimplicationsfortheabilityof ourcitizens to meetthedemandsofparticipatingwiselyinafunctionaldemocracy within an increasingly complex world.
The ACT findings in relationto performance onthe science test bearrepeating.Theneedforscientific andtechnicalliteracy increasesyearly. Numerous “STEM” (Science Technology EngineeringMath) programs are beginning to dot the educational map.Yet only 5% of studentswhodidnot meetthe ACT reading benchmarkmetthescience benchmark.Science isaprocess, but it isalsoa body of knowledge. This body of knowledge is most efficientlyaccessed through its texts.Thiscannot be done withoutthe ability to comprehend complexexpositorytext.
A final thought:the problems notedhere are not“equal opportunity” in their impact. Studentsarrivingat schoolfrom less‐educated families are disproportionally represented in many of these statistics.The stakesare high regarding complex text for everyone, but they are even higher forstudentswhoare
largely disenfranchised from text prior to arriving at the schoolhouse door.
Bibliography:Why TextComplexityMatters
Achieve,2007 Closing theExpectations Gap.Washington, DC:Achieve, Inc. Afflerbach, P., Pearson, P., & Paris, S.G. (2008, February). Clarifying Differences
BetweenReading SkillsandReading Strategies.TheReading Teacher,61(5),364–373
American CollegeTesting (2006)“Reading BetweentheLines:Whatthe ACTRevealsaboutCollege Readiness in Reading” American College Testing, Iowa City:Iowa.
Biancarosa, G & Snow, C, (2004) Reading Next:AVisionforAction Research InMiddle andHighSchool
Literacy. Alliance for ExcellentEducation.New York:Carnegie Corporation.
Best Practices NationalGovernors Association(2005)Reading toAchieve:A Governor’sGuide ofAdolescent
Literacy.Wasington, DC:National Governor’sAssociation.
Bowen, G.M., & Roth, W.‐M. (1999, March). Do‐able questions,covariation, and graphical representation: Do we adequately prepare reservice scienceteachersto teach inquiry? Paper presented atthe annual conference of theNational Association for Researchin Science Teaching, Boston,MA.
Bowen, G.M., Roth, W.‐M., & McGinn, M.K. (1999). Interpretations of graphs by university biology students and practicingscientists: towardsasocial practice viewof scientific representation practices.JournalofResearch
in Science Teaching .
Bradshaw,T.,(2004)“Reading at Risk”Washington,DC:NationalEndowmentfor the Arts
Chall, J. (1977) An Analysis of Textbooks in Relation to Declining SATScores. EducationalTesting Service, Princeton,NJ.
Glenn B. Milewski, Daniel Johnsen,NancyGlazer, and Melvin Kubota 2005,Asurveytoevaluatethealignment of the new SAT Writing and Critical ReadingSections to
Curricular and Instructional PracticesCollege Board ResearchReportNo.2005‐1.ETSRR‐05‐07
Gioia, D., (2007)”ToRead or NottoRead:A Questionof National Consequence”Washington, DC:National
Endowment forthe Arts.
Hayes, D & Ward, M (1992)LearningfromTexts:Effectsof SimilarandDissimilar Features ofAnalogiesin Study Guides. Paper presentedattheAnnualMeeting oftheNationalReadingConference(42nd,San Antonio, TX, December 2‐5, 1992). Education Trust.
Hayes, D., Wolfer, L. & Wolfe, M. (1996) SchoolbookSimplificationandItsRelationtotheDeclineinSAT‐ Verbal Scores.Scholastic AptitudeTest
Heller, R & Greenleaf,C (2007)LiteracyInstructionin the ContentAreas: Getting tothe CoreofMiddle and High
SchoolImprovement. Washington, DC: Alliance for ExcellentEducation.
Hiebert, E.H., & Kamil, M.L. (Eds.) (2005). Teachingand learning vocabulary:Bringing research to practice. Mahwah, NJ:Lawrence ErlbaumAssociates.
Hiebert, E. (Ed.),Reading more, reading better: Are American studentsreading enoughofthe right stuff? New
York:GuilfordPublications,2009.
Hoffman, J. et al., (1994)Building aCulture ofTrust.Journalof SchoolLeadership,v4n5p484‐501Sep1994. Jenkins, D & Boswell,K.( 2002). State Policies on Community CollegeRemedialEducation: Findings froma
National Survey. Denver: Education Commissionof theStates.
Kays, J., & Duke, N. K. (1998). Getting studentsinto information books.TeachingPreK8,
29(2),52‐54.
Kintsch, W. (1998) Comprehension: A paradigm for cognition. New York: CambridgeUniversity Press. Kintsch, W. (2009) Learning and constructivism. In S. Tobias & T.M. Duffy(Eds.) Constructivist Instruction:
Successorfailure?NewYork:Routledge.Pp.223‐241
Lyon, G. R. (2002). TestimoniestoCongress1997–2002.Covington, LA:CenterforDevelopmentLearning.
Magliano, J. P.,Millis, K. K.,Ozuru, Y., & McNamara, D.S. (2007).Amultidimensionalframework toevaluate reading assessment tools. In D.S.McNamara (Ed.),Reading comprehension strategies: Theories, interventions, and technologies(pp. 107136). Mahwah, NJ: Erlbaum
McNamara,D, Graesser, A, and Louwerse:Sources of Text Difficulty: Acrossthe Agesand GenresMemphis: University of Memphis
Moss & Newton, 2002; An Examinationof theInformationalTextGenre in Basal Readers. ReadingPsychology, v23 n1 p1‐13Jan‐Mar 2002.
National Assessment ofAdult Literacy(2003) Washingon, DC:NationalCenterfor EducationStatistics. National Summary EducationalPipelineData Profile,AchieveInc. February 2005
Perfetti, C. A., Landi, N., & Oakhill, J. (2005). The acquisition of readingcomprehension skill. In M. J.Snowling
& C.Hulme (Eds.),Thescienceofreading:Ahandbook(pp. 227‐247). Oxford: Blackwell.
Shanahan,Timothy;Shanahan,Cynthia.(2008) TeachingDisciplinary Literacy toAdolescents:Rethinking
Content‐AreaLiteracyHarvardEducationalReview(0017‐8055)Spr 2008. Vol.78,Iss.1;p.40‐59Snow,2002
Stenner, A J Koons,Hand Swartz,CW:TextComplexity,the TextComplexity Continuum,
and Developing Expertise in Reading.ResearchTrianglePark,North Carolina:MetaMetrics
U. S. Department of Education. National Center for Educational Statistics. (2004b). Tuition costsof colleges and universities.
van den Broek, P., Risden, K. &Husebye‐Hartmann, E. (1995). The Roleof Readers' Standards for Coherence
in the Generation of Inferences during Reading. In Lorch, R. F.and O'Brien, E. J. (eds.) <i>Sourcesof Coherence inReading</i>,353‐373.Lawrence Erlbaum: Hillsdale, NJ
P vanDen Broek, R.F. Lorsch, Jr, T. Linderholm,M.Gustafson.(2001) The effects of readers' goals on inference generation and memoryfortextsMemCognit 200129:1081‐1087
Williamson,GAligning the Journey With the Destination: A ModelforK‐16ReadingStandards.North
Carolina: Metametrix.
WilliamsonGL 2006Aligning theJourney With the Destination A Model for K‐6 Reading Standards, AWhite
PaperfromtheLexileFramework forReading
Yopp & Yopp, 2006 Primary StudentsandInformationalTexts Science and Children, v44 n3 p22‐25Nov 2006.
4 pp. (Peer Reviewed Journal)