The Scholastic Aptitude Test (SAT) Has Long Been a Measuring Stick Used by Colleges To

The Impact of Family Income on SAT Scores

Bryan LaBrecque

PADM 9040 Research methods in Public administration

April 9, 2012

Br. Baracskay
Introduction

The Scholastic Aptitude Test (SAT) has long been a measuring stick used by colleges to aid in the selection process of prospective students. In an annual right of spring (and often summer and fall), high school students anxiously await their test scores. Colleges use these scores in several ways: 1) to sift through the thousands of applicants, 2) to predict prospective students’ first year grades, 3) to select scholarship awards and other types of financial assistance, and 4) as means of academic pride between colleges.

The SAT clearly carries a great deal of “weight” in the academic world, but is also not without its detractors or its controversy. In recent years, naysayers have identified several biases that they believe exist in the system, which render the test scores less worthy. Issues such as racial composition, cultural biases, age distributions, wealth, and the availability of quality secondary education, have all been mentioned as reasons to either eliminate the SAT altogether or at minimum, reduce its weighting in the selection process. While the foundation of the SAT exam has remained relatively constant, the test has received several “makeovers” over the years, in an attempt to quell some of the controversy.

On might ask why this topic is important. Standardized tests have been in existence for a hundred years or so and impact the students and their schooling from kindergarten through higher education. Logically, there must be a reasonable amount of benefit to the metrics associated with them. What makes the SAT such a polarizing test is the impact it has on a young person’s future earnings potential. Scores of articles and research papers have dedicated themselves to understanding the value of a college degree as it relates to wage compensation. This paper will not attempt to supplement that research but will use their outcome – college degrees having a strong positive association with earnings potential – as the motivation for better understanding some key drivers for SAT performance results. Clearly, students with higher SAT scores will have more opportunities to attend prestigious colleges and universities, providing them with higher earnings potential.

The question under consideration in this research paper is whether or not higher scores on the SAT can “purchased” and if so, what ramifications are associated with this result. In other words, this paper will investigate the many variables associated with the SAT and attempt to isolate the impact of wealth – or family income – on those scores. The analysis uses secondary data, available from a number of sources, and utilizes statistically based research methodology to attempt to project family income as a cause of successful testing.

Before embarking on the analysis itself, several reference articles and contemporary research was reviewed to better understand other’s views and to take advantage of work previously accomplished and knowledge previously gained on the topic. What follows is a brief literature review.

Literature review

The subject of SAT score bias has received as much research attention as there are theories as to why the scores differ from student to student. Among the more prevalent research regarding the Scholastic Aptitude Test involves comparative studies. These range over a myriad of variables including racial bias, test preparation bias, gender bias, geographic bias, testing bias and income bias (the principle topic of this paper). Since each of these variables may or may not have an impact on the income study, a better understanding of each was needed to assure that their impact was not overlooked. A review of a select few of these studies is helpful in laying the groundwork for this study.

Among the more frequent studies, geographic impact – specifically state to state comparison studies – has garnered a great deal of interest, especially from the both the media and state politicians who are each jockeying for good news/bad news stories. Authors Wainer, Holland, Swinton and Wang produced a compelling analysis of state-by-state SAT comparisons using the 1984 and 1985 “State Education Statistics” released by then Secretary of Education Bell. This table compared the 50 states on several interesting and related educational variables. According to the authors, the very nature of the data points be exhibited side-by-side, compels one to begin comparing numbers – especially SAT scores v. many of the aforementioned variables. The authors contend that this compulsion could lead to some very faulty conclusions. In particular, Wainer et al, are concerned that the impact of participation rates both between states and within states is glossed over. The primary concern of this missing input is the self-selection error associated with the data. Ultimately, the authors conclude that while “the correlation between income and SAT Verbal is substantial”, the self selection process – “those with both high income and high SAT capability tend to take the test…” (Wainer et al, 1985, 307)” - skews the data.

Stephan F. Gohmann expands on the issues of comparing geographic SAT scores in his journal article. He also expresses considerable concern over the research methodology and the impact of selection bias on the data set. He suggests the need to inject a “selection” correction to consider the impact of self-selection. In his article, he produced a table of SAT scores, by state, with their predicted rankings without selection correction. He then updated the equation with one of several forms of corrections, resulting in extremely different residual rankings.

“For example, Alabama’s ranking fell from its observed value of 26 to 29 when the standard estimating equation residual is used. When the selection equation is used, Alabama is ranked sixth.” (Gohmann, 1988, 145)

Gohmann strongly suggests that participation rate and the bias of self-selection are keys to predicting SAT scores.

Another instrumental variable in SAT scores results from familiarization with the test itself. This situation arises when students have taken the PSAT previously or have participated in public or private SAT tutoring, or even via the SAT Test-taking Booklet itself. Authors Powers and Alderman (who, as members of the Educational Testing Service, have authored several articles regarding standardized testing) tested the concept of test familiarization on SAT test scores to help understand the impact of preparation classes. They concluded that while familiarization with a test such as the SAT assists students both academically and emotionally, preparation courses do not significantly impact test scores.

“In summary, this study strongly suggests that the opportunity for self-study of additional, more extensive information about a test like the SAT is welcomed by test candidates, but that any additional benefits beyond those resulting from previous exposure to the test and from familiarization information available in test bulletins is not likely to effect the test scores of typical candidates.” (Alderman and Powers, 1983 , 79).

Alderman and Powers team up again in another article slightly more pertinent to the topic of this paper, concerning themselves with the impact of special preparation for the SAT – a potential proxy for family income. In that article, the authors studied students at eight secondary schools, assigned some to a treatment group (special preparation for the SAT) and others to a control group (delayed access to the same preparation), in an attempt to determine if the additional preparation impacted scores significantly. In this study, Alderman and Powers concluded that “regular school practices or characteristics exert a stronger influence on test scores than does special preparation for the test.” (Alderman and Powers, 1980, 250).

Finally, like many other researchers, Rebecca Zwick asserts that test scores are positively associated with socioeconomic factors, but through her own research determines that wealth is also a measure of most other previous educational achievement. In that sense, the SAT is not any more biased than other measures of academic capability.

“This evidence fails to support the claim that score gaps result mainly from the inclusion of esoteric test content that is more familiar to upper-crust test-takers or from inequalities in access to coaching services.” (Zwick, 2002, 310)

Research Design and Hypothesis

A fairly standard statistical methodology will be utilized in the research for this assessment. As the intent of the study is to determine the strength of relationship between SAT scores and household income, the mean scores and their associated standard deviations will be computed for several increasing income levels, in a given year. The mean was selected because the data follows a fairly normal distribution with very little skew. The Standard Deviation is computed for each income level to determine the variability of data points. One point of emphasis here is that since the secondary data being used represents the population of data, random sampling errors do not result, significantly simplifying the analysis.

Variables

In building the analysis, it is critical to first understand the difference between the independent variable(s) and the dependent variable. In our case, SAT scores (both Math and Verbal) represent our dependent variable. Several independent variables, when applied appropriately will result in the specific scores. For this study, our independent variables are:

·  Family Income

·  High School GPA

·  Parental Education level

High School GPA – A student’s high school GPA represents his or her grasp of specific subject knowledge and is an indicator of work ethic, organization skills and learning capacity. As such it is reasonable to assume that this metric is a contributor to the ability of a student to achieve scores on the SAT.

Family Income – Many experts believe that household income affects students’ ability to score on the SAT. Higher income often means that the student lives in a more affluent area which equally often results in better schools and resources. In addition, higher incomes usually result in more disposable income to be used for computers, internet access, tutoring sessions and SAT preparation courses.

Parental Education Levels – Parents with college degrees tend to focus their children on the need for college and as a result may create an environment more conducive to success on the SAT.

Data Manipulation

The data itself represents the population of SAT takers and is presented as a mean over a range of family incomes. Since averages were used and there is no sampling error (since there is no sampling done), the data was manipulated to produce a single set of discrete data. First, the mean of test scores of SAT Verbal and SAT Math were combined and the resulting figures were paired with the mid-point of the family income level. In addition, since we used the population of data versus random sampling, the need for null hypotheses is eliminated because there are no sampling errors to be concerned over. The means of the data represent the true differences.

Research Hypothesis

Since we are attempting to determine our relationship strength between family income and SAT scores we will set our research hypothesis thusly:

“Family income is a key and significant driver of higher test scores and has a distinctly positive relationship.”

Regression

From there simple regression will be run as a foundational predictor. This will produce a best-fit line through our data set and allow the researcher to make a broad projection of SAT outcomes based on our equation. While a multiple regression analysis would provide a more refined association, the complexity of the analysis goes beyond the preliminary nature of this paper. The use of multiple regression and control variables will be discussed in the assessment of the analysis results and possible next steps.

A simple regression of our data set will provide us with the coefficient of the independent variable (slope) as well as the constant (y-intercept) of our best-fit linear equation and

Correlation

Once the independent variable and dependent variable are established, the methodology continues by determining the correlation between the variables. Correlation is determined by computing the Pearson R. The Pearson product-moment correlation coefficient, as it is properly known, will range from -1.00 to 1.00. The closer the Pearson R is to -1.00 or 1.00, the stronger the relationship between the two variables, with a negative coefficient indicating an inverse relationship and a positive coefficient denoting a direct relationship.

The association between the two variables is further refined by determining the Coefficient of Determination. The Coefficient of Determination is computed by squaring the Pearson R value and represents a percentage of how much variance on one variable is accounted for by the variance on another. In our case, the Coefficient of Determination will help us understand the percent of variance on SAT scores which is accounted for by the variance on family income.

Practical Significance

Finally, once we have determined the association and correlation strength, we will assess the overall practical significance of the variances.

Analysis of Data

Data

The data used to assess our hypothesis came from the “2009 College-Bound Seniors Total Group Profile Report” as provided by The College Board. As previously discussed the data, being categorical in nature, was manipulated to create quantitative data sets more useful to the researcher. The data, in its delivered form, is shown in charts 1 and 2 below.

Chart 1 – Mean of SAT–Math scores shown as a function of ordinal levels of family income

Chart 2 – Mean of SAT–Verbal scores shown as a function of ordinal levels of family income

The data shown above are the mean scores within each income level computed from the population of test-takers in 2009. The total number of participants was 1,530,128 students.

Scatter Plot

Chart 3 below, represents a scatter plot graph of the combine SAT scores paired with the mid-point of the family income ordinal level.

Chart 3 – Scatter Plot of Mean of SAT–Combined scores shown as a function of the mid-point of the ordinal levels of family income