Chat 14
1. Notes 9d: One-way ANCOVA
1.1 Purpose
(a) Basics
Analysis of covariance = ANCOVA
ANCOVA is similar to ANOVA in that we can
· has an IV = qualitative and DV = quantitative;
· determine if group means appear to differ; and
· make pairwise comparisons to determine which group means differ.
ANCOVA is also similar to regression because ANCOVA
· has IV = quantitative and DV = quantitative;
· provides partial effects estimates (i.e., controlling for an IV) that accounts for other IVs;
· and provides predicted values of DV taking into account all other IVs.
ANCOVA ---like ANOVA, regression, t-test, and correlation --- is part of the general linear model, so it is mathematically related to each of the statistical procedures cited.
(b) Type of Study and Statistical Adjustments
Research Scenarios
ANCOVA is suitable when one wishes to compare two or more groups while also taking into account possible confounding variables.
A common research scenario is to compare two (or more) groups of students to learn if a treatment produces better results than a control situation after taking into account one or more covariates, for example
· maybe comparing reading achievement (DV) between those who use Accelerated Reader and those who don’t (qual. IV) after first considering their pretest scores or levels of motivation to read (covariate, quant. IV), or
· comparing achievement (DV) between those using supplemental instructional such as cooperative learning vs. peer tutoring (qual. IV) after controlling for IQ levels or self-efficacy levels (covariate, quant. IV).
While both examples above have only two groups to compare, these could easily be extended to more groups.
Example
We wish to learn whether a recently purchases tutorial program helps students with science scores. The tutorial program is implemented in Class A while Class B does not use the tutorial program.
Mean scores on a common posttest to measure science achievement are reported below for the two classes.
Tutorial Use / Posttest Score MeanClass A / yes / 88.20
Class B / no / 74.20
Question
What is the possible cause of this 88.20 - 74.20 = 14.00 mean difference between the two classes?
Answer
The difference could result from the tutorial program, or could result from pre-existing differences between the two classes in terms of ability, motivation, or something else that could confound achievement interpretation.
Question
Suppose we learn of the following IQ differences between classes, now what explains the 14 point difference in posttest scores?
Tutorial Use / IQ Mean / Posttest Score MeanClass A / yes / 103.60 / 88.20
Class B / no / 97.60 / 74.20
Answer
The difference is likely the result of both the tutorial program presence in Class A but not in Class B, and the IQ difference.
Adjustments
ANCOVA can be helpful for providing statistical adjustments that allow us to estimate what difference may exist if both groups had similar covariate scores.
For example, if both groups had a mean score of 100.60 on IQ, what would be the predicted (estimated) mean scores on the posttest for both classes?
Tutorial Use / IQ Mean / (Observed)Posttest Score Mean / Adjusted Posttest Score Mean (assumes IQ = 100.6 in both classes)
Class A / Yes / 103.60 / 88.20 / 86.01
Class B / No / 97.60 / 74.20 / 76.39
Note. Adjusted scores based upon IQ mean of 100.60.
Study the difference between the observed posttest mean and the adjusted posttest mean. What do you see?
Answer
· Adjusted scores are based upon a common mean IQ of 100.6 (i.e., what would be the mean posttest if both classes had an average IQ of 100.6)
· The class with the lower IQ (97.60) had their mean posttest score adjusted upward (74.20 to 76.39)
· The class with the higher IQ (103.60) had their mean posttest score adjusted downward (88.20 to 86.01)
· The difference in adjusted posttest score means (86.01-76.39 = 9.62) is less than the original difference of 14.00, but there is still a difference
Note
Observe the direction of adjusted on the DV. Note that the group will higher IQ scores was adjusted downward (since we would expect their DV score to be higher) and the group with lower IQ scores was adjusted upward (since they started lower, they get bumped up). This is the equating effect of ANCOVA, the statistical controlling effect that allows us to make statistical estimates of group equivalency. (Also note, the direction of adjustment described above assumes a positive correlation between IQ and posttest science scores.)
Question
The difference in adjusted posttest score means (86.01-76.39 = 9.62) is less than the original difference of 14.00, but there is still a difference --- what does this tell us?
Answer
The difference between classes shrank from 14.00 to 9.62 when IQ was taken into account. This tells us that
· the ANCOVA model estimates that if both classes had the same IQ level, then tutorial program may produce an achievement difference of 9.62, and that
· the ANCOVA model estimates that 4.38 (14.00 - 9.62 = 4.38) points of the original mean difference is due to IQ differences between the classes. So IQ contributed to the original 14 point difference, but was not the complete reason for that difference.
Question
What would be the likely adjusted posttest score means if the IQ means were the same for both groups in the original data? In which direction would the posttest mean scores be adjusted for each class?
Tutorial Use / IQ Mean / ObservedPosttest Score Mean / Adjusted Posttest Score Mean
Class A / Yes / 99.00 / 88.20 / ?
Class B / No / 99.00 / 74.20 / ?
Answer
If both classes start with equal IQ means, then there will be no adjusted to the DV.
Tutorial Use / IQ Mean / ObservedPosttest Score Mean / Adjusted Posttest Score Mean
Class A / Yes / 99.00 / 88.20 / 88.20
Class B / No / 99.00 / 74.20 / 74.20
Note
Why no adjustment if IQ starts the same for both groups? There is no need for adjustment because variation observed in posttest scores mean differences between classes cannot be attributed to mean differences in IQ across classes. Consider this example of sex differences in classes.
Tutorial Use / Sex / ObservedPosttest Score Mean
Class A / Yes / All female / 88.20
Class B / No / All female / 74.20
Can student sex composition within each class explain the 14 point difference in posttest scores?
Answer
Sex is a constant and cannot vary or covary with posttest scores, so sex is controlled and cannot influence posttest. Sex cannot explain the 14 point difference in mean scores.
If we had both males and females in the study, how can we equate the two classes in terms of sex composition? Assume n = 100 with 60 females and 40 males.
Sex Composition / Posttest Score MeanClass A / ? / 88.20
Class B / ? / 74.20
To balance the groups by sex, this distribution would work to eliminate, or control, sex as a possible confounding variable.
Answer
Since there are 60 females, we put half in Class A and half in Class B. Also equally divide males by class.
Sex Composition / Observed Posttest Score MeanClass A / 30 F, 20 M / 88.20
Class B / 30 F, 20 M / 74.20
In Summary – ANCOVA provides statistical adjustment to DV means when covariates show differences between groups. If the covariate is positively related to the DV, then the adjustments will be upward for the group starting lower on the covariate, and downward for the groups starting higher on the covariate. Thus, ANCOVA provides a statistical means of equating groups on possible confounding variables (included as covariates in ANCOVA) and estimating group differences when covariates are held constant for each group.
ANCOVA limitation – statistical adjustment cannot always be trusted (folks often place too much faith in statistical adjustment); better to control by design if possible (e.g., designing study to have a balanced sex composition in both classes).
(c) Graphical Display of Statistical Adjustments
Recall that regression lines can be used to make predictions.
· Someone with an IQ = 90 would be predicted to score what on the posttest for Class B?
· IQ = 105, what is predicted Posttest for those in Class A?
Question
What would be the predicted scores for both classes if IQ = 95 in both classes?
Answer
About 82.5 for Class A, and 73 for Class B.
Question
What would be the predicted scores for both classes if IQ = 105 in both classes?
Answer
About 89 for Class A and 79.5 for Class B.
Question
Using this model, is there a differential benefit of the tutorial depending upon one’s IQ level? That is, do students with IQs below 95 benefit more, or less, from using the tutorial than students with an IQ of 105 or greater?
How do we answer this?
To answer, calculate the mean difference between Classes A and B for an IQ of 95, then again for IQ of 105. If the mean difference in posttest scores is about the same, then there is no differential benefit of the tutorial due to IQ.
[Draw line and move up and down range of IQ to show constant tutorial difference]
Note constant difference across range of IQ between both classes. So there is not a differential benefit of the tutorial for different IQ ranges.
Simple explanation for meaning of this: if asked how much benefit the tutorial offers, you can say about 9 points for all students.
Below is example in which differences on the DV vary across levels of the covariate, so the differences are not constant.
1.2 SPSS Commands
Use these data
http://www.bwgriffin.com/gsu/courses/edur8131/data/chat_14_ancova_example_1.sav
IQ / Posttest / Class94 / 77 / 1
97 / 86 / 1
101 / 90 / 1
99 / 85 / 1
103 / 86 / 1
106 / 87 / 1
108 / 94 / 1
104 / 90 / 1
111 / 95 / 1
113 / 92 / 1
88 / 63 / 0
91 / 72 / 0
95 / 76 / 0
93 / 71 / 0
97 / 72 / 0
100 / 73 / 0
102 / 80 / 0
98 / 76 / 0
105 / 81 / 0
107 / 78 / 0
1. Analyze, General Linear Model, Univariate
2. Variables:
· DV (Posttest) to “Dependent Variable” box
· Factor (Class) to “Fixed Factor(s)” box
· Covariate (IQ) to “Covariate(s)” box
3. Options:
· Move “Class” to “Display Means for” box
· Select “Compare Main Effects” (and choose Bonferroni if more than 2 groups)
· Select “Descriptive Statistics”
See images below
1. Analyze, General Linear Model, Univariate
2. Variables:
· DV (Posttest) to “Dependent Variable” box
· Factor (Class) to “Fixed Factor(s)” box
· Covariate (IQ) to “Covariate(s)” box
3. Options:
· Move “Class” to “Display Means for” box
· Select “Compare Main Effects” (and choose Bonferroni if more than 2 groups)
· Select “Descriptive Statistics”
SPSS Results
Descriptive StatisticsDependent Variable: Posttest
Class / Mean / Std. Deviation / N
.00 / 74.2000 / 5.24510 / 10
1.00 / 88.2000 / 5.24510 / 10
Total / 81.2000 / 8.81148 / 20
Note observed, unadjusted means above. Note that Class 1 = A, Class 0 = B.
Tests of Between-Subjects EffectsDependent Variable: Posttest
Source / Type III Sum of Squares / df / Mean Square / F / Sig.
Corrected Model / 1334.704a / 2 / 667.352 / 80.750 / .000
Intercept / 3.899 / 1 / 3.899 / .472 / .501
IQ / 354.704 / 1 / 354.704 / 42.919 / .000
Class / 363.928 / 1 / 363.928 / 44.035 / .000
Error / 140.496 / 17 / 8.264
Total / 133344.000 / 20
Corrected Total / 1475.200 / 19
a. R Squared = .905 (Adjusted R Squared = .894)
The above table shows the ANOVA summary calculations. The rows in green are those numbers one would report.
Adjusted means can be found in the table below. SPSS calls adjusted means “Estimated Marginal Means.” --
Estimated = predicted
marginal = predicted across all variables from regression equation (ANCOVA equation)
Predicted Means using the ANCOVA equation
Estimated Marginal Means
EstimatesDependent Variable: Posttest
Class / Mean / Std. Error / 95% Confidence Interval
Lower Bound / Upper Bound
.00 / 76.391a / .969 / 74.348 / 78.435
1.00 / 86.009a / .969 / 83.965 / 88.052
a. Covariates appearing in the model are evaluated at the following values: IQ = 100.6000.
Chat lasted about 1:10 (spring 2014)
Next chat session
1. Extra example with 3 groups (car data?)
2. APA style presentation
3. Possibly learn to test for interaction
Example
Is there a difference in overall mean MPG among country/area of origin of cars: American, European, and Japanese.
http://www.bwgriffin.com/gsu/courses/edur8132/data/cars_with_dummies.sav