SW 981 - LECTURE NOTES FOR WEEK 7
Multiple Regression Analysis
General MR Model
Y = a + b1X1 + b2X2 + ... + bkXk + e
Y'= a + b1X1 + b2X2 + ... + bkXk
As with simple regression:
SStotal = SSregression + SSresidual
R2 = SSreg/SSt = proportion of variance in Y accounted for
by the independent variables (X's)
Note: R2 = R2Y.X1X2 = R2Y.12
R = square root of R2 = Multiple correlation coefficient = Coefficient of determination
However, the regression coefficients (b's) now have a different interpretation.
b = partial (unstandardized) regression coefficient
Indicates the expected change in Y associated with a unit change in X while controlling for, or holding constant, the effects of the other, independent variables.
Relative Contribution of the X's
r2y1 + r2y2 - 2ry1ry2r12
R2y.12 =
1 - r212
if r12 = 0, then R2y.12 = r2y1 + r2y2
If X's are correlated the interpretation of "relative contribution" is complicated. The correlation of the independent variables is referred to as "multicollinearity".
The order of entry of variables into the regression equation is important in determining the contribution of individual variables. However, the final b's and R2 (when all variables are entered) will be the same regardless of the order in which the independent variables are entered.
Standardized and Unstandardized Regression Coefficients
b = unstandardized coefficient ß = standardized coefficient
Note: New use of ß as standardized coefficient, not population parameter, not Type II error probability.
ß1 = (ry1 - ry2r12)/(1 - r212)
ß2 = (ry2 - ry1r12)/(1 - r212)
If r12 = 0 , then
ß1 = ry1 and ß2 = ry2
Tests of Significance
Test for R2:
SSreg/dfreg SSreg/k
F = =
SSres/dfres SSres/(N-k-1)
R2 / k
=
(1-R2)/(N-k-1)
= Overall test of the regression (all b's)
Test of b's:
b/sb = t b + t ( /2, df)sb
Test of Change in R2:
(R2y.12...k1 - R2y.12...k2)/(k1-k2)
F =
(1-R2y.12...k1)/(N-k1-1) where k2 < k1
Testing the increment in proportion of variance accounted for by a single variable when entered last (i.e., change in R2 is equivalent to the test of the b associated with the variable in the equation with all variables entered. The increment may be considerably different from the proportion of variance it accounts for by itself.
Relative Importance of X's
Unstandardized coefficients (b's) are affected by the scale of measurement. Comparisons of the size of the b's cannot be used to judge substantive or statistical significance.
Standardized coefficent (ß's) can vary as a function of the variability of X. Comparisons across samples not safe.
Increment in R2 depends on order of entry when X's are correlated. The semipartial correlation provides a measure of increment in R2:
SP2= R2y.12...k - R2y.12...k-1
We can use the partial F test (see above) to test the statistical significance of the increment in R2. When the increment is associated with only one X, this F is equivalent to the t-test of the significance of the corresponding b coefficient.
1