11/27/6 Estimating and Comparing Means 13
Estimating and Comparing Means
SEM has been primarily used as a set of tools to investigate relationships. As we know from our study of regression analysis, relationship can be used to compare means. So, since SEM is a generalization of regression techniques, SEM can also be used to compare means through relationships.
The covariates or group-coding-variables or relationships method of comparing means
Model: G:\MdbT\P595\AmosMeansEstimated\IndtExample
Data: G:\ MdbT\P595\AmosMeansEstimated\IndtExample.sav
Simplest Example: Comparing two independent means. Mean performance of Men vs. Women in an intro statistics course. Men are Group=1, Women are Group=2.
Data matrix:
The SEM
Raw coefficients
Standardized coefficients
FYI- The critical ratio for the regression coefficient is -.55, p = .583. So performance is not related to Gender. That is, there is no significant difference between Group means.
This method is called the “covariates” method because the variable representing group membership is treated as a predictor or “covariate” in SEM terminology.
The Separate Groups method of comparing group means.
Model: G:\MdbT\P595\AmosMeansEstimated\IndtExampleSeparateMeans
Data: G:\ MdbT\P595\AmosMeansEstimated\IndtExample.sav
Overview . . .
1) A multiple-group model is created, in which a path diagram is specified separately for each group.
2) A general model is applied in which a separate mean value for each group is estimated.
3) A special model, in which the same mean value for each group is estimated.
4) The chi-square difference test between the general and special models is used to test the significance of the difference. If the chi-square is significant, that means that the general model allowing separate means fit significantly better than the special model that required means to be equal, leading to the conclusions that the means must differ significantly.
Independent Groups t Example: A Two group model comparing Performance between the two groups.
Separate groups comparisons can be performed in all popular CFA/SEM programs. Alas, the details of performing the separate groups comparison are specific to the program. So, while you must learn the specifics of Amos for this class, please focus on the big picture so that if you end up with a different program, you can apply the big picture steps using the details of that different program.
1. This first part of the process tells Amos that we’re going to create a multi-group model and gives some details of that process.
a. With a blank Amos Input window, double-click on “Group number 1”.
This opens a “Manage Groups” dialog box.
b. Type the name of the first group in the box.
c. For each subsequent group, click on “New” and type the name of the group.
d. When done specifying group names, click on the “Close” button.
2. Connect Amos to the data and identify the groups in the data.
a. File -> Data Files . . . -> File Name. (Follow the steps below carefully.)
b. Click on [File Name] and identify the data file.
c. Click on [Grouping Variable] button and click on the name of the variable identifying the groups.
d. Click on the Group Value button and identify the value for the first group.
e. Highlight each subsequent group in the Data Files window, and repeat a, b, and c.
3. Draw the path diagram for each group.
By default, Amos will make the path diagram will be the same for each group, although the parameters of the objects in the diagram, in this case, the means, will not be the same.
a. Highlight Males.
b. Draw the path diagram. For an independent samples t, the path diagram will simply be a single rectangle. (Note that this is different from the covariates method, for which we had to tell Amos the name of a covariate. In this method, the covariate is not specified, but is implied by the different group models.)
When you draw a path diagram for one group, all characteristics of the path diagram are copied to the other group(s). Highlight Females. Note that the path diagram is the same for both.
4. Make sure that the program is set up to estimate means.
a. View -> Analysis Properties -> Estimation -> Estimate Means and Intercepts.
Amos does NOT automatically estimate means. We’re telling to estimate them in this setp.
5. Specify that different means are to be estimated for each group.
a. Right-click on an object -> Object Properties.
As mentioned above, Amos automatically copies all characteristics of a path diagram to all groups.
Here were overriding that option for the means. We’ll allow the means to be different in each group.
b. click on the “Parameters” tab and uncheck the [All Groups] button. VERY IMPORTANT.
c. Highlight Males.
d. Enter a name for the mean of the Male group, e.g., malemean, the Mean field of the Object Properties Parameters dialog.
e. Highlight Females, then enter a name for the mean of the Female group, e.g. femmean.
The path diagram should look like the following
:
6. Save the file.
7. Tell the program to apply two models – one with separate means and one with equal means.
a. Double-click on “Default Model”.
b. Type “Means Separate” in the Model Name field.
`Since the means have separate names, Amos will estimate them separately for each group by default.
c. Click on the New button, then type Means Equal in the Model Name button.
For this model, we want the two means to be estimated as identical values. So . . .
d. Type malemean = femmean in the Parameter Constraints field.
e. Click on the “Close” button.
We’ve now created two models – one in which the means are estimated separately and one in which the means are estimated as having the same value.
8. Run the models.
Output from Amos
1. First, the mean separate model
2. Now the means equal model.
The chi-square is not significant, indicating that the there is no significant difference in goodness-of-fit between the two models. From this we can infer that the means are NOT significantly different.
From the Text output, Chi-square = .260, p = .610.
Note that the test was conducted allowing the variances to be unequal.
Why, Why, Why? Why go to all this trouble to compare two means?
The Separate means method was much more difficult to carry out than the “Covariates” method.
Why would anyone do it?
1. The separate means method is much more flexible. It can be conducted assuming variances are equal or assuming they’re unequal.
2. The separate means method extends quite easily (once you get the hang of specifying the models) to multiple groups.
Example 2: Analysis of variance. (This is designed as an in-class exercise.)
Comparing mean performance in P511 for 3 years.
SPSS Output
So, there were no significant differences in mean performance across the 3 years.
The Amos output from a multigroup model, set up in the same way as the above independent groups t-test model.
Amos chi-square for the model in which the three group means were constrained to be equal was .0.1 with df=2, p > .05.
The nonsignificant of the chi-square says that the model with means constrained to equality fit essentially as well as a model with means estimated separately.
From this we can infer that there are NOT significant differences between the means.
Example 3. Extending the multiple groups conceptualization to comparison of correlations.
This example is from research we’ve been conducting on respondent inconsistency – the tendency of persons to give different self-reports to items from the same personality dimension. Some persons are quite consistent from item to item within the same personality dimension. Others are more inconsistent, giving different responses to items even though all the items represent the same personality dimension.
Although we’d expect some differences in responses because, after all, the items are different, we’ve found that there are reliable differences in the amount of inconsistency shown by people. If a person is inconsistent in responding to one personality questionnaire, he/she’ll be inconsistent in respond to other personality scales. Inconsistency in self report appears to be a personality characteristic, one that cuts across different questionnaires. (Need a MS thesis on whether or not it cuts across time periods.)
Comparing correlations across groups.
A. Comparing convergent validities between corresponding measures of the Big 5
From Biderman, M. D., & Reddock, C M. (2012). The relationship of scale reliability and validity to respondent inconsistency. Personality and Individual Differences, 52, 647-651.
We formed three groups based on inconsistency of responses to the original IPIP 50-item questionnaire.
We then created an Amos model which assessed the convergent validity between corresponding Big Five dimensions as measured by two other Big Five questionnaires – the IPIP “Other” 50-item questionnaire and the Thompson minimarkers.
We expected convergent validity to be greatest for the most consistent respondents and least for most Inconsistent respondents.
We used the Amos models to assess the significance of the differences in inconsistency across the three groups.
The Amos file is “MDBR\1BalancedScale\Inconsistency II\Inconsistency II Amos\Table 5 convergent validity Revised 3.amw”
The Input model
The parameters circled in red are the convergent validities. It was assumed that they would be largest for inconsistency group 1 (least inconsistent respondents) and smallest for group 3.
The names of the convergent validity parameters were different for each group.
Group 1: covee1 covaa1 covcc1 covss1 covoo1
Group 2: covee2 covaa2 covcc2 covss2 covoo2
Group 3: covee3 covaa3 covcc3 covss3 covoo3
The constrained model was
Group 1 results. Mean of convergent validities = .784.
Group 2 results. Mean of convergent validities = .720. (The red’d double-headed arrow is of no consequence.)
Group 3 results. Mean of convergent validities = .680.
Convergent Validities constrained model. Chi-square difference (10 df) = 20.81. p < .05.
Since the chi-square p-value was less than .05 (gotten from the text output), this indicates that the model which constrained the convergent validities to be equal fit significantly worse than the model which let the convergent validities be separate for each group.
From this we can infer that the convergent validities are less for the more inconsistent respondents.
So, if you have developed a new questionnaire and want to show that its measures exhibit high convergent validity with existing measures of the same constructs, make sure you assess convergent validity using consistent responders.
B. Comparing criterion-related validities between corresponding measures of the Big 5
From the article . . .
Criterion-related validity defined as correlations of GPA with measures of conscientiousness from Big Five Questionnaire 2 and the Mini-Marker questionnaire were computed for each group. Table 5 presents the validity coefficients. Inspection of the table shows that validity was roughly the same for both the most consistent group and the middle group but fell off dramatically for the most inconsistent group for both scales. To provide some evidence of the significance of differences across inconsistency groups, two multigroup regression models were formed using Amos. In the first, GPA was regressed simultaneously onto Questionnaire 2 and Mini-Marker Conscientiousness domain scores allowing regression coefficients and variances to be unique within each inconsistency group, creating a saturated model. In the second, restricted model, variances and regression weights were constrained to be equal across groups, yielding eight degrees of freedom. The chi-square difference statistic was 18.10 (p < .05), suggesting that criterion-related validity was related to inconsistency.
For this analysis, the convergent validities of two measures of conscientiousness were compared across three groups defined by inconsistency of respondents.
The input model is
The constraints are
Note that the variances of the predictors are constrained to be equal across inconsistency groups as are the unstandardized regression coefficients (slopes).
The models
Group 1 (most consistent respondents) results (p values not printed because Chi-square = 0 and p is undefined.)
Group 2 results
Group 3 (most Inconsistent respondents) results. (Worst validity.)
The constrained parameters result
Constraining the parameters to equal across groups resulted in a significantly poorer fitting model, suggesting that there were differences in criterion-related validity across groups. (Although I have to admit, the differences were not huge.)