Computer Use for Statistics

Computer Use Classes Accompanying Statistics Lectures 2007-2008:

Coordinator: Dr Alan PickeringTutors: Ian Tharp; Elizabeth Ward

Weeks 11 -15 Spring Term: Selected General Linear Model Topics

Time and venue: Monday 4-5.15 pm in WB304 (Psychology Department Computer Classroom) for ALL students.

For students on the following courses:

Computer Use (PS71021A) MSc Research Methods in Psychology

Computer Use (PS81021A) Research Training for 1st Year PhD students

Multivariate Statistical Methods in Psychology (PS53011B) BSc Psychology

WEEK 11: MANOVA

Learning Outcomes:

After the session (and suggested self-study exercises) students should be able to:

Check for multivariate outliers within groups using the Mahalanobis measure.
Use the SPSS GLM:Multivariate command from the Analyse menu to execute a MANOVA.
Report Bonferroni-corrected univariate ANOVA results.
Use the SPSS GLM:Repeated Measures command from the Analyse menu to analyse a doubly-multivariate design.

Dataset To Be Used:

If you have access to the Goldsmiths computer network then the file can be downloaded from the J drive:

J:\psycholo\APstats\comp_classes\fwmanova.sav

or it can be downloaded from the web by clicking and following the relevant links at the following URLs:

This dataset is a real set of data based on a treatment study of 98 participants suffering from severe personality disorder (PD). Individuals were either admitted to a specialist treatment unit (admit =1) or were not admitted (admit = 0), the latter serving as controls. Outcome measures were 8 standardised measures of psychological well-being. These measures were given at baseline before assignment to groups and they were readministered at long-term follow-up one year later/one year after treatment. There are three variables recorded for each outcome measure: one at baseline, one at follow-up and the change score (scored so that positive change scores reflect improvements). For each variable, there is a measure ending in f or fup (designating follow-up) and a measure ending in c, ch, or chge (denoting change score). The variables (baseline names are given in parentheses) measure borderline symptoms (bsi), impulsive “deeds” (mis_dmr), impulsive “wants” (mis_wmr), anxiety (idaanx), depression (idadep), outwardly directed anger (idaout), inwardly directed anger (idain), and self-esteem (se_mean).

Specific Tasks and Questions:

(i)We are interested in the change scores within this treatment study and want to answer the question as to whether the treated group shows a significantly larger improvement in psychological well-being than the control group. We could carry out 8 separate univariate ANOVAs to determine the answer for each variable separately. However, carrying out a family of 8 separate tests means that each test would have to be significant at p=0.05/8, if we used the Bonferroni correction to preserve the familywise Type I error rate. It might be suggested that all the outcome measure change scores index a common underlying construct (change in psychological well-being), albeit reflecting somewhat different facets in each measure. If so, we can use MANOVA to compare the groups on a multivariate DV combining all the 8 DVs, but requiring only one statistical test (with an uncorrected significance level of p=0.05).

(ii)To check that all the 8 DVs measure the same construct, use SPSS to compute a correlation matrix between the 8 change scores. Are they all moderately correlated with one another? Why would a lack of correlation between the measures (or an almost perfect correlation) indicate that MANOVA is inappropriate?

Skip to Point (viii) During Computer Class: You can try (iii) to (vii) in your own time

(iii)Before carrying out a MANOVA, we need to screen the data in various ways. We shall concentrate on checking for multivariate outliers using the Mahalanobis distance (MD) technique. We did this in Week 6 before Christmas. The difference here is that we are going to do it separately within each treatment group (as is appropriate for a between-subjects design). To carry out analyses within groups you should use the SPLIT FILE command from the Data menu. Select the “organise output by groups” option and use the admit variable for the grouping.

(iv)With SPLIT FILE “on”, carry out a forced entry Linear Regression using all 8 of the change scores as predictor variables. As we are not interested in the results of these analyses (other than the MDs) we can use any dummy variable as the DV (use subject ID -- id). The Save button contains the MD option, which should be checked. The distances are stored in the dataset under the name Mah_1.

(v)The MD measure is compared against the critical value of the chi-squared distribution with p=0.001 and df=8 (the number of variables involved). The critical value is 26.1. Do you remember how to use SPSS to find out what this critical value is?

(vi)Any subject with an MD larger than the critical value is a multivariate outlier and so can be deleted. Were any multivariate outliers identified by this method?

(vii)N.B. REMOVE SPLIT FILE (chose “analyse all cases”) BEFORE GOING ANY FURTHER.

(viii)Now we can do some MANOVAs. We are going to begin by analysing 3 of the most important of our change score variables: bsi_chge, misdmrc, and miswmrc. The majority of the PD patients in this research have a diagnosis of borderline personality disorder. Hence producing changes in borderline symptoms generally, and in the critical symptoms of impulsive deeds and desires (“wants”) are of primary importance. Select GENERAL LINEAR MODEL:Multivariate from the Analyse menu. Enter the 3 change scores as dependent variables and the admit variable as the “Fixed Factor”. Hit the Options button and select the “Descriptive Statistics” and “Homogeneity Tests” options.

(ix)Go through the printout and make sure you understand what every part of the printout indicates. Some key questions:

What do the homogeneity tests reveal and what, if anything, should we do about the results?
Do the means indicate greater improvements for treated as opposed to non-treated subjects?
Which is the principal MANOVA result and what does it tell us?
How many of the ANOVAs would have been significant with a Bonferroni correction?

(x)We could have analysed this design in a “doubly multivariate” fashion, using our 3 measures as the basis for a multivariate DV as before but, instead of using the change scoers, by including the scores at the two timepoints (baseline and follow-up) as repeated measures. Use GLM:Repeated Measures to carry out this analysis. Enter time as the within-subjects factor name with 2 levels. Open up the Measure box at the bottom of the first window. For each of the 3 pairs of repeated measures generate a measure name (must not be same as any variables in the dataset) and “add” it. (For example you might call the BSI scores bordsymp etc). Then you hit the Define button and enter the relevant variables (bsitot bsifup for borderline symptoms baseline and follow-up; mis_dmr misdmrf for impulsive deeds baseline and follow-up; mis_wmr miswmrf for impulsive wants baseline and follow-up). Remember that the baseline score is repeated measure 1 for each variable and the follow-up score is repeated measure 2. Enter admit as the “Fixed Factor”.

(xi)Look at the output. Some key questions:

What do the multivariate tests of the Admit, Time, and Admit*Time effects tell us?
Which multivariate effect is critical for our hypothesis of interest (that larger improvements might be found in the treated group)?
Does this give exactly the same results as found for the group effect from the earlier MANOVA?
Why is this strictly not a doubly multivariate analysis?
Which univariate results are equivalent to the univariate results produce earlier?

(xii)Repeat the basic MANOVA with all 8 change variables (bsi_chge idaanxch idadepch idainch idaoutch misdmrc miswmrc settotch). Look at the output:-

What is the principal MANOVA result and what does it tell us?
How many of the ANOVAs would have been significant with a Bonferroni correction?
Is there a conflict between the MANOVA and ANOVA findings? (Consider how many of the ANOVA findings would be expected to be significant by chance alone.)

(xiii)One of the reasons for the conflict listed above is the low power of MANOVA in this example. One important factor in determining the power of the MANOVA is the size and sign of the "pooled within-cell correlations". These are just the partial correlations between each of the DVs after controlling for the IVs (admit in this case). If these correlations are large and negative, then MANOVA has much more power than if the correlations are moderately negative, zero or positive. These correlations can be shown by SPSS directly using the old MANOVA syntax (see T & F). However, they are easily estimated by using the partial correlation option (with admit as the variable to be controlled). Use SPSS to produce these correlations and note what size and sign they have.