In this section we turn to Max & Onghena’s (“Some issues in the statistical analysis of completely randomized and repeated measures designs for speech, language and hearing research”, JSLHR 42, 261-270, 1999) second critique of common practice in doing ANOVAs: the fact that we’ve been ignoring the problem of possible sphericity violations in repeated measures designs. StatView does not deal with this problem, and thus it presents a limitation on using StatView to do your repeated measures analyses.
ANOVA assumes homogeneity of variance across
conditions but is robust against violations of this assumption IF the data
are uncorrelated. Heterogeneity of variances in correlated data gives
a positive bias in F tests (the critical value of the F is too low). Sphericity
means that data are uncorrelated; violations of sphericity require strict
homogeneity of variance. If there are only 2 levels of a factor, this
problem of correlation doesn't arise; but if there are 3 or more levels you
have to guard against it. StatView does not correct for sphericity violations
in RM analyses. Thus you can only reliably use StatView for RM designs
in which each factor has only two levels. If a design has more than
2 levels in any factor, then you should not use StatView. For example,
StatView's own sample Wine file, with 7 levels in its 1 factor, should not
be analyzed with StatView!
It can be seen that StatView does not correct for sphericity violations by comparing the output from StatView with the sample outputs in Max & Onghena’s article. Use the file m&o-2.xls with StatView’s RM analysis, as described above, and compare the result with Table 1 from the article (below). It is the same as the first row in the table, the “ANOVA with uncorrected df” analysis. This row is included by Max & Onghena to illustrate the WRONG result (in that it overestimates the significance).
Some options and partial solutions
Option 0: Of course if you run the test in StatView and there’s no significant effect, then you’re ok – we’re talking here about over-estimating significance and getting a false positive; if you get no effect even with such a method, then you know there really is no effect, and there’s no need to go to special lengths to continue to fail to get an effect.
Option 1: Test for sphericity violations, i.e. test for correlations among variables. StatView provides Bartlett’s test of sphericity. A resulting high chi-square value with a low p value is BAD, but if the data are uncorrelated you’re probably OK. (However, Max & Onghena are dubious about the integrity of such tests in the first place, so even this isn’t clear. We’ll see proof that this is a waste of time with our later sample analysis.) But in any event note that StatView’s test doesn’t do anything as a result of a significant correlation, or tell you what to do to correct for the violation. All the test does is tell you if there’s a problem (maybe).
how to do Bartlett’s test in StatView (though generally a waste of time):
Data cannot be in a compact variable: you need all the levels sitting there like variables (the use of the term “variable” here is somewhat misleading). Under Analyze – Correlations, select Bartlett’s test; drag all the levels of your repeated measure into the box. Do this with Wine Tasting file (don’t make a compact variable). Data are correlated, i.e. analysis above was invalid.
2: Use StatView’s MANOVA
instead of ANOVA; a significant result from this test is valid; but this procedure
is very conservative, i.e. unlikely to show significant differences.
how to do MANOVA in StatView:
First, try a regular MANOVA (simultaneous analysis of 2 dependent variables): Open their Exercise file, and under Analyze – ANOVA, select MANOVA. Ignore “independent variable”, drag the variables into “dependent variables” and “factors” as appropriate. This gves all the separate ANOVA tables, plus the other stuff.
Now try a MANOVA as a substitute for ANOVA with a repeated measure: Do some analysis so that you have the Analysis Browser open (to the left) -- e.g. the file 3-level-compact -- do RM ANOVA (select View - Variable Browser if it’s not already there); then under Analysis Browser - ANOVA, double-click MANOVA Tables (you can’t use the regular menu for this as it defaults to factorial design; but the browser gives you the choice); be sure to select Repeated Measures design; AND use the Variable Browser to declare your compact variable as dependent (note that you can do this after the analysis fails to run – it will run after you do this)
You use the Wilks’ Lambda result from MANOVA Tables (though here all are same).
If you do this analysis on the Max & Onghena data, you’ll see that you do get the “MANOVA (Wilk’s lambda)” result in row 3 of their table. Note that the difference is not even close to significant this way – this is a very conservative test! Another catch to this analysis is that it requires more subjects than there are repeated measures (e.g. if you have 5 factors in your experiment then you need at least 6 subjects).
Option 3: use SPSS – it allows you to correct the df according to the degree of correlation in the data – see next section
last updated Dec. 2006 by P. Keating
back to the UCLA Phonetics Lab statistics page
Back to the UCLA Phonetics Lab overall facilities page