Repeated-measures ANOVA

Loading the data

Paste columns directly from the sheet (including the header row) – select them in Excel, copy, and paste here:

Variables and settings

When to use?

When the same participants contribute to all the means: they are measured in all the conditions of an experiment, or at several points in time (Field, 2024, Section 15.2). Examples: anxiety before a program, at its end and at follow-up; reaction time under three noise conditions; ratings of the same items by the same people. When there are also separate groups – program versus control – the design is mixed: a between-subjects factor and a within-subjects factor (ibid., section 16.2). With only two measurements – the page computes, but the paired-samples t-test gives the same decision (F = t²; Gravetter & Wallnau, 2017, pp. 433-435) and in addition d and a confidence interval for the difference. Different groups in each condition – one-way analysis of variance; several between-subjects factors – design of experiments.

Data structure

Wide format, as in SPSS: a row for each participant, a column for each measurement (Field, 2024, Section 15.8.1). With two within-subjects factors – a column for each combination, in the order in which the second factor changes fastest: (1,1), (1,2), (1,3), (2,1)... – like the list in the SPSS dialog (ibid., section 15.14). Below the list the page shows which column was assigned to which combination, and names like absent_0, absent_4... are split automatically into level names. A participant missing one measurement is excluded entirely (listwise deletion, as in SPSS), and the page reports how many participants were excluded.

The logic: partitioning the variance

The total variance is divided into variance between the conditions and variance within the conditions, and then – and this is the difference – the variance within the conditions is partitioned again: part of it is the consistent differences between the participants, and it is removed from the error (Gravetter & Wallnau, 2017, pp. 420-424). What remains – the interaction between the conditions and the participants – is the denominator of the F. In the example there: SS between conditions 84, between participants 66 and error 22, and F(2, 10) = 19.09 (ibid., p. 426) – the page gives the same numbers. This is why a within-subjects design is more sensitive: in Gravetter and Wallnau's numerical example the same differences give an F of 1.91 with separate groups and 11 with repeated measures (ibid., pp. 430-431). In a design with two within-subjects factors each effect has its own error – the interaction of the effect with the participants (Maxwell et al., 2018, p. 673).

Sphericity and the ε correction

The univariate test assumes sphericity: that the variance of the difference between any two measurements is equal (Maxwell et al., 2018, p. 627; Field, 2024, Section 15.5.1). With two measurements there is one difference, and the assumption always holds (Maxwell et al., 2018, p. 628). With measurements over time it is often violated – adjacent measurements are more highly correlated than distant ones – and then the Type I error rate can reach .10 or .15 instead of .05 (ibid., pp. 629-630).

The solution: multiply the degrees of freedom of the test by ε, an estimate of the degree of departure from sphericity – 1 when it holds, and down to 1/(k − 1) at the largest departure (ibid., p. 630; Field, 2024, Sections 15.5.2, 15.5.4). The F itself does not change, only the p. Four options, from the most liberal to the most conservative: no correction, Huynh-Feldt, Greenhouse-Geisser and lower bound. Maxwell and colleagues "think the unadjusted test should never be used" (2018, p. 632), their general recommendation is Greenhouse-Geisser, because Huynh-Feldt may fail to maintain the error rate (ibid., p. 634), and the lower bound is too conservative (ibid.). Field (2024, section 15.5.4) also gives the common rule – Huynh-Feldt when the estimate is above .75 – and the option of taking the average of the two. The page reports according to Greenhouse-Geisser, and another option can be selected.

Mauchly: a significance test of the sphericity assumption. Field recommends not relying on it: it depends on the sample size – in a large sample it is significant even for a negligible departure, and in a small one nonsignificant even for a large one – so it is better to always correct by ε (2024, section 15.5.2). Maxwell and colleagues also write that it has "little value as a preliminary test" (2018, p. 629). The page shows it because SPSS shows it.

The multivariate approach

Another way, which does not assume sphericity at all: turn the k measurements into k − 1 difference variables and test whether all their means are zero – a multivariate analysis of variance (Maxwell et al., 2018, pp. 728, 741; Field, 2024, Section 15.5.4). SPSS prints four tests: Pillai, Wilks, Hotelling and Roy. For a purely within-subjects effect they are always identical (Maxwell et al., 2018, pp. 832-833); when they differ, Pillai is the most robust (ibid., p. 829). Pallant (2020, chapter 18) always reads the multivariate test.

Which is better? The multivariate test is exact when the distribution is normal, but weak in small samples, and impossible when there are fewer participants than measurements (Maxwell et al., 2018, pp. 775-777). The rule of Algina and Keselman (1997, as cited in Maxwell et al., 2018, p. 776): multivariate when there are up to 4 measurements, ε ≤ .90 and at least k + 15 participants, or 5-8 measurements, ε ≤ .85 and at least k + 30; otherwise – univariate with a correction. The rule refers to the population ε, and choosing by the sample estimate "may be problematic" (ibid.). In their summary Maxwell and colleagues give the multivariate approach "a slight edge" (ibid., p. 778). "Interpreting the result" in the output computes the rule on your data.

Mixed design

When there is also a between-subjects factor, there are two kinds of tests (Maxwell et al., 2018, pp. 691-693). The between-subjects effect is tested against the variance between participants within the groups – it is equivalent to a one-way analysis of variance on each participant's mean, and sphericity is not relevant to it (ibid., pp. 692, 706). The within-subjects effect and the interaction with the group are tested against the interaction of the within-subjects factor with the participants within the groups. The interaction is usually the question: is the change over time different in the program and in the control group? It is reported first (Pallant, 2020, Chapter 20).

An additional assumption: that the covariance matrices of the measurements are equal across the groups. The tests are robust to its violation when the groups are equal in size (Maxwell et al., 2018, p. 704). SPSS tests it with Box's M; Pallant (2020, chapter 20) suggests reading it at α = .001 because it is very sensitive, and Maxwell and colleagues (2018, p. 840) consider its practical usefulness limited. The page shows it and Levene's test for each measurement.

Contrasts, pairwise comparisons and simple effects

The F-test says there is a difference, not where. Three ways to proceed – choose one according to the question (Field, 2024, Sections 15.9.5, 15.15.3):

Planned contrasts – questions defined in advance: a linear or quadratic trend over time (polynomial, the SPSS default), each condition versus a baseline condition (simple), or each level versus the next (repeated) (ibid., section 15.8.2). Here it is important that each contrast is tested with its own error – the variance of the contrast scores – and not with the pooled error, which assumes sphericity; contrasts are much more sensitive to violations of sphericity than the omnibus test (Maxwell et al., 2018, pp. 638-639). This is what the page does, and what SPSS does.

Pairwise comparisons – every pair of measurements, with a Bonferroni correction. With repeated measures this is the recommendation, and not Tukey, which does not maintain the error rate when sphericity does not hold (Maxwell et al., 2018, pp. 757-758; Field, 2024, Section 15.5.3).

Simple effects – when there is an interaction: the effect of one factor at each level of the other – for example, the change over time in the program separately and in the control group separately (Field, 2024, Section 15.15.3). Each such effect is an additional test; Field recommends breaking down an interaction with as few tests as possible and correcting for their number (ibid.).

Effect sizes

SPSS prints partial eta squared: SS_effect/(SS_effect + SS_error) – the differences between the participants are not in the denominator (Gravetter & Wallnau, 2017, p. 427). Hence it is large, and not comparable with the eta squared of a between-subjects study. Maxwell and colleagues (2018, pp. 634-635) prefer omega squared, which keeps the variance between participants in the denominator, as in a between-subjects study – "it seems desirable not to ignore it" (ibid., p. 635) – and which estimates the population rather than the sample. In McCarthy's example there: ω² = .04 (ibid.), while the partial eta squared of the same data is .22. The page shows both, and omega squared – in the designs for which the book has a formula (one or two within-subjects factors, or one with a between-subjects factor; ibid., Equations 11.35, 12.6 and 12.22-12.24). For a contrast or a pairwise comparison d can be computed (paired-samples t-test).

How to read the output?

The tables are in SPSS order: the means; sphericity (Mauchly and ε); the within-subjects tests by the selected correction; a table comparing the p in the four ways; the multivariate tests; in a mixed design – the between-subjects test, Levene and Box's M; the contrasts; the marginal means; the pairwise comparisons; the simple effects; and a plot. Read the interaction first, then the main effects. "Interpreting the result" points to what matters in your case – the degree of departure from sphericity, univariate or multivariate, and the interaction.

Assumptions

The assumptions of ordinary analysis of variance – independence between participants, normality (mainly in a small sample) and equality of variances – plus one more: "homogeneity of covariance", that is, the participants keep more or less their relative position across the conditions (Gravetter & Wallnau, 2017, p. 428). In the exact version – sphericity (above). In a mixed design – also equal matrices across the groups (Maxwell et al., 2018, p. 704). When the assumptions do not hold: a robust test on trimmed means (rmanova in the WRS2 package in R; Field, 2024, section 15.10), the Friedman test as a last resort (ibid., section 15.7), or a multilevel model (ibid., section 15.5.4).

And how do you do it in SPSS?

Analyze → General Linear Model → Repeated Measures: define the within-subjects factor and the number of levels, assign the columns in order, and add a between-subjects factor if there is one (Field, 2024, Sections 15.8, 16.5; Pallant, 2020, Chapters 18, 20). In Options – descriptives, effect size, and in a mixed design the homogeneity tests; in EM Means – comparison of main effects with Bonferroni. The relevant table: Tests of Within-Subjects Effects, in the Greenhouse-Geisser row (Field, 2024, Section 15.9). The page downloads ready-made syntax (GLM ... /WSFACTOR). PSPP does not run repeated measures in GLM, so the syntax was not tested in it.

And how do you do it in R?

The closest route to SPSS: a multivariate model (lm with a response matrix) and car::Anova with idata and idesign – gives the multivariate tests, the univariate ones, Mauchly and the corrections; emmeans – marginal means and comparisons. Two small differences from SPSS: Mauchly in R includes a second-order term, and Huynh-Feldt in the car package is computed with a different correction when there are groups. The script below was run as is on the sample data file (values in the comments).

# Repeated measures ANOVA in R – the page's sample data file (36 students, program/control × three measurements of test anxiety)
library(car)       # Anova on a multivariate model
library(emmeans)   # Marginal means and comparisons
options(contrasts = c("contr.sum", "contr.poly"))   # Required for Type III

d <- data.frame(group = factor(c(1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2), labels = c("program", "control")),
                pre = c(46, 53, 34, 59, 59, 50, 39, 39, 41, 61, 38, 79, 60, 59, 68, 75, 55, 43, 38, 51, 48, 44, 56, 51, 64, 51, 44, 60, 60, 61, 33, 48, 45, 39, 52, 32),
                post = c(48, 34, 39, 48, 53, 42, 32, 27, 25, 41, 44, 65, 54, 59, 51, 67, 39, 38, 40, 46, 47, 59, 45, 54, 64, 55, 55, 64, 50, 56, 30, 39, 46, 43, 50, 30),
                followup = c(34, 45, 27, 42, 55, 43, 41, 42, 40, 55, 45, 67, 55, 57, 51, 70, 44, 36, 37, 51, 53, 52, 54, 64, 78, 50, 49, 62, NA, 53, 37, 53, 47, 47, 51, 30))
d <- na.omit(d)                                     # Listwise deletion: 35 remain
fit <- lm(cbind(pre, post, followup) ~ group, data = d)
idata <- data.frame(time = factor(1:3))
av <- Anova(fit, idata = idata, idesign = ~ time, type = 3)
summary(av, multivariate = TRUE)
# Time (time): F(2, 66) = 5.94, GG eps = .94
# Interaction (group:time): F(2, 66) = 9.46, p (GG) = .0003
# Pillai for the interaction: V = .42, F(2, 32) = 11.50

ml <- list(time = c("pre", "post", "followup"))
pairs(emmeans(fit, ~ time | group, mult.name = "time", mult.levs = ml), adjust = "bonferroni")
# In the program: pre - post = 8.44 (p = .0001)

Reporting the results

Report the design, which correction was applied and the ε, and for each effect F with the corrected degrees of freedom (e.g. F(1.60, 11.19)), p and an effect size – stating which one (Field, 2024, Section 15.12). When reporting the multivariate test – Pillai's trace (V) and its F (ibid.; Pallant, 2020, chapter 18 – there, Wilks). Means and standard deviations – in a table. In a mixed design – the interaction first (Pallant, 2020, Chapter 20). The page writes such a paragraph in English.

Common mistakes

Analyzing repeated measures as if they were separate groups – the differences between the participants remain in the error, and the test loses most of its advantage (Gravetter & Wallnau, 2017, pp. 430-431).

Reading the uncorrected row – "Sphericity Assumed" – without checking sphericity (Maxwell et al., 2018, p. 632).

Deciding on the correction by Mauchly's test – the test depends on the sample size; correct by ε (Field, 2024, Section 15.5.2).

Choosing between Greenhouse-Geisser and Huynh-Feldt after seeing which one is significant – Field (2024, section 15.9.3) shows how on the same data one gives .063 and the other .048; the decision should be made in advance, and what matters is the effect size.

Tukey, or contrasts with the pooled error – both assume sphericity; use Bonferroni and a separate error for each contrast (Maxwell et al., 2018, pp. 639, 757-758).

Comparing partial eta squared from repeated measures with that of a between-subjects study – the denominators differ (Gravetter & Wallnau, 2017, p. 427; Maxwell et al., 2018, p. 635).

Inferring causality from change over time without a control group – order effects such as practice, boredom and fatigue can explain it (Field, 2024, Section 1.6.5; Gravetter & Wallnau, 2017, p. 430); with conditions – counterbalance the order by random assignment (Field, ibid.; the Variables, scales and research designs page).

References

American Psychological Association. (2020). Publication manual of the American Psychological Association (7th ed.). https://doi.org/10.1037/0000165-000

Field, A. (2024). Discovering statistics using IBM SPSS statistics (6th ed.). SAGE.

Gravetter, F. J., & Wallnau, L. B. (2017). Statistics for the behavioral sciences (10th ed.). Cengage Learning.

Maxwell, S. E., Delaney, H. D., & Kelley, K. (2018). Designing experiments and analyzing data: A model comparison perspective (3rd ed.). Routledge.

Pallant, J. (2020). SPSS survival manual: A step by step guide to data analysis using IBM SPSS (7th ed.). Routledge.

Algina and Keselman (1997) – as cited in Maxwell and colleagues (2018); Girden (1992) and others – as cited in Field (2024). According to the site rule, the reference list includes only sources that were read in the original. Field (2024) and Pallant (2020) in a reflowable digital copy – cited by section and chapter; Maxwell and colleagues (2018) and Gravetter and Wallnau (2017) – by the printed page. Site conventions: wide format and listwise deletion as in SPSS; Greenhouse-Geisser as the default in reporting; Huynh-Feldt in the original formula as in SPSS; Type III and unweighted marginal means; two digits after the decimal point in the tables. The computation is verified against R 4.3.3 (car, emmeans) and against the printed numbers: in Maxwell and colleagues – Tables 11.5, 12.1, 12.7 and 12.15 (with 12.5, 12.19 and the ε estimates), 13.2 and 14.9; in Gravetter and Wallnau – Table 13.2 and the SPSS output in Figure 13.4; and by comparison with the SPSS printouts in Field (2024) and Pallant (2020).