Analysis of covariance (ANCOVA)

Calculator

Loading the data

Paste columns directly from the sheet (including the header row) – select them in Excel, copy, and paste here:

Variable selection

Guide

When to use?

When group means are compared (as in one-way ANOVA), and there is an additional continuous variable related to the dependent variable – an accompanying variable (covariate; in the past it was called in English concomitant variable, a "variable that comes along" with the main focus of the study – Maxwell et al., 2018, p. 452; also called a "control variable"). The common case: an experiment with a before and after measurement – the comparison is on the "after" score, and the "before" score is the covariate (Pallant, 2020, Chapter 22; Field, 2024, Section 13.5.1). ANCOVA is a linear model that contains both a categorical factor and a continuous predictor (Field, 2024, Section 13.3). When there are several measurements of the same participants and the question is the change itself – repeated-measures ANOVA.

The logic: parallel lines

The model fits to each group a regression line of the dependent variable on the covariate – with the same slope for all the groups, the common within-group slope (b_W) – and tests whether these parallel lines are at different heights (Maxwell et al., 2018, pp. 463-466). The error is the deviation of each participant from the line of their group, and so when the covariate predicts well, the error is much smaller than in an ordinary analysis of variance. In the small example of Maxwell and colleagues, the error sum of squares drops from 28 to 12 (ibid., pp. 467-468). In the scatter plot the page draws, this can be seen directly.

Adjusted means

The adjusted mean of a group is the height of its line at the grand mean of the covariate: Ȳ′_j = Ȳ_j − b_W(X̄_j − X̄) (Maxwell et al., 2018, Equation 9.27, p. 469). A group that started high on the covariate "goes down" along the line, and a group that started low "goes up" – the adjustment is "sliding the mean along the regression line" (ibid., p. 471). The F test of the factor tests the differences between the adjusted means, and so they are the ones interpreted, not the raw ones (Field, 2024, Section 13.7.2). In the example of Maxwell and colleagues: the depression means after treatment are 7.3, 12.2 and 14.0, and the adjusted ones 7.54, 11.98 and 13.98 (2018, pp. 483-484) – the page gives the same numbers.

With and without random assignment

When the groups were randomly assigned, they are equal on average on the covariate, and the main contribution of ANCOVA is reducing the error – a more powerful test and narrower confidence intervals, while the adjusted and raw means estimate the same difference (Maxwell et al., 2018, pp. 454-455, 471). Even if the groups turn out to differ slightly on the covariate by chance, ANCOVA is preferable (ibid., p. 476).

When the groups exist in advance – classes, schools, those who chose to participate – the adjustment may change the result and even reverse it. ANCOVA "is not a substitute for random assignment" (Maxwell et al., 2018, p. 454): it answers the question "would the groups have differed had they been equal on the covariate", and there is no way to guarantee that it correctly adjusts for differences that were not measured (ibid., p. 455; Field, 2024, Section 13.5.1). In addition, when the covariate is measured with measurement error, the adjustment is incomplete (Maxwell et al., 2018, p. 482). The page checks whether the covariate differs between the groups ("balance"). This is not a statistical requirement, but it determines how to interpret (ibid., pp. 475-476; Field, 2024, Section 13.5.1).

Choosing the covariate

The covariate should be measured before the treatment – a variable that the treatment could have affected "takes out" part of the effect itself (Pallant, 2020, Chapter 22; Maxwell et al., 2018, p. 478). It is chosen in advance, based on previous studies showing that it predicts the dependent variable, and not based on the data of the current study (ibid., p. 494). A "before" measurement of the dependent variable is a good candidate (ibid., p. 451). It should be reliable – Pallant suggests a Cronbach's alpha above .7, preferably above .8 (chapter 22; see Cronbach's alpha). How many covariates? In a small study no more than two or three (Maxwell et al., 2018, p. 493), and not ones that are highly correlated with each other – around r = .80 it is worth dropping one (Pallant, 2020, Chapter 22).

Homogeneity of slopes

The central assumption: the association between the covariate and the dependent variable is the same in all the groups – the lines really are parallel. It is tested with a model that adds the factor × covariate interaction: if it is significant, the slopes differ (Pallant, 2020, Chapter 22; Field, 2024, Section 13.8; Maxwell et al., 2018, p. 520). The page always runs the check, and also shows the slope in each group. What to do when it is violated? Pallant (2020, chapter 22) writes that ANCOVA should then not be run; Field (2024, section 13.8) suggests keeping the model with the interaction and interpreting it; Maxwell and colleagues (2018, pp. 536-537) recommend a model with separate slopes – already when the homogeneity test is close to significance, e.g., p < .2 – and the Johnson-Neyman technique, which shows at which values of the covariate the difference between the groups is significant (ibid., pp. 531-535). The page does not run Johnson-Neyman; SPSS has a macro that runs it (ibid., pp. 535-536).

ANCOVA or change scores?

Another common approach: compute "after minus before" for each participant and compare the differences in an analysis of variance. In an experiment with random assignment both approaches test the same hypothesis, but ANCOVA has a smaller error, and so it is "almost always preferable" (Maxwell et al., 2018, pp. 499-501). A change score is equivalent to ANCOVA with a slope fixed in advance at 1 (ibid., p. 499). When the groups are not random, the two approaches ask different questions and may give different answers – "Lord's paradox" (ibid., pp. 501-502). A third approach – computing residuals from a regression on the covariate and analyzing them – is not equivalent to ANCOVA, and it is best avoided (ibid., p. 498).

Pairwise comparisons

When there are more than two groups, the adjusted means are compared in pairs. The standard error of each difference includes the difference in the covariate means (Maxwell et al., 2018, Equation 9.37, p. 490). The page corrects with Bonferroni – Field's recommendation (2024, section 13.6.6), and also the SPSS approach, in which the regular Post Hoc tests are blocked when there is a covariate (ibid., section 13.6.5). Maxwell and colleagues suggest Tukey for all pairwise comparisons when the covariate values were fixed in advance, and Bryant-Paulson when they were measured (2018, pp. 490-492); since a covariate is usually measured like the dependent variable, some recommend regarding it as a random variable (ibid., p. 475).

Effect sizes

SPSS prints partial eta squared – the variance the factor explains out of what the other variables did not explain (Field, 2024, Section 13.4). Maxwell and colleagues (2018, pp. 486-489) suggest thinking of the covariate as an "intrinsic part" of the design – it exists in the population – and therefore not removing it from the total variance: omega squared according to equation 9.31 (in their example .12; for comparison, the R² of the factor is .18 out of all the variance and .22 when the covariate is partialed out – pp. 487-488), and d for a pairwise comparison – the difference between the adjusted means divided by the standard deviation from the analysis of variance without the covariate (equation 9.32). The page shows all three (omega squared – when there is one factor).

How to read the output?

In order: the raw means (also of the covariate); the preliminary checks – balance, homogeneity of slopes and Levene; the ANCOVA table – a row for each covariate and each factor; the slope; the adjusted means vs. the raw ones; the pairwise comparisons; a table that compares with the analysis of variance without the covariate – how much the error dropped and whether the conclusion changed; and charts. "Interpreting the result" points to what matters in your case.

Assumptions

The assumptions of analysis of variance – independent errors, normality and equal variance – and two more: a linear association between the covariate and the dependent variable in each group, and homogeneity of slopes (Maxwell et al., 2018, pp. 473-475; Pallant, 2020, Chapter 22). Linearity is checked in the scatter plot; a curved association can be included with a quadratic term (Maxwell et al., 2018, p. 493). The page runs Levene on the absolute value of the residuals from the model. When the assumptions are not met – Field (2024, sections 13.5.3, 13.9) describes bootstrapping and robust ANCOVA.

And how do you do it in SPSS?

Analyze → General Linear Model → Univariate: the dependent variable, the factor in Fixed Factor(s) and the covariate in Covariate(s) (Field, 2024, Section 13.6; Pallant, 2020, Chapter 22). Homogeneity of slopes is checked in a separate run with Model → Build terms that includes the interaction (Field, 2024, Section 13.8). In EM Means – comparisons with Bonferroni; in Options – effect size, homogeneity tests and Parameter estimates. The page downloads syntax with both runs (UNIANOVA ... WITH). PSPP does not run GLM with covariates, and so the syntax was not tested in it.

And how do you do it in R?

A linear model (lm) with sum-to-zero contrasts, car::Anova(type = 3) for the table, anova(m, ms) for homogeneity of slopes, and emmeans for the adjusted means and the comparisons (the covariate at its mean – the default). The script below was run verbatim on the example of Maxwell and colleagues (2018, Table 9.7, p. 483), and the values are in the comments.

# ANCOVA in R – the example of Maxwell and colleagues (2018, Table 9.7): depression before and after, three groups of 10
library(car)       # Anova of Type III
library(emmeans)   # adjusted means
options(contrasts = c("contr.sum", "contr.poly"))   # required for Type III

d <- data.frame(group = factor(rep(c("SSRI", "Placebo", "Wait"), each = 10), levels = c("SSRI", "Placebo", "Wait")),
                pre  = c(18, 16, 16, 15, 14, 20, 14, 21, 25, 11, 18, 16, 15, 14, 20, 25, 11, 25, 11, 22, 15, 19, 10, 29, 24, 15, 9, 18, 22, 13),
                post = c(12, 0, 10, 9, 0, 11, 2, 4, 15, 10, 11, 4, 19, 15, 3, 14, 10, 16, 10, 20, 17, 25, 10, 22, 23, 10, 2, 10, 14, 7))
m  <- lm(post ~ pre + group, data = d)
ms <- lm(post ~ pre + group + group:pre, data = d)
anova(m, ms)                          # homogeneity of slopes: F(2, 24) = 1.06, p = .36
Anova(m, type = 3)                    # group: F(2, 26) = 3.73, p = .038; pre: F(1, 26) = 10.77
coef(m)["pre"]                        # common slope b = 0.645
emmeans(m, pairwise ~ group, adjust = "bonferroni")
# adjusted means: 7.54, 11.98, 13.98; SSRI vs. wait: -6.44, p = .039

Reporting the results

Report the covariate and the preliminary checks, mainly homogeneity of slopes (Pallant, 2020, Chapter 22), and then the F of the covariate and of the factor, with p and effect size (Field, 2024, Section 13.11; Pallant, ibid.). In the table – the raw means with standard deviation and the adjusted ones with standard error, and n in each group (ibid.). The page writes such a paragraph in English.

Common mistakes

"Controlling" for differences between pre-existing groups and inferring causation – ANCOVA does not replace random assignment (Maxwell et al., 2018, p. 454; Field, 2024, Section 13.5.1).

A covariate measured after the treatment – may take out part of the effect (Maxwell et al., 2018, p. 478).

Interpreting the raw means instead of the adjusted ones (Field, 2024, Section 13.7.2).

Skipping the check of the slopes – when they differ, the difference between the groups depends on the value of the covariate (Field, 2024, Section 13.8).

Choosing covariates based on the data – for example, the one most strongly correlated with the dependent variable in the current sample (Maxwell et al., 2018, p. 494).

An analysis of residuals instead of ANCOVA (Maxwell et al., 2018, p. 498).

References

American Psychological Association. (2020). Publication manual of the American Psychological Association (7th ed.). https://doi.org/10.1037/0000165-000

Field, A. (2024). Discovering statistics using IBM SPSS statistics (6th ed.). SAGE.

Maxwell, S. E., Delaney, H. D., & Kelley, K. (2018). Designing experiments and analyzing data: A model comparison perspective (3rd ed.). Routledge.

Pallant, J. (2020). SPSS survival manual: A step by step guide to data analysis using IBM SPSS (7th ed.). Routledge.

According to the site rule, the reference list includes only sources that were read in the original. Field (2024) and Pallant (2020) in a reflowable digital copy – cited by section and chapter; Maxwell and colleagues (2018) – by the printed page. Site conventions: listwise deletion; Type III; adjusted means at the grand mean of the covariates, unweighted across a second factor; Levene on the absolute value of the residuals from the model; two digits after the decimal point in the tables. "Covariate" (in the Hebrew original "concomitant variable") – the site's rendering of covariate (following concomitant variable), so as not to confuse it with a mediating variable (mediator). The calculation is validated against R 4.3.3 (car, emmeans) and against the numbers printed in Maxwell and colleagues (2018): Tables 9.1, 9.7-9.10 and 9.12, exercise 4.