One-way analysis of variance (ANOVA)

Loading the data

Paste columns directly from the sheet (including the header row) – select them in Excel, copy, and paste here:

Variable selection

When to use?

When you want to know whether the mean of a quantitative variable differs between three or more separate groups – three education levels, four experimental conditions, five regions. Pallant (2020, chapter 18) defines the recipe: one categorical independent variable with three or more groups ("one-way" – one independent variable; "between-subjects" – different people in each group), and one continuous dependent variable. Gravetter and Wallnau (2017, pp. 366-370) explain why not a series of t-tests: each test carries a 5% risk of a Type I error, and three groups mean three tests – the "experimentwise alpha level" (experimentwise) accumulates; ANOVA tests all the differences in one test. Field (2024, section 12.3) shows that this is the linear model with dummy variables: the same F obtained from a regression with the groups as predictors.

Only two groups – the page computes it, but the independent-samples t-test gives the same decision (F = t²) and in addition Cohen's d and a confidence interval for the difference; the same participants in several measurements – repeated-measures analysis of variance (Pallant, Chapter 18; Gravetter & Wallnau, Chapter 13), on the Repeated-measures ANOVA page; a comparison between the groups with an additional continuous variable measured beforehand (e.g., a "before" score) – ANCOVA, analysis of covariance; two or more independent variables, or blocks – multifactor ANOVA, on the Design of experiments page (where you will also find sample size and a random run sheet); an ordinal dependent variable or a severe violation of the assumptions in a small sample – the Kruskal-Wallis test (Pallant, Chapter 16; Gravetter & Wallnau, Appendix E). A multi-item scale score (the Cronbach's alpha and scale score page) is treated as an interval variable according to the site convention (the variables and levels of measurement page); a single Likert item is not.

The logic: between-group variance divided by within-group variance

Gravetter and Wallnau (2017, pp. 371-375) decompose all the variability in the data into two parts. Between-group variance – how far the group means are from one another; it stems from the treatment (if it has an effect) and also from individual differences and random error. Within-group variance – how much the participants within the same group differ from one another; it stems only from individual differences and random error, because within a group everyone received the same treatment. F is the ratio between the two: if the treatment has no effect, both sources measure the same thing and F is close to 1; if it does, the numerator grows and F rises. Field (2024, sections 12.3.4-12.3.8) phrases the same idea in the language of the model: SS_T (all the variability around the grand mean) = SS_M (what the group means explain) + SS_R (what remains), and each is divided by its degrees of freedom to obtain mean squares – an "average" of variance, because a sum of squares grows with the number of observations.

The degrees of freedom: df_between = k − 1 (the number of groups minus one), df_within = N − k (Gravetter & Wallnau, pp. 378-379). The example of Gravetter and Wallnau (Example 12.2, pp. 385-388): three learning strategies, six students in each – SS_between = 84, SS_within = 88, MS of 42 and 5.87, F(2, 15) = 7.16, significant; Field's example (section 12.2, Table 12.1): puppy therapy at three doses, five in each group – F(2, 12) = 5.12, p = .025. Both are computed on the page and give exactly the numbers in the books.

How to read the output?

Table 1 – Group statistics (Descriptives). Pallant (2020, chapter 18): "always check this table first – is the N for each group correct?" An empty or tiny group usually indicates a wrong code or many missing values. The "Total" row – the grand mean, around which the between-group variance is measured.

Table 2 – The ANOVA table, which also includes the Welch row (in SPSS it is in a separate table, Robust Tests of Equality of Means). Which row do you read? As in the t-test, the sources are divided, and the page presents both positions:

The "Interpreting the result" box in the output says whether the two rows agree. In Field's puppy example they disagree: regular F p = .025, Welch p = .054 – and Field (section 12.9.1) says this is confusing "only if you attribute magical powers to the .05 threshold". F, df and p. p smaller than .05 – there is a significant difference "somewhere" between the groups; the table does not say between which (Pallant, Chapter 18; Gravetter & Wallnau, p. 393). F has no one-tailed version – p cannot be divided by two (Field, Tip 12.1).

Post hoc tests: between which groups?

A significant F says that at least one difference is significant; the post hoc tests (post hoc) go over all the pairs and correct the significance level so that the overall risk of a Type I error across all pairs together stays at 5% (Gravetter & Wallnau, 2017, pp. 393-394; Field, 2024, Section 12.6). SPSS offers about 18 methods; Field (section 12.6.3, following Toothaker, 1993) sums up: groups of equal size and similar variances – Tukey (or REGWQ); "guaranteed control" of the Type I error – Bonferroni; doubt about equality of variances – Games-Howell, "which I recommend running in addition to any other method". The page shows all three, and Table 3 switches between them with a toggle:

When are they interpreted? Pallant (chapter 18) and Gravetter and Wallnau (p. 393): only after a significant overall F and with three or more groups (with two groups there is nothing to ask). The table is always shown, and when the F is not significant the note says that no differences are reported from it. Set the method in advance – do not choose after the test the one that gives more asterisks; the page's default (Tukey when Levene is not significant, Games-Howell when it is) is such a rule. Planning specific comparisons in advance (planned contrasts, Field, Section 12.5; Scheffé, Gravetter & Wallnau, pp. 395-396) – not on the page.

Effect size

Significance is not size: Pallant (2020, chapter 18) demonstrates with N = 435 – F(2, 432) = 4.64, p = .01, but the means are 21.36, 22.10 and 22.96 and η² = .02: "with a large enough sample, quite small differences can become statistically significant, even if they have no practical importance". The table prints two measures:

Eta squared (η²) – the proportion of the variance of the dependent variable explained by group membership: SS_between/SS_total (Gravetter & Wallnau, 2017, pp. 388-389, formula 12.14 – "the percentage of variance accounted for", η² = 0.488 in Example 12.2; Pallant, 2020, chapter 18 – calculated by hand from the ANOVA table; Field, 2024, equation 12.33 – this is the R² of the linear model). Cohen's thresholds (1988, pp. 284-287) as given by Pallant: .01 small, .06 medium, .14 large.

Omega squared (ω²) – Field (section 12.7, equation 12.34): η² is "slightly biased" because it is based only on the sample sums of squares; ω² corrects toward an estimate for the population: (SS_between − df_between·MS_within) / (SS_total + MS_within) – in the puppy example η² = .46 but ω² = .35, and Field reports ω² (section 12.12) with the same thresholds (Kirk, 1996). In a small sample or with many groups the gap between them is large, and in a large sample they are almost identical; the "Interpreting the result" box shows both. Field adds that the overall effect size is less interesting than the size of the difference between a pair of groups – for each pair Cohen's d can be computed on the t-test page. SPSS 27 and later prints both (ANOVA Effect Sizes) with confidence intervals, which are not on the page.

Assumptions and warnings

The three assumptions (Gravetter & Wallnau, 2017, pp. 391-392): the observations in each group are independent; the populations are normal; and all have the same variance. Independence is the critical assumption: Field (section 12.4.2) cites Scariano and Davenport – a correlation of 0.5 between observations raises the Type I error from 5% to 74%. A participant measured twice, students from the same classes, spouses – violate it, and the correct test is a different one. Normality: Gravetter and Wallnau – "usually not a concern, especially in large samples"; Field (section 12.4.2) disputes the myth that "ANOVA is robust": skew, heavy tails and unequal variances affect F even with equal groups, and with heavy tails the central limit theorem requires samples much larger than 30 – so look at the histogram of each group (the Frequencies and descriptive statistics page) and at outliers (the Outliers and assumptions page), and when in doubt also run a robust test (Field, Section 12.10) or Kruskal-Wallis.

Homogeneity of variances. Gravetter and Wallnau (p. 392, and p. 314): "an important assumption", with the rule of thumb – one variance three to four times larger than the other is cause for concern; Field (Box 12.2): when the groups are unequal in size and the larger ones are more spread out, the regular F is conservative, and in the reverse situation liberal – Welch and Brown-Forsythe correct this. The page prints Levene (around the mean, as SPSS does) and the Welch row; the choice – the "How to read" section.

Groups of unequal size. Gravetter and Wallnau (p. 390): ANOVA is valid even so, "especially when the samples are relatively large and the discrepancy in sizes is not extreme", but it is most accurate with equal groups. Field (section 12.6.2): it is the post hoc tests that break down when sizes and variances differ – Games-Howell was designed for this situation. A level with fewer than two observations is omitted from the analysis (and the page says so).

Missing values. A participant missing a value on the dependent variable or on the group variable is excluded from the analysis of that dependent variable only – analysis-by-analysis deletion (/MISSING=ANALYSIS), as in the default of SPSS and of Pallant; therefore N can change from one dependent variable to the next. What to do about missing values – the Missing values page.

Several dependent variables. Field (section 12.8.1) has reservations about the SPSS option of running several dependent variables at once – "you lose control over the Type I error" – and suggests MANOVA or a combined measure; Pallant (chapter 18) suggests reporting a series of analyses in one table. The page allows several, and reminds in the interpretation box that each analysis carries a 5% risk.

What this page does not compute. Brown-Forsythe, planned contrasts and trend analysis (Field, Section 12.5), Scheffé and REGWQ, Dunnett, confidence intervals for η² and ω² (SPSS 27 and later), bootstrap and a robust test on trimmed means (Field, Section 12.10), two-way and multifactor ANOVA (on the Design of experiments page) and repeated-measures ANOVA (on the Repeated-measures ANOVA page), the Bayes factor.

And how do you do it in SPSS?

Version note. The procedure is stable; from version 27 the dialog includes Estimate effect size for overall tests and prints an ANOVA Effect Sizes table with η², ε² and ω² and confidence intervals (Field, 2024, sections 12.8.1 and 12.9.1 – based on version 29), whereas Pallant (2020, based on version 26) writes that SPSS "does not generate an effect size for this analysis" and calculates eta squared by hand. In old versions – delete the /ES=OVERALL line from the syntax.

The procedure (Pallant, 2020, Chapter 18; Field, 2024, Sections 12.8.1-12.8.4): Analyze › Compare Means › One-Way ANOVA… – the dependent variable (or several) into the Dependent List box, the independent variable into Factor; Options… – Descriptive, Homogeneity of variance test, Welch (and Brown-Forsythe), Means plot, and Exclude cases analysis by analysis; Post Hoc… – Tukey and Bonferroni under "Equal Variances Assumed", Games-Howell under "Not Assumed". The syntax:

ONEWAY attitude_score age BY education
  /ES=OVERALL
  /STATISTICS=DESCRIPTIVES HOMOGENEITY WELCH
  /MISSING=ANALYSIS
  /POSTHOC=TUKEY GH BONFERRONI ALPHA(0.05).

Reading the output. Descriptives = Table 1 (N, Mean, Std. Deviation, Std. Error, confidence interval for the mean, minimum and maximum); Test of Homogeneity of Variances = Levene's test in the note of Table 2 – SPSS 26 and later prints four versions, and the page (and Pallant) read Based on Mean; ANOVA = Table 2 (Between Groups, Within Groups, Total); Robust Tests of Equality of Means = the Welch row (Statistic, df1, df2, Sig.); ANOVA Effect Sizes – Eta-squared and Omega-squared Fixed-effect in the table note; Multiple Comparisons = Table 3 – SPSS prints each pair twice (I−J and J−I) in the three blocks (Tukey HSD, Games-Howell, Bonferroni), with an asterisk next to a significant difference. The "Download SPSS syntax" button in the output generates the loading, the dictionary (if there was a legend) and the ONEWAY command on the selected variables.

And how do you do it in R?

In base R: aov() for the ANOVA table, oneway.test() for Welch (its default), TukeyHSD() for the Tukey tests (Tukey-Kramer, as on the page), pairwise.t.test(p.adjust = "bonferroni") for Bonferroni; Levene's test around the mean – leveneTest(center = mean) from the car package; Games-Howell – the rstatix or PMCMRplus package (not in the script). η² and ω² are computed by hand from the ANOVA table. The script was run verbatim on the sample file and gave the values in the tables (in the comments). The site rule for Hebrew in R: a UTF-8 file, every comment line starts with an English word, and a UTF-8 locale for Hebrew names (on Windows: R 4.2 and later, or Sys.setlocale('LC_ALL', 'Hebrew')).

# One-way ANOVA – Professor Stat. This file is saved in UTF-8; every comment line starts with an English word
library(readxl)   # reading Excel
library(car)      # Levene's test: leveneTest

# Loading: the standard template – row = participant, column = variable; the sheet "data"
d <- read_excel("C:/Rdata/DATA_anova_example_eng.xlsx", sheet = "data")
d <- d[complete.cases(d[["education"]], d[["attitude_score"]]), ]          # analysis-by-analysis deletion (117 of 120)
g <- factor(d[["education"]], levels = 1:4, labels = c("high school", "post-secondary", "bachelor's degree", "graduate degree"))
y <- d[["attitude_score"]]

# Table 1 – group statistics
tapply(y, g, function(v) c(n = length(v), M = mean(v), SD = sd(v)))
# Result: 17 / 18 / 50 / 32 participants; M 3.51, 3.21, 3.06, 3.51; SD 0.67, 0.81, 0.66, 0.81

# Levene's test around the mean – as Based on Mean in SPSS
leveneTest(y ~ g, center = mean)                         # F(3, 113) = 0.218, p = .884

# Table 2 – the ANOVA table, and the Welch row
m <- aov(y ~ g); summary(m)                              # F(3, 113) = 3.21, p = .026; SSB 5.08, SSW 59.51
oneway.test(y ~ g)                                       # Welch F(3, 43.82) = 3.26, p = .030

# Effect size: eta squared = SSB/SST, omega squared = (SSB − dfB·MSW)/(SST + MSW)
s <- summary(m)[[1]]; ssb <- s[["Sum Sq"]][1]; ssw <- s[["Sum Sq"]][2]; msw <- s[["Mean Sq"]][2]; dfb <- s[["Df"]][1]
ssb / (ssb + ssw)                                        # eta squared = .079
(ssb - dfb * msw) / (ssb + ssw + msw)                    # omega squared = .054

# Table 3 – post hoc tests: Tukey HSD (Tukey-Kramer) and Bonferroni
TukeyHSD(m)                                              # pair graduate degree–bachelor's degree: difference 0.448, p = .037; the other pairs not significant
pairwise.t.test(y, g, p.adjust.method = "bonferroni")    # pair graduate degree–bachelor's degree: p = .044

Reporting the results

Report the finding and then the statistics: the mean and standard deviation of each group (in the text, in a table or in a chart), F with the two degrees of freedom in parentheses, an exact p, the effect size, and then the post hoc tests – which pairs differed (Gravetter & Wallnau, 2017, pp. 389-390; Field, 2024, Section 12.12; Pallant, 2020, Chapter 18; American Psychological Association [APA], 2020, Section 6.43). The example of Gravetter and Wallnau: "The analysis of variance revealed significant differences among the three learning strategies, F(2, 15) = 7.16, p = .007, η² = 0.488" – and when an exact p is available from the computer, report it, not p < .05. Field's example, with Welch and ω²: "There was no significant effect of puppy therapy on levels of happiness, F(2, 7.94) = 4.32, p = .054, ω² = .35 [−.15, .59]" – and note that he reports both the non-significant F and the considerable effect size. Pallant's example: "There was a statistically significant difference at the p < .05 level in LOT scores for the three age groups: F(2, 432) = 4.64, p = .01. Despite reaching statistical significance, the actual difference in mean scores between the groups was quite small; the effect size, calculated using eta squared, was .02. Post hoc comparisons using the Tukey HSD test indicated that the mean score for Group 1 (M = 21.36, SD = 4.55) was significantly different from Group 3 (M = 22.96, SD = 4.49); Group 2 (M = 22.10, SD = 4.15) did not differ significantly from either of them." For a series of analyses – an APA table with n, M, SD for each group, F, p and η² (Pallant).

The formatting rules (APA, 2020, Section 6.36): F with a zero before the decimal point (it can exceed 1), p, η² and ω² without (they cannot); non-integer Welch degrees of freedom – with two decimals. The report sentences in the output are built this way, in English, with the post hoc tests of the method selected in the toggle.

Common mistakes

A series of t-tests instead of ANOVA. Four groups mean six t-tests – and the overall risk of a Type I error climbs well above 5% (Gravetter & Wallnau, 2017, pp. 369-370; Field, 2024, Section 2.10.8).

Reporting "there is a difference" without saying between whom. A significant F says only that at least one pair differed; without post hoc tests (or planned contrasts) the finding is incomplete (Gravetter & Wallnau, p. 393).

Interpreting post hoc tests when F is not significant – or choosing a method after seeing the results. Decide in advance (Pallant, 2020, chapter 18; Field, section 12.6 – "it's a bit like p-hacking, only you correct the p").

Ignoring the Welch row. When Levene is significant, or when the groups differ greatly in size and spread, the regular F row can mislead in both directions (Field, Box 12.2); Field always reports Welch.

Reporting p without an effect size. In a large sample a tiny difference comes out significant – Pallant's example with η² = .02; Field reports ω² even when F is not significant.

One-way ANOVA on repeated measures. The same participants at three time points violate independence (Field, Section 12.4.2) – repeated-measures ANOVA.

Relying on the means chart. An axis that does not start at zero inflates tiny differences (Pallant, Chapter 18), and overlapping confidence intervals do not mean "not significant" (Field, Section 12.9.1).

References

American Psychological Association. (2020). Publication manual of the American Psychological Association (7th ed.). https://doi.org/10.1037/0000165-000

Cohen, J. (1988). Statistical power analysis for the behavioral sciences (2nd ed.). Lawrence Erlbaum Associates.

Field, A. (2024). Discovering statistics using IBM SPSS statistics (6th ed.). SAGE.

Gravetter, F. J., & Wallnau, L. B. (2017). Statistics for the behavioral sciences (10th ed.). Cengage Learning.

Pallant, J. (2020). SPSS survival manual: A step by step guide to data analysis using IBM SPSS (7th ed.). Routledge.

Welch (1951), Brown and Forsythe (1974), Toothaker (1993), Kirk (1996), Scariano and Davenport (1987), Wilcox (2017) and Weinstein, McDermott and Roediger (2010) – as cited in Field (2024) and in Gravetter and Wallnau (2017). According to the site rule, the reference list includes only sources that were read in the original. Field (2024) and Pallant (2020) in a reflowable digital copy – cited by section and chapter; Gravetter and Wallnau (2017), Cohen (1988) and APA (2020) – by page and section of the printed edition.