Independent-samples t-test

Loading the data

Paste columns directly from the sheet (including the header row) – select them in Excel, copy, and paste here:

Variable selection

When to use?

When you want to know whether the mean of a quantitative variable differs between two separate groups of participants – an experimental group vs. a control group, men vs. women, customers vs. non-customers. Pallant (2020, chapter 17) defines the recipe: one categorical independent variable with exactly two groups, and one quantitative dependent variable; each participant belongs to one group only. Field (2024, section 10.5) distinguishes between two "packages" of the same statistic: the independent t-test – when the observations come from different entities; and the paired t-test – when the same participants (or matched pairs) are measured twice. Field adds that the independent t-test is a special case of the linear model: a regression with a binary predictor, in which the coefficient b is the difference between the two means (section 10.4) – the same t, the same p.

For more than two groups – one-way analysis of variance (ANOVA), not a series of t-tests (section "Assumptions and warnings"); for the same participants on two measures – the paired t-test; when the dependent variable is ordinal or severely violates the assumptions – Mann-Whitney (Pallant, 2020, Chapter 16), which is not on this page. A multi-item scale score (mean of items, the Cronbach's alpha and scale score page) is treated as an interval variable according to the site convention (the variables and levels of measurement page), and is therefore suitable as a dependent variable; a single Likert item is not.

The logic: signal divided by noise

Field (2024, section 10.5.1): the two samples give two means, and they differ from each other by a little or by a lot. If both came from the same population, the difference between them stems from sampling fluctuation only, and it is measured by the standard error of the difference; t is the observed difference divided by this standard error – "signal divided by noise". The larger the difference relative to the standard error, the harder it is to believe it is due to chance. The p value is the probability of obtaining such a t or a more extreme one if the true difference in the population is zero.

The standard error of the difference is built from the variances of the two groups. When the groups differ in size, the variances are pooled (pooled variance): a weighted mean of the two variances, where the weight of each group is its degrees of freedom (n − 1), because a large sample estimates the population better (Field, 2024, Section 10.5.3, Equation 10.15; Gravetter & Wallnau, 2017, pp. 305-307). The degrees of freedom of the test: df = n₁ + n₂ − 2. This is the "equal variances" row in the table. The second row, Welch, does not pool the variances but uses each one separately, and corrects the degrees of freedom downward (so they are usually not whole numbers – 21.54 in Field's example, section 10.8.3); when the variances and sizes are equal, the two rows are identical.

How to read the output?

Table 1 – Group statistics. Pallant (2020, chapter 17) starts here: check that the N of each group makes sense – an empty group or one much smaller than expected usually indicates a wrong code (0 and 1 instead of 1 and 2) or many missing values. Alongside the mean and standard deviation – the standard error of the mean, from which the confidence intervals in the chart are built.

Table 2 – The test, in two rows. This is SPSS's Independent Samples Test table. The first row assumes equal variances, the second does not (Welch), and the first row also prints Levene's test for equality of variances. Which row do you read? Here the sources are divided, and the page presents both positions:

In practice the two rows almost always give the same conclusion, and the "Interpreting the result" box in the output says whether they agree. When they disagree – usually groups of very different sizes with very different variances – Gravetter and Wallnau (2017, pp. 313-314) explain why the pooled-variance row cannot be trusted: if the two variances estimate different things, their average is meaningless, and an extreme t may stem from it and not from the difference in means; in such a situation the Welch row is the safe report, regardless of the rule you chose.

t, df and p. Two-tailed p (Sig. 2-tailed): smaller than .05 – the difference is significant; larger – no significant difference was found (Pallant, 2020, Chapter 17). The sign of t depends only on which group was chosen as "Group A": the difference is computed A minus B (Field, 2024, Section 10.8.3, Note 5). The confidence interval for the difference. The range in which, at a 95% confidence level, the difference in means in the population lies (Gravetter & Wallnau, 2017, pp. 319-320). When it includes zero, the test is not significant – and vice versa; Gravetter and Wallnau (p. 320) show that this is the same decision in two languages. The confidence interval is wider the smaller the sample, and so it is a description of uncertainty and not a pure measure of effect size.

Effect size

Significance is not size: in a large enough sample even a tiny difference comes out significant, and in a small sample even a large difference does not (Field, 2024, section 3.7.3 – the two examples in Figures 2.21 and 2.22). That is why the table prints two measures that do not depend on sample size:

Cohen's d – the difference between the means in standard-deviation units: "if the person sings instead of talking, the time the listener stays drops by 0.667 standard deviations" (Field, Section 3.7.3). The page divides the difference by the square root of the pooled variance, as do Gravetter and Wallnau (2017, p. 317, formula 10.8) and as does Cohen's d in the SPSS output; Field (section 3.7.3) mentions an alternative – the standard deviation of the control group, when the treatment may also change the spread. Cohen's conventions (1988, pp. 25-27): 0.2 small, 0.5 medium ("visible to the naked eye"), 0.8 large – and with them the warning of Cohen himself, which Field cites: the thresholds encourage lazy thinking, and the effect is judged in the context of the field and the measurement instrument. d is usually reported as a positive value (Gravetter & Wallnau, p. 317).

Eta squared (η², in Gravetter and Wallnau r²) – the proportion of variance in the dependent variable explained by group membership: t²/(t² + df) (Gravetter & Wallnau, 2017, pp. 317-318, Equation 10.9; Pallant, 2020, Chapter 17). Pallant gives Cohen's thresholds (1988, pp. 284-287): .01 small, .06 medium, .14 large – and warns not to confuse the yardsticks: d = 0.5 is medium, η² = .06 is medium, and these are different numbers for the same effect. The page computes η² from the equal-variances row, as in Pallant's example.

Assumptions and warnings

The three assumptions (Gravetter & Wallnau, 2017, p. 313): the observations in each group are independent of one another; the two populations are normally distributed; and both have the same variance (homogeneity). Independence is the assumption that has no fix – a participant measured twice, pairs, or students from the same classes violate it, and then the correct test is a different one (paired, or a model that takes the structure into account). Normality is "less important, especially in large samples" (Gravetter & Wallnau, p. 313); Field (2024, section 10.6) refers to chapter 6: what is required is a normal sampling distribution of the difference, and the central limit theorem guarantees it in a reasonable sample – so normality is not tested with a significance test (Box 6.7); instead, look at the histogram, the skewness and the kurtosis of each group (the Normality testing page – by group; the Frequencies and descriptive statistics page) and at outliers (the Outliers and assumptions page).

Homogeneity of variances. Gravetter and Wallnau (pp. 313-314): the assumption matters mainly when there is a large gap in group sizes, and a rule of thumb before any test – if one variance is three to four times larger than the other, there is cause for concern. Field (section 6.9.6) adds that with groups of equal size the assumption is "almost irrelevant". The page prints Levene's test as SPSS does (around the mean – this is how it appears in the SPSS t-test output; Explore prints several versions of the test, around the mean and around the median), and both rows; the "How to read" section explains the two positions.

Missing values. A participant missing a value on the dependent variable or on the grouping variable is excluded from the comparison of that dependent variable only – analysis-by-analysis deletion (/MISSING=ANALYSIS in the syntax), as in SPSS's default; therefore N can change from one dependent variable to the next. The groups that are not selected (in a grouping variable with more than two levels) are simply not included. What to do about missing values – the Missing values page.

Multiple tests. Several dependent variables in one click are several tests, and each carries a 5% risk of a Type I error; Field (2024, section 10.8.2) notes that multiple t-tests inflate the error rate, and for comparing more than two groups ANOVA is used, not a series of t-tests. When several tests are reported – report all of them, including the non-significant ones, and consider a correction (the correlations page: Holm / FDR).

What this page does not compute. A confidence interval for d (SPSS 27 and later prints it – Field, section 10.8.5), bootstrap confidence intervals and a robust t-test (Yuen – Field, Section 10.8.4), a one-tailed test (Field, section 10.8.3: "in most situations, ignore the one-tailed value"), and the Bayes factor. The one-sample t-test – on its own page; the nonparametric alternative, Mann-Whitney – likewise.

And how do you do it in SPSS?

Version note. The procedure is stable across versions; one difference you will encounter on this page: from version 27 the dialog includes the Estimate effect sizes check box and prints an Independent Samples Effect Sizes table with Cohen's d and a confidence interval (Field, 2024, sections 10.8.2 and 10.8.5 – based on version 29), whereas Pallant (2020, based on version 26) writes that SPSS "does not provide effect sizes for t-tests" and calculates eta squared by hand. In old versions – delete the /ES DISPLAY(TRUE) line from the syntax the page generates.

The procedure (Pallant, 2020, Chapter 17; Field, 2024, Section 10.8.2): Analyze › Compare Means › Independent-Samples T Test… – the dependent variable (or several) into the Test Variable(s) box, the grouping variable into Grouping Variable, and in the Define Groups… button type the two codes of the groups (1 and 2 in the sample file; if you do not remember – right-click on the variable name and Variable Information). Field mentions the Cut point option – splitting into two groups by a cutoff value – and warns against a median split (Box 10.1). Options… – the confidence-interval level (95%) and the handling of missing values; Bootstrap… – confidence intervals that do not assume normality (BCa). The syntax:

T-TEST GROUPS=group(1 2)
  /MISSING=ANALYSIS
  /VARIABLES=attitude_score age
  /ES DISPLAY(TRUE)
  /CRITERIA=CI(.95).

Reading the output. Group Statistics = Table 1 (N, Mean, Std. Deviation, Std. Error Mean); Independent Samples Test = Table 2 – Levene's Test (F, Sig.) in the first row, and then in both rows t, df, Sig. (2-tailed) (in SPSS 27 and later also One-Sided p – ignore it), Mean Difference, Std. Error Difference, 95% Confidence Interval of the Difference; Independent Samples Effect Sizes – the Cohen's d row, Point Estimate column, is the d in the table note (SPSS also prints Hedges' correction and Glass's delta, which are not on the page). The "Download SPSS syntax" button in the output generates the loading, the dictionary (if there was a legend) and the T-TEST command on the selected variables and codes.

And how do you do it in R?

In base R, t.test() runs Welch by default; var.equal = TRUE gives the equal-variances row. Levene's test around the mean – leveneTest(center = mean) from the car package (its default is the median, and that is not what SPSS prints in the t-test). d and η² are computed by hand from the formulas in the guide. The script was run verbatim on the sample file and gave the values in the tables (in the comments). The site rule for Hebrew in R: a UTF-8 file, every comment line starts with an English word, and a UTF-8 locale for Hebrew names (on Windows: R 4.2 and later, or Sys.setlocale('LC_ALL', 'Hebrew')).

# Independent-samples t-test – Professor Stat. This file is saved in UTF-8; every comment line starts with an English word
library(readxl)   # reading Excel
library(car)      # Levene's test: leveneTest

# Loading: the standard template – row = participant, column = variable; the sheet "data"
d <- read_excel("C:/Rdata/DATA_ttest_example_eng.xlsx", sheet = "data")
g <- factor(d[["group"]], levels = c(1, 2), labels = c("experimental", "control"))
y <- d[["attitude_score"]]

# Group statistics – Table 1
tapply(y, g, function(v) c(n = sum(!is.na(v)), M = mean(v, na.rm = TRUE), SD = sd(v, na.rm = TRUE)))
# Result: experimental n 60, M 3.474, SD 0.682; control n 60, M 3.065, SD 0.754

# Levene's test around the mean – as in the SPSS t-test output
leveneTest(y ~ g, center = mean)                         # F = 0.387, p = .535

# Row 1 – equal variances; row 2 – Welch (the R default)
t.test(y ~ g, var.equal = TRUE)    # t = 3.11, df = 118, p = .0023, difference 0.409, CI [0.149, 0.669]
t.test(y ~ g)                      # t = 3.11, df = 116.8, p = .0024

# Effect size: Cohen's d on the pooled variance, and eta squared = t²/(t² + df)
a <- y[g == "experimental" & !is.na(y)]; b <- y[g == "control" & !is.na(y)]
sp2 <- ((length(a) - 1) * var(a) + (length(b) - 1) * var(b)) / (length(a) + length(b) - 2)
(mean(a) - mean(b)) / sqrt(sp2)                          # d = 0.568
tt <- t.test(a, b, var.equal = TRUE); unname(tt$statistic^2 / (tt$statistic^2 + tt$parameter))   # eta squared = .076

Reporting the results

Report the finding and then the statistics: the mean and standard deviation (or standard error) of each group, t with the degrees of freedom in parentheses, an exact p, the difference with its confidence interval, and the effect size (Field, 2024, Section 10.10; Gravetter & Wallnau, 2017, p. 321; American Psychological Association [APA], 2020, Section 6.43). Field's example (section 10.10.1): "On average, participants given a cloak of invisibility engaged in more acts of mischief (M = 5, SE = 0.48) than those not given a cloak (M = 3.75, SE = 0.55). This difference, −1.25, 95% CI [−2.70, 0.07], was not significant, t(21.54) = −1.71, p = .101; however, it did represent an effect of d = 0.70 [−1.52, 0.13]" – note that the conclusion "not significant" and "a considerable effect" sit in the same sentence, and both are true. Gravetter and Wallnau (p. 321): when an exact p is available from the computer, report it (p = .018, not p < .05), and the confidence interval comes right after the test. Pallant (2020, chapter 17) reports "two-tailed" and eta squared, and for a series of tests – an APA table with n, M, SD for each group, df, t, p, the difference with its confidence interval, and the effect size.

The formatting rules (APA, 2020, Section 6.36): t and the mean difference with a zero before the decimal point (they can exceed 1), p and η² without (they cannot), d with a zero; non-integer Welch degrees of freedom – with two decimals (t(21.54)). The report sentences in the output are built this way, in English.

Common mistakes

Reading the wrong row – or reading both and choosing the convenient one. Set a rule in advance (Levene, or "always Welch") and report according to it; when the rows disagree, the Welch row is the safe one (Gravetter & Wallnau, 2017, pp. 313-314).

Interpreting "not significant" as "no difference". A confidence interval that includes zero says that the true difference can be negative, positive or zero (Field, 2024, Section 10.8.4) – not that it is zero. Report d and the interval.

Reporting p without means, standard deviations and effect size. The reader needs to know not only whether there is a difference but how large it is (Field, 2024, Section 3.7.6; Pallant, 2020, Chapter 17).

A series of t-tests instead of ANOVA. Three groups mean three t-tests, and each additional test inflates the Type I error (Field, 2024, Section 10.8.2).

An independent test on paired data. Before-after of the same people, pairs, siblings – violate independence; switch to the paired-samples t-test (Field, 2024, Section 10.5).

Confusing the yardsticks of d and eta squared. 0.5 is a medium d; .06 is a medium η² (Pallant, 2020, Chapter 17).

Splitting a continuous variable into two groups at the median in order to run a t-test. Power and information are lost (Field, 2024, Box 10.1); keep the variable continuous and use correlation or regression.

References

American Psychological Association. (2020). Publication manual of the American Psychological Association (7th ed.). https://doi.org/10.1037/0000165-000

Cohen, J. (1988). Statistical power analysis for the behavioral sciences (2nd ed.). Lawrence Erlbaum Associates.

Field, A. (2024). Discovering statistics using IBM SPSS statistics (6th ed.). SAGE.

Gravetter, F. J., & Wallnau, L. B. (2017). Statistics for the behavioral sciences (10th ed.). Cengage Learning.

Pallant, J. (2020). SPSS survival manual: A step by step guide to data analysis using IBM SPSS (7th ed.). Routledge.

Student (1908), Levene (1960), Yuen (1974) and Zimmerman (2004) – as cited in Field (2024). According to the site rule, the reference list includes only sources that were read in the original. Field (2024) and Pallant (2020) in a reflowable digital copy – cited by section and chapter; Gravetter and Wallnau (2017), Cohen (1988) and APA (2020) – by page and section of the printed edition.