Paired-samples t-test

Loading the data

Paste columns directly from the sheet (including the header row) – select them in Excel, copy, and paste here:

Selecting the two measurements

When to use?

When each participant has two measurements of the same kind and the question is whether the mean changed: the same questionnaire before and after an intervention, two conditions that each participant went through, two items on the same scale rated by the same people – or matched pairs, in which each participant in one group is matched one-to-one to a participant in the other group (twins, spouses, or matching by age and sex). Pallant (2020, chapter 17) defines: one group of participants, two sets of scores on the same continuous variable – and one categorical independent variable with two levels (Time 1 / Time 2). Field (2024, section 10.5) presents the "confusion machine": paired-samples t-test, dependent t-test, repeated-measures test, matched-pairs test – all the same test. Gravetter and Wallnau (2017, pp. 336-338) call the two designs – repeated measures and matched pairs – by the common name related-samples designs, and stress that they are statistically equivalent: the data are pairs of scores, and what is tested is the difference within each pair.

The unit tested is the difference within the participant (or within the pair) – so this is a different test from the independent-samples t-test, in which each group consists of different people. More than two measurements of the same participants – repeated-measures analysis of variance (Pallant, Chapter 18; Gravetter & Wallnau, Chapter 13), on the Repeated-measures ANOVA page, not a series of paired t-tests; when the differences severely violate the normality assumption in a small sample – the Wilcoxon test (Pallant, 2020, Chapter 16; Gravetter & Wallnau, Appendix E), on its own page. A multi-item scale score (mean of items, the page Cronbach's alpha and scale score) is treated as an interval variable per the site convention (the page Variables and levels of measurement), and is therefore suitable as a measurement; a single Likert item – not.

The logic: the differences, not the scores

The first step is to compute for each participant the difference between the two measurements – D (Gravetter & Wallnau, 2017, pp. 339-340, Equation 11.1). From here on, the test does not see the original scores but only the differences: their mean, M_D, is the estimate of the effect; their standard deviation, s_D, tells how consistent the effect is across participants. The question is whether the mean of the differences in the population, μ_D, is zero. Field (2024, section 10.5.2, Equations 10.6-10.7): t is the mean of the differences divided by the standard error of the differences (s_D/√N) – "signal over noise": the observed difference relative to what is reasonable to get from two random samples from the same population. Degrees of freedom: df = n − 1, where the unit is the pair (Gravetter & Wallnau, pp. 341-342).

Why the paired design is more powerful. Gravetter and Wallnau (pp. 353-354, Table 11.4) show this on exactly the same numbers: three participants, a mean difference of 5 points. As an independent-samples test – the variance within each condition includes the large differences between people (Bill 33, John 18), standard error 5.77, t = 0.87, not significant. As repeated measures – the differences between people disappear in the difference scores (each one dropped by a few points), standard error 1.15, t = −4.35, significant. Each participant serves as their own control. Field (section 10.9.4) puts the same idea through the correlation in Table 2: the more correlated the measurements, the smaller the standard deviation of the differences and the more sensitive the test.

How to read the output?

The three tables are the three SPSS tables (Pallant, 2020, Chapter 17; Field, 2024, Section 10.9.4):

Table 1 – descriptive statistics of the two measurements (Paired Samples Statistics): the mean, N, standard deviation and standard error of each measurement, on the participants who have both. Pallant begins "Step 2" here: after a significant difference has been found, this table shows in which direction – which measurement is higher. Field notes that the numbers here are identical to those of the same data in an independent design; what changes is Table 3.

Table 2 – the correlation (Paired Samples Correlations): the Pearson correlation between the two measurements with p. In Field (section 10.9.4) it is high and significant in the invisibility cloak example (r = .806, p = .002) – as expected, because the two measurements came from the same people and people are consistent with themselves. In Pallant (chapter 17), r = .862 between fear of statistics at Time 1 and Time 2. A low correlation is not an error, but it says that the paired design gains almost nothing (Gravetter & Wallnau, p. 354) – and sometimes that the pairing in the file is wrong.

Table 3 – the differences (Paired Samples Test): the mean of the differences (M_D; first minus second, as in SPSS), the standard deviation of the differences, their standard error, a 95% confidence interval for the difference, and then t, df and two-tailed p. Pallant (chapter 17, "Step 1"): first look at the Sig. (2-tailed) column – smaller than .05, there is a significant difference; record t, df and the difference with the confidence interval. Field (section 10.9.4) breaks down the numbers: t = −1.25/0.329 = −3.804, df = N − 1 = 11, p = .003 – "only a 0.3% chance of such a t if the mean of the differences in the population is zero". The sign of t and of the difference depends only on the order you chose: if the second measurement is higher, the difference is negative. The confidence interval. The range in which, at a 95% confidence level, the population mean of the differences lies (Gravetter & Wallnau, pp. 348-349, Equation 11.5): when it includes zero the test is not significant, and vice versa – the same decision in two languages. The smaller the sample, the wider the interval, so it is a description of uncertainty and not a pure measure of effect size (p. 349).

Effect size

Significance is not size: in a large sample even a tiny change comes out significant (Field, 2024, Section 3.7.3). The table prints three measures, because the three sources report different measures – and the page shows all of them:

What the difference is in practice: d_z measures the consistency of change within people, and therefore depends on the correlation – a high correlation shrinks the standard deviation of the differences and inflates it; the corrected d and eta squared are closer to what would be obtained in an independent design. The "Interpreting the result" box in the output shows all three and says whether they tell the same story. Gravetter and Wallnau (pp. 350-351, Figures 11.3-11.4) illustrate: the same mean difference of 2 points with s = 2 is a large and significant effect, and with s = 6 – when some go up and some go down – it is not significant: "large variance obscures patterns".

Assumptions and warnings

Two assumptions (Gravetter & Wallnau, 2017, p. 346): the observations within each condition are independent of each other (different people), and the distribution of the differences in the population is normal. Field (2024, section 10.9.2) is precise: what matters is the distribution of the differences, not the scores themselves – "two clearly non-normal measurements can yield differences that are nicely distributed"; so if normality is checked, compute the difference column and check it (the page Frequencies and descriptive statistics; outliers – the page Outliers and assumptions). Pallant (chapter 17) and Gravetter and Wallnau (p. 346) agree that in a sample of 30 pairs or more the violation is not worrying; in a small sample with non-normal differences – Wilcoxon, or bootstrap confidence intervals (Field, Section 10.9.5) and a robust t-test (Yuen) – which are not on the page.

The pairing must be correct. The row is the participant: the two measurements in the same row belong to the same person (or the same matched pair). A file re-sorted by one column only destroys the pairing without any error message – the correlation in Table 2 is the sanity check: a near-zero correlation between "before" and "after" of the same questionnaire is almost always a data problem.

Missing values. A participant missing one of the two measurements is not included at all – there is no difference for them (/MISSING=ANALYSIS, the SPSS default); the page reports how many dropped out. In before-after this is dropout (attrition): if those who dropped out differ from those who stayed, the comparison is biased. What to do about missing values – the page Missing values.

Order, time and practice. Gravetter and Wallnau (pp. 354-355): the drawback of repeated measures is that factors that change over time – mood, health, weather – and experience with the first measurement (practice effect, order effect) can change the second score regardless of the treatment. The design remedy: counterbalancing – half the participants in order A-B and half in B-A; when strong time or order effects are expected, an independent design is preferable. Pallant (chapter 17, "Caution"): a significant before-after difference is not proof that the intervention caused it – the passage of time and other events can explain it; a control group that did not receive the intervention would improve the design.

The confidence intervals in the chart. Field (section 10.9.2) warns: confidence intervals of each measurement separately ignore the pairing, and in repeated measures they can overlap even when the difference is highly significant (he suggests a correction based on the participant's mean). The chart on the page shows them as a description of each measurement; the interval that decides is the confidence interval for the difference in Table 3.

What the page does not compute. A confidence interval for d (SPSS 27 and later prints it – Field, section 10.9.6), bootstrap and a robust t-test (Yuen – Field, Section 10.9.5), a one-tailed test (Gravetter & Wallnau, pp. 345-346, Example 11.3 – the two-stage procedure), repeated-measures analysis of variance, the Bayes factor (Field, Section 10.9.7), and several pairs in one click (SPSS allows it; here one pair at a time – and when several are run, each test carries a 5% risk of a Type I error). The nonparametric alternative, the Wilcoxon signed-rank test – on its own page.

And how do you do it in SPSS?

A note on versions. The procedure is stable; from version 27 the dialog includes Estimate effect sizes and prints a Paired Samples Effect Sizes table with Cohen's d and a confidence interval, which also includes the choice of the Standardizer: the standard deviation of the differences (d_z, the default), corrected for the correlation (Field's d, section 10.9.3), or the average of the variances. Pallant (2020, based on version 26) writes that SPSS "does not provide an effect size for t-tests" and computes eta squared by hand. In older versions – delete the /ES line from the syntax.

The procedure (Pallant, 2020, Chapter 17; Field, 2024, Section 10.9.3): Analyze › Compare Means › Paired-Samples T Test… – select the two variables (the first, and then with Ctrl the second) and move them together to the Paired Variables box; the order determines the direction of the difference (Variable1 − Variable2). Additional pairs can be added in the same dialog. Options… – the confidence interval level (95%) and the handling of missing values; Bootstrap… – BCa confidence intervals that do not assume normality (Field, Section 10.9.5). The syntax:

T-TEST PAIRS=attitude_score WITH attitude_score_2 (PAIRED)
  /ES DISPLAY(TRUE) STANDARDIZER(SD)
  /CRITERIA=CI(.9500)
  /MISSING=ANALYSIS.

Reading the output. Paired Samples Statistics = Table 1 (Mean, N, Std. Deviation, Std. Error Mean); Paired Samples Correlations = Table 2 (N, Correlation, Sig.; in SPSS 27 and later Two-Sided p); Paired Samples Test = Table 3 – under Paired Differences: Mean, Std. Deviation, Std. Error Mean, 95% Confidence Interval of the Difference (Lower, Upper), and then t, df, Sig. (2-tailed) (in 27 and later One-Sided p and Two-Sided p – report the two-sided one); Paired Samples Effect Sizes – the Cohen's d row, Point Estimate column, is the d_z in the table note (Hedges' correction – not on the page). The "Download SPSS syntax" button in the output generates the loading, the dictionary (if there was a legend) and a T-TEST PAIRS command on the selected pair.

And how do you do it in R?

In base R, t.test(x, y, paired = TRUE) gives Table 3 (the difference x − y), and cor.test(x, y) gives Table 2. R itself ignores rows in which one of the measurements is missing (with paired = TRUE it omits the pair). d_z, the corrected d and η² are computed by hand from the formulas in the guide. The script was run as is on the sample data file and gave the table values (in the comments). The site rule for R: a UTF-8 file, every comment line starts with an English word, and a UTF-8 locale for Hebrew names (on Windows: R 4.2 and later, or Sys.setlocale('LC_ALL', 'Hebrew')).

# Paired-samples t-test – Professor Stat. This file is saved in UTF-8; every comment line starts with an English word
library(readxl)   # reading Excel

# Loading: the standard template – row = participant, column = variable; the sheet "data"
d <- read_excel("C:/Rdata/DATA_tpaired_example_eng.xlsx", sheet = "data")
ok <- complete.cases(d[["attitude_score"]], d[["attitude_score_2"]])   # Only participants with both measurements (113 of 120)
x <- d[["attitude_score"]][ok]; y <- d[["attitude_score_2"]][ok]

# Table 1 – statistics of the two measurements
rbind(first = c(n = length(x), M = mean(x), SD = sd(x)), second = c(n = length(y), M = mean(y), SD = sd(y)))
# Result: first n 113, M 3.262, SD 0.754; second n 113, M 3.412, SD 0.731

# Table 2 – the correlation between the measurements
cor.test(x, y)                       # r = .852, p < .001

# Table 3 – paired-samples t-test (difference = x − y)
t.test(x, y, paired = TRUE)          # t = -3.94, df = 112, p < .001, difference -0.150, confidence interval [-0.226, -0.075]

# Effect size: dz, d corrected for the correlation, and eta squared = t²/(t² + df)
D <- x - y; r <- cor(x, y)
mean(D) / sd(D)                                         # dz = -0.370
mean(D) / (sd(D) / sqrt(2 * (1 - r)))                   # Corrected d = -0.202
tt <- t.test(x, y, paired = TRUE); unname(tt$statistic^2 / (tt$statistic^2 + tt$parameter))   # Eta squared = .122

Reporting the results

Report the finding and then the statistics: the mean and standard deviation (or standard error) of each measurement, t with the degrees of freedom in parentheses, an exact p, the difference with the confidence interval, and the effect size (Field, 2024, Section 10.10; Gravetter & Wallnau, 2017, p. 349; American Psychological Association [APA], 2020, Section 6.43). Pallant's example (2020, chapter 17): "There was a statistically significant decrease in FOST scores from Time 1 (M = 40.17, SD = 5.16) to Time 2 (M = 37.5, SD = 5.15), t(29) = 5.39, p < .001 (two-tailed). The mean decrease was 2.67, with a 95% confidence interval from 1.66 to 3.68. The eta squared statistic (.50) indicated a large effect size." Field's example (section 10.10.1), on the same data as a paired design: "The difference, −1.25, BCa 95% CI [−1.75, −0.75], was significant, t(11) = −3.80, p = .003, and represented an effect of d = 0.68 [−1.19, −0.18]". Gravetter and Wallnau (p. 349): "Changing from a neutral word to a swear word reduced perceived pain by an average of M = 2.00 points, SD = 2.00. The effect was significant, t(8) = −3.00, p = .017, r² = 0.529" – and when an exact p is available from the computer, report it and not p < .05.

Formatting rules (APA, 2020, Section 6.36): t, the difference and d with a zero before the decimal point (they can exceed 1), p, r and η² without (they cannot). The report sentences in the output are built this way, in English, with d_z; whoever reports Field's corrected d or eta squared – replaces the measure from the table note and says which measure it is.

Common mistakes

An independent-samples test on repeated measures. Before-after of the same people violates independence; an independent-samples test on the same data also loses the advantage of pairing – t = 0.87 instead of −4.35 in the example of Gravetter and Wallnau (pp. 353-354).

A paired test on different people. The reverse: two separate groups pasted side by side in "before" and "after" columns are not pairs; the differences are meaningless. The question "who is in the row" has one answer – the same participant (or the same matched pair).

Interpreting change as causality. Without a control group, the time that passed, other events and practice are alternative explanations (Pallant, 2020, Chapter 17; Gravetter & Wallnau, 2017, p. 354).

Reporting d without saying which. d_z and the corrected d can differ by a factor of two on the same data (−0.37 vs. −0.20 in the sample data file); state the standardizer, and do not compare d_z with the d of an independent design.

Checking normality of the measurements instead of the differences. The assumption is about the differences (Field, 2024, Section 10.9.2).

Interpreting "not significant" as "no change". A confidence interval that includes zero says that the true change may be negative, positive or zero – not that it is zero. Report d_z and the interval.

A series of t-tests on three or more measurements. Three time points call for repeated-measures analysis of variance (Repeated-measures ANOVA), not three t-tests.

References

American Psychological Association. (2020). Publication manual of the American Psychological Association (7th ed.). https://doi.org/10.1037/0000165-000

Cohen, J. (1988). Statistical power analysis for the behavioral sciences (2nd ed.). Lawrence Erlbaum Associates.

Field, A. (2024). Discovering statistics using IBM SPSS statistics (6th ed.). SAGE.

Gravetter, F. J., & Wallnau, L. B. (2017). Statistics for the behavioral sciences (10th ed.). Cengage Learning.

Pallant, J. (2020). SPSS survival manual: A step by step guide to data analysis using IBM SPSS (7th ed.). Routledge.

Student (1908), Yuen (1974) and Stephens, Atkins and Kingston (2009) – as cited in Field (2024) and in Gravetter and Wallnau (2017). Per the site rule, the reference list includes only sources read in the original. Field (2024) and Pallant (2020) in a reflowable digital copy – cited by section and chapter; Gravetter and Wallnau (2017), Cohen (1988) and APA (2020) – by page and section of the printed edition.