Reliability: Cronbach's alpha and scale score

Loading the data

Paste columns directly from the sheet (including the header row) – select them in Excel, copy, and paste here:

Selecting the scale items and defining the score

Why check reliability?

A questionnaire that measures a single construct – satisfaction, commitment, anxiety – does so with several items, from which a single score is computed. Before using this score in an analysis you need to know that the items really "agree with each other": that someone who is high on the scale as a whole is also high on any set of items chosen from it at random (Field, 2024, Section 18.9.1). This is internal consistency, one of the types of reliability explained on the Reliability and validity: the concepts page; when the measuring instrument is people who code – intercoder reliability is measured instead.

Formally: each participant's observed score is a true score plus measurement error, X = T + E, and reliability is the proportion of the variance in the observed scores that comes from the variance of the true scores (Field, 2024, Equations 18.9-18.12) – or, equivalently, the squared correlation between the true and the observed scores. Field stresses a point that is easy to forget: reliability is not a property of the questionnaire but of the questionnaire, the people who answered it and the context – a scale found reliable in one sample is not necessarily reliable in another. Therefore Pallant (2020, chapter 9) requires checking reliability in your own sample, even when the scale is well known and validated, and reporting it in the method section.

What is Cronbach's alpha?

The variance of the true scores cannot be measured directly. The long-standing practical solution is split-half reliability: the items are split into two random sets, a score is computed for each half, and the correlation between the halves is an estimate of reliability – one half serves as a parallel test for the other. The problem: there are many ways to split, and each gives a different number. Cronbach (1951, as cited in Field, 2024, Section 18.9.2) proposed a measure that is, approximately, the mean of all possible splits – and that is α. The formula the page computes:

α = k/(k − 1) · (1 − Σs_i²/s_T²)

k – the number of items; s_i² – the variance of each item; s_T² – the variance of the sum of the items. The idea: if the items measure the same thing, the variance of the sum is much larger than the sum of the variances (because the items vary together, and the covariances accumulate); if they are unrelated, the variance of the sum is roughly equal to the sum of the variances and α is close to zero. Field (section 18.9.2) presents the same formula in terms of the covariance matrix: the squared number of items times the average covariance, divided by the sum of all the elements in the matrix. α on standardized items (the second column in Table 1; "Cronbach's Alpha Based on Standardized Items" in SPSS) is the same formula applied to the correlation matrix instead of the covariances: k·r̄/(1 + (k − 1)·r̄), where r̄ is the average inter-item correlation. Field: the ordinary α is appropriate when the items are summed or averaged as they are (the usual case, and also what the page does for the score); the standardized one – only when the items are standardized before summing. Raykov (2024) sharpens this: the standardized one can be much higher or much lower than the reliability of the score that is actually analyzed, so it should be neither reported nor interpreted unless there is a strong reason to standardize in the first place – in Table 1 it is shown for comparison with SPSS, and the ordinary α is the one reported.

How to read the output?

Table 1 – α. Field (2024, section 18.11) looks for values "around .7-.8"; Pallant (2020, chapter 9) writes that the scale should be above .7 (DeVellis, 2012), and that above .8 is preferable. Both authors add in the same breath that general thresholds are a bad idea, because α depends on the number of items (see the assumptions section below) – so the table also shows the average inter-item correlation and its range beside it. Pallant: for short scales (fewer than 10 items) a fairly low α is common, and then the average correlation should be reported, whose optimal range is .2-.4 (Briggs & Cheek, 1986, as cited in Pallant, 2020). N in the table is the number of participants who answered all the items – listwise deletion, as in SPSS – and the "Excluded" column says how many participants were not included in the reliability computation (they can still receive a scale score, see below).

Table 2 – confidence intervals for α. The α in Table 1 is a sample estimate, and Raykov (2024) recommends reporting a confidence interval beside it. The table gives it at three levels (95%, 99%, 99.9%) and by three methods: ADF (asymptotically distribution-free; Maydeu-Olivares, Coffman & Hartmann, 2007) – a delta method in which the variance of the covariances is estimated from the fourth-order moments of the data themselves, without assuming normality or a model – this is the family Raykov (2024) calls "model-free and nearly assumption-free" (Raykov & Marcoulides, 2015, as cited in Raykov) and it is the default for reporting on the page (the bold row, 95%); normal theory (van Zyl, Neudecker & Nel, 2000, as cited in Maydeu-Olivares et al.) – the same delta method under the assumption of multivariate normality – and Feldt's F (1965, as cited in Maydeu-Olivares et al.) – an "exact" interval assuming parallel items and normality, which is also what psych::alpha prints. Maydeu-Olivares et al. (2007) found in simulations: with items close to normal the three methods are almost identical, and even in small samples the normal interval covers well; when skewness or kurtosis exceeds 1 the normal intervals are too narrow, and ADF is preferable – from N = 100 upward. Therefore the table note compares the two standard errors: when they differ by more than 20% – the data are far from normal, and the normal interval (and with it Feldt's) is not reliable. The limits must be read together with the threshold: an interval entirely above .7 is reassuring; an interval that crosses it means the sample is not decisive. An upper limit above 1 (this happens when α̂ is close to 1 with a symmetric interval) is shown as 1 and flagged.

Table 3 – item statistics. The last two columns are the ones to read. The corrected item-total correlation (Corrected Item-Total Correlation in SPSS) – the correlation between the item and the sum of the other items (the "correction": the item itself is not included in the sum it is correlated with). In a reliable scale all the items correlate well with the scale; Field and Pallant set the same threshold – below .3 the item measures something other than the scale as a whole. α if item deleted – what α would be without this item. In a good scale no item changes α by much, and all the values in the column are close to the overall α and below it; an item whose deletion raises α (yellow background) is a candidate for removal – but Pallant qualifies this: it is useful for someone developing a scale, whereas in a validated, well-known scale deleting an item prevents comparison with other studies that used it; Field adds that if items are deleted, the factor analysis should be rerun to make sure the structure holds; and Raykov (2024) goes further: when alpha is lower than the true reliability of the scale – a common situation – dropping an item on the basis of this statistic can lower both reliability and validity, so he generally recommends not using it as a guide for revising a scale. The site convention: the column stays in the output, as in SPSS, as a diagnostic – and not as a basis for removal (more on this on the concepts page, section 2). An item with a negative correlation (red background) is almost always a reversed item that was not reverse-scored – see the next section.

Table 4 – the correlation matrix. Pallant (2020, chapter 9) actually starts here: all the correlations between the items should be positive, because they all measure the same characteristic; a negative value indicates an item that was not reversed properly, and it also appears as a negative item-total correlation in Table 3. The mean of the correlations in Table 4 is the number in Table 1 (Summary Item Statistics in SPSS).

Reversed items

Many questionnaires include items worded in the opposite direction – "Community affairs do not interest me" among items that are agreed with when commitment is high. Field (2024, section 18.9.4) explains why they are there (they force the participant to read, and reduce response bias) and why they ruin α if they are not dealt with: the covariance between such an item and the rest is negative, it reduces the average covariance in the numerator of the formula, and α drops – even to a negative value, which has no meaning. His example (section 18.11, Output 18.14 versus 18.15): a subscale with one reversed item that was not reverse-scored – α = .537 instead of .707, a negative item-total correlation for that item, and an "α if item deleted" of .687. This is the telltale sign, and the page highlights it with a red background and a warning.

Reversing is simple: subtract the score from the scale maximum plus 1 – on a 1-5 scale, from 6 (Field, Section 18.9.4): 5 becomes 1, 1 becomes 5, and 3 stays 3. The page uses the general formulation (minimum + maximum) − value, which gives the same result on a scale that starts at 1 and is also correct for a 0-10 scale or a −3 to 3 scale; so it is important that the response range in the box is the possible range of the scale, not the range observed in the data. Marking "reversed item" in the list reverses the item before reliability and the score are computed; if the file has a codebook sheet (legend) and the word "reverse" appears in the item's notes, the marking is done automatically. Pallant (2020, chapter 8) reminds you to check in the scale manual or the original article which items are reversed – not to guess from the wording alone.

The scale score

Once reliability has been found adequate, the items are combined into a single score for each participant. Pallant (2020, chapter 8) computes a sum, and notes that many researchers divide by the number of items so that the score returns to the original response scale (1-5) and is easier to interpret – this is the page's default, "Mean of the items". The practical difference between the two lies in the handling of missing values: in SPSS a simple sum of items gives a missing value to every participant who is missing a single item (Pallant, Chapter 8), whereas the mean of the valid items – MEAN.n in SPSS syntax – uses what is there. The page computes the score for a participant only if they answered a sufficient proportion of the items (default 75%: five out of six), and the sum, when selected, is prorated as the mean times the number of items; anyone who did not pass the threshold gets an empty cell. Pallant asks you to record in the codebook the name of the new variable, what it represents, which items were summed, which were reversed and what the possible range is – exactly what the page writes to the codebook row in the downloaded file.

The site convention (the Variables and levels of measurement page): a single rating item is an ordinal variable; a multi-item scale score is treated as interval, and therefore enters the Pearson correlation, the t-test and regression. The histogram in the output shows whether the score is reasonably distributed and whether there is a floor or a ceiling.

Assumptions and warnings

What α assumes. Field (2024, section 18.9.3) explains: α assumes that all the items measure the true score to the same degree – essential tau-equivalence, which is mathematically equivalent to equal loadings of all the items on a single factor. In reality the loadings differ, and then α underestimates reliability – a "lower bound". Field goes as far as saying that those who know the subject consider α a poor measure, and refers to articles on alternatives; but it is what every journal expects to see, which is why the page computes it.

The number of items. The numerator of the formula contains the squared number of items, so α rises the longer the scale is – even without the items being more consistent. Cortina's example (1993, as cited in Field, Section 18.9.3) in Figure 1: three items with a respectable average correlation of .57 and ten items with an average correlation of .28 – both α = .80. This is why the table shows the average correlation next to α, and why a uniform threshold for every scale makes no sense.

α is not a measure of unidimensionality. A high α does not mean that the scale measures one thing: Grayson (2004, as cited in Field, Section 18.9.3) showed that α = .8 is obtained from one factor, from two correlated factors and from two uncorrelated factors alike. Field: if a questionnaire has subscales, compute α for each subscale separately and not for all the items together – as Cronbach himself suggested. The structure of the scale is examined with factor analysis, not with α.

Thresholds. Field (section 18.9.4) cites Kline (1999): .8 is appropriate for cognitive tests, .7 for ability tests, and for diverse psychological constructs values below .7 can realistically be expected; Nunnally (1978) – in the early stages of research .5 is enough. Field's conclusion: general thresholds should not be used; instead, α should be interpreted in the context of how it was measured and of the study (Pedhazur & Schmelkin, 1991). The note "good / acceptable / below the accepted threshold" that the page prints uses Pallant's .7 and .8, and it is a starting point for discussion, not a verdict.

The sample. Reliability depends on the sample (Field, Section 18.9.1; Pallant, Chapter 9), so it is reported for the current sample alongside what the scale developers reported. Field adds that the composition of the sample – people from different populations in the same sample – also biases α, sometimes upward and sometimes downward (Waller, 2008, as cited in Field).

Missing values. Reliability is computed on participants who answered all the items (listwise deletion, as in the SPSS default); when many were excluded, the page warns. The score, in contrast, is computed for everyone who passed the threshold of valid items – so the N of the reliability and the N of the score can differ on the same data. For the question of when missing items may be imputed and when not – the Missing values page.

What this page does not compute. Split-half reliability and the intraclass correlation coefficient (ICC) – the two other options in the SPSS dialog (Field, Section 18.10); a bootstrap interval for α (Raykov, 2024, mentions it alongside the delta method – the three intervals in Table 2 are analytic); and the alternatives Field refers to (section 18.9.3) for scales that do not meet the equivalence assumption – they require factor analysis: composite reliability (ω, Raykov, 2024, Equations 13-15) with a confidence interval, alongside alpha, is computed on the Confirmatory factor analysis page (Table 4). Factor analysis itself – including the question of whether the items really load on a single factor – is examined there (when the structure is set in advance) or on the Exploratory factor analysis page (when the structure is unknown), not on this page.

And how do you do it in SPSS?

First of all – reverse-scoring (Pallant, 2020, Chapter 8): Transform › Recode into Different Variables…, the reversed items to Input Variable → Output Variable, a new name for each (Pallant: Rop2 for op2), and in the Old and New Values button define 1 → 5, 2 → 4, 3 → 3, 4 → 2, 5 → 1 ("not all scales have 5 points – check that you have reversed all the possible values"). Field (section 18.9.4) does it with a single compute command, Transform › Compute: 6 − item. The syntax the page generates uses Field's method, into new variables, so that the original is preserved.

Reliability (Pallant, Chapter 9; Field, Section 18.10): Analyze › Scale › Reliability Analysis… – the items (the reversed ones in their reversed version!) to the Items box; Model: Alpha; the scale name in Scale label (adds a title to the output); and in the Statistics… button – Descriptives for: Item, Scale, Scale if item deleted; Inter-Item: Correlations; Summaries: Correlations. The syntax:

RELIABILITY
  /VARIABLES=item1 item2 item3_rev item4 item5_rev item6
  /SCALE('Commitment') ALL
  /MODEL=ALPHA
  /STATISTICS=DESCRIPTIVE SCALE CORR
  /SUMMARY=TOTAL CORR.

Reading the output (Pallant, Chapter 9): Case Processing Summary – Valid and Excluded (listwise deletion) = N and "Excluded" in Table 1; Reliability Statistics – Cronbach's Alpha, Cronbach's Alpha Based on Standardized Items, N of Items = the first three columns; Item Statistics – the mean and standard deviation in Table 3; Inter-Item Correlation Matrix = Table 4; Summary Item Statistics – the Inter-Item Correlations row, the Mean, Minimum, Maximum columns = the average correlation and its range; Item-Total Statistics – Corrected Item-Total Correlation and Cronbach's Alpha if Item Deleted = Table 3. A confidence interval for α (Table 2) is not in the RELIABILITY output – for that, use R (the R section below). Pallant first checks that the number of cases and the number of items are correct – a simple mistake that explains many strange alphas.

The score (Pallant, Chapter 8): Transform › Compute Variable… – a name for the new variable, and in the expression item1 + item2 + Ritem3 + … for the sum, or MEAN.5(item1, item2, …) for the mean of the valid items with a requirement of at least five items – this is the command in the page's syntax (multiplied by the number of items when sum is selected). The "Download SPSS syntax" button in the output generates all the steps – loading, dictionary (if there was a codebook sheet), reverse-scoring, RELIABILITY, score and DESCRIPTIVES – on the loaded data and with its variable names.

And how do you do it in R?

The psych package computes everything with a single command, alpha(): the ordinary α (raw_alpha), the standardized one (std.alpha), the average correlation (average_r), the corrected item-total correlation (r.drop) and α if item deleted (alpha.drop). The script was run as is on the sample data file and gave the table values (in the comments). The site rule for R: the file is saved in UTF-8, every comment line starts with an English word, and Hebrew sheet and variable names require a UTF-8 locale (on Windows: R 4.2 or later, or Sys.setlocale('LC_ALL', 'Hebrew')).

# Cronbach's alpha and scale score – Professor Stat. This file is saved in UTF-8; every comment line starts with an English word
library(readxl)   # Reading Excel
library(psych)    # alpha
library(alphaci)  # Confidence intervals for α (ADF)

# Loading: the standard template – row = participant, column = variable; the sheet "data" (data)
d <- read_excel("C:/Rdata/DATA_scale_example_eng.xlsx", sheet = "data")
items <- d[, c("commit_1", "commit_2", "commit_3", "commit_4", "commit_5", "commit_6")]

# Reverse-scoring the reverse-worded items: (minimum + maximum) − value
items$"commit_3" <- (1 + 5) - items$"commit_3"
items$"commit_5" <- (1 + 5) - items$"commit_5"

# Reliability: listwise deletion as in SPSS (RELIABILITY)
complete <- items[complete.cases(items), ]
a <- alpha(complete)
a$total$raw_alpha       # Cronbach's alpha = .809
a$total$std.alpha       # Alpha on standardized items = .809
a$total$average_r       # Average inter-item correlation = .414
nrow(complete)          # Listwise N = 171
a$item.stats$r.drop     # Corrected item-total correlation: .689 .568 .662 .559 .584 .366
a$alpha.drop$raw_alpha  # Alpha if item deleted: .753 .779 .757 .781 .776 .822

# Confidence intervals for α (Table 2 on the page): ADF – alphaci; psych – Feldt (F) and ase (Duhachek's normal standard error)
alphaci(complete, type = "adf", conf_level = 0.95)      # ADF 95%: .764 .854 (alphaci: t percentile; the page uses z – the limits differ in the third digit)
alphaci(complete, type = "normal", conf_level = 0.99)   # Normal theory 99%: .750 .868
a$feldt                 # Feldt's F 95%: .761 .850
a$total$ase             # Normal standard error = .0225; z limits: .809 ± 1.96·.0225 = .765 .853

# Scale score: the mean of the valid items, only for participants with at least 75% valid items
valid <- rowSums(!is.na(items))
score <- ifelse(valid / ncol(items) >= 0.75, rowMeans(items, na.rm = TRUE), NA)
c(n = sum(!is.na(score)), mean = mean(score, na.rm = TRUE), sd = sd(score, na.rm = TRUE))   # 197, 3.199, 0.742

The confidence intervals: the alphaci package (version 1.0.1) computes the same standard errors as the page – ADF and normal – but builds the limits with a t(N − 1) percentile and divides by N − 1, so they are slightly wider (in the example: .7641-.8542 versus .7644-.8539 on the page); the page follows Maydeu-Olivares et al. (2007) – a z percentile, and the van Zyl formula with N – like the ase of psych ("both/and": both conventions are legitimate, the difference is in the third digit). alpha() also computes additional measures that are not on the page; psych can reverse items by itself (keys), but the explicit reversal above is more transparent and documented. The mean score in R differs from the one in the table in the fifth digit, because the page rounds each score to four digits before it is written to the file.

Reporting the results

Pallant (2020, chapter 9): reliability is reported in the method section, under "Materials" – after describing the scale (number of items, response scale, history of use), report the reliability reported by the scale developers and other researchers, and then a sentence on the current sample; her example: "According to Pavot et al. (1991), the Satisfaction with Life Scale has good internal consistency, α = .85; in the current study α = .89". Field (section 18.12) reports in the body of the text with the symbol α, and notes that according to APA the zero before the decimal point is omitted because α cannot exceed 1 (APA, 2020, Section 6.36) – this is how the tables and report sentences on the page are built. What else should be in the sentence: the number of items, the response scale, which items were reversed, the N on which α was computed, the 95% confidence interval beside it in APA format (section 6.43) – α = .81, 95% CI [.76, .85] – as Raykov (2024) recommends (site convention: the reported interval is the ADF one, Table 2), and for a short scale also the average correlation (Pallant). The scale score is described in the same place – how it was computed (mean/sum, the missing-values threshold) and what the possible range is – and its descriptive statistics go to the results chapter.

Common mistakes

Computing α without reversing the reversed items. Field (section 18.11): α plummets, one item gets a negative item-total correlation, and "α if item deleted" jumps. If the output looks like this – do not delete the item but reverse it.

Reversing with the wrong range. (minimum + maximum) − value with a maximum of 5 on a 7-point scale gives negative values and a distorted scale. The range is the possible range according to the questionnaire, even if nobody marked 7.

Computing a single α for a whole questionnaire with subscales. Field (section 18.9.3): α for each subscale separately; the α of "everything together" will be high because of the number of items and will say nothing about any subscale.

Hunting for a high α by deleting items. The "α if item deleted" column invites you to delete every item that raises α by a hundredth. Pallant: in a validated scale this prevents comparison with other studies; Field: deletion changes the factor structure, which then has to be checked again; Raykov (2024): it may lower reliability and validity. Delete only with a content-based justification, and report that you deleted.

Reporting the standardized α because it is higher. It is the reliability of a different scale – of standardized items that nobody sums (Raykov, 2024). Report the α of the score you analyze.

Reading α = .9 as proof that the scale measures one thing. Grayson (as cited in Field): the same α is obtained from completely different structures. Unidimensionality is examined with factor analysis.

Citing the reliability from the original article instead of computing it. Field and Pallant: reliability belongs to the scale in this sample. Report both – the developers' and yours.

Computing a sum score in SPSS and losing participants. item1 + item2 + … returns a missing value for anyone who skipped a single item (Pallant, Chapter 8). MEAN.n with a stated threshold keeps them – and the threshold is reported.

References

American Psychological Association. (2020). Publication manual of the American Psychological Association (7th ed.). https://doi.org/10.1037/0000165-000

Field, A. (2024). Discovering statistics using IBM SPSS statistics (6th ed.). SAGE.

Maydeu-Olivares, A., Coffman, D. L., & Hartmann, W. M. (2007). Asymptotically distribution-free (ADF) interval estimation of coefficient alpha. Psychological Methods, 12(2), 157-176. https://doi.org/10.1037/1082-989X.12.2.157

Pallant, J. (2020). SPSS survival manual: A step by step guide to data analysis using IBM SPSS (7th ed.). Routledge.

Raykov, T. (2024). Reliability. In A. L. Nichols & J. E. Edlund (Eds.), The Cambridge handbook of research methods and statistics for the social and behavioral sciences: Vol. 2. Performing research (pp. 189-210). Cambridge University Press. https://doi.org/10.1017/9781009000796.010

Cronbach (1951), Cortina (1993), Grayson (2004), Kline (1999), Nunnally (1978), Pedhazur and Schmelkin (1991) and Waller (2008) – as cited in Field (2024); DeVellis (2012), Briggs and Cheek (1986) and Pavot et al. (1991) – as cited in Pallant (2020); Feldt (1965), van Zyl, Neudecker and Nel (2000), Duhachek and Iacobucci (2004) and Satorra and Bentler (1994) – as cited in Maydeu-Olivares, Coffman and Hartmann (2007); Raykov and Marcoulides (2015) – as cited in Raykov (2024). R packages that were run: psych (alpha) and alphaci 1.0.1 (Moss). According to the site rule, the reference list includes only sources that were read in the original.