Variables, levels of measurement and research designs

Guide

On this page

Once there is a prediction (see Research question and hypotheses), two decisions remain that determine everything that follows them: what to measure – which variables, and at what level of measurement, because the level determines what arithmetic is allowed and which calculator fits; and how to collect – observe what happens, or intervene and compare, with the same people or with different people, because the design determines what may be concluded. The page follows Field (2024, sections 1.5-1.6) and connects each decision to the site's standard Excel template and to the calculator that expects it.

1. Variables – predictor and outcome

A variable is something that varies: between people (IQ, behavior), between places (unemployment) or over time (mood, profit). Most hypotheses are formulated in two variables – a proposed cause and a proposed outcome – and the key to testing a scientific statement is to measure both (Field, 2024, Section 1.5.1). The terms "independent variable" (the cause) and "dependent variable" (the outcome) are closely tied to the experiment, in which the researcher manipulates the cause; in correlational research, in which no variable is manipulated, there is no point in talking about dependent and independent – they are all in effect dependent. Field therefore prefers predictor and outcome: in an experiment the cause is a predictor and the outcome is an outcome, and in correlational research one or more variables predict – statistically at least – another variable. The site follows him: in every calculator you will meet "predictor" and "outcome", and the causal inference – if there is one – comes from the design (section 4), not from the name.

2. Levels of measurement – what the number says, and what may be done with it

The level of measurement is the relation between what is measured and the numbers that represent it (Field, 2024, Section 1.5.2). The rough distinction: a categorical variable sorts entities into distinct categories – human, cat, bat; you cannot be a bit of a cat. A continuous variable gives each entity a score on a scale. And within each – levels:

Binary (binary) – only two categories: yes/no, took part/did not take part. Nominal – more than two, with no order: type of locality, sector, reason for referral. Even when categories are marked with numbers (1 = economic, 2 = security), the numbers are names: Field's example is rugby shirt numbers, which say which position the player plays and not how good they are – and no one would let number 4 give number 6 a piggyback to kick in place of the injured number 10. With nominal there is one thing to do: count frequencies. Ordinal (ordinal) – categories with a logical order: first, second, third place; fail, pass, distinction. You know who comes before whom, not by how much – the winner could have won by a huge margin or by a single vote. Interval (interval) – equal intervals on the scale represent equal differences in the property: the difference between 1 and 2 is the difference between 4 and 5. Most statistical tests require this level or above. Ratio (ratio) – in addition, there is a true zero and the ratios are meaningful: a reaction time of 200 milliseconds is twice 100, and zero milliseconds is an absence of time. Age, income, number of children – ratio; a score on an anxiety scale on which 16 is not "twice as anxious" as 8 – at most interval.

A further distinction cuts across the continuous ones: a truly continuous variable is measured to any level of precision (age in years, months, days...); a discrete variable takes only certain values, usually whole numbers – a rating of 1 to 5 is discrete even though beneath it there is a plausible continuum (3.24 makes sense, but it cannot be entered). The boundary is blurred in both directions: age is recorded in whole years, and number of children – discrete – is reported as a mean of 2.4 that no family has ever had (Field, 2024, Box 1.3). The calculators on the site treat numbers as continuous; the researcher decides whether the mean is meaningful.

3. Rating scales – ordinal or interval?

The question that comes up in every questionnaire study: a five-point rating "to a very great extent ... not at all" – is it an interval scale, for which a mean and a Pearson correlation may be computed, or an ordinal scale? Field (2024, Box 1.2) is sharp: a judge who gave 10 was surely more impressed than a judge who gave 2, but five times as impressed? And if both gave 8 – were they equally impressed? Probably not, because the rating depends on each one's subjective perception of what "talent" is. So, he writes, in any situation in which people are asked to rate something subjective – a preference, confidence in an answer, understanding of instructions – the data should be treated as an ordinal scale, "although many researchers do not". And in section 1.5.2 he adds: variables that look interval and are treated as interval are often ordinal.

The site convention, which follows from this and from common practice: a single item on a rating scale – ordinal; report the median and frequencies, and the Spearman correlation. A multi-item scale score – the sum or mean of several items whose internal consistency has been checked (see Reliability and validity) – is treated as interval, as is customary in the social sciences, and this is stated explicitly in the method section. This is the researcher's decision, not the software's: SPSS will compute a mean for any numeric column, including shirt numbers.

4. Correlational or experimental – what may be concluded

In the abstract, there are two ways to test a hypothesis: observe what happens naturally, or manipulate an aspect of the environment and observe its effect. In correlational (or cross-sectional) research the world is observed without intervening – in one snapshot of many variables, or in repeated measurements over time (longitudinal research); in experimental research one variable is manipulated to see its effect on another (Field, 2024, Section 1.6). Correlational research has an advantage: it gives a natural view, because the researcher does not influence what happens and the measurements are not biased by their presence – an important aspect of ecological validity; and sometimes it is the only way, because manipulation is impossible or forbidden (no one would randomly assign people to 30 years of smoking). Its price: it says nothing about causal effect.

Why? Field (2024, section 1.6.2) goes to Hume: Hume's definition of a cause requires that the cause precede the effect and that the two consistently occur together. In correlational research the variables are usually measured at the same time, so there is no information on what came first – longitudinal research partly solves this. But the deeper problem is that even consistent co-occurrence does not distinguish between coincidence and causation: a third variable may cause both. This is the "third thing" – tertium quid – the confounding variable (confound). In the site's running example: participation in community activity and a sense of belonging are correlated – but length of residence in the locality can cause both (Figure 1). Mill added to Hume the missing condition: all other explanations must be ruled out – the effect should occur when the cause is present and be absent when the cause is absent. And this is exactly what an experiment does: it compares situations ("conditions") in which the proposed cause is present or absent. In Field's example – three groups of students who receive supportive feedback, harsh feedback or no feedback, and a statistics test at the end: the "no feedback" group is the condition in which the cause is absent, and only the comparison with it allows us to say that the feedback style caused the difference.

A note that saves a common confusion (Field, 2024, Box 1.4): it is not the statistical test that allows causal inference, but the design. People think ANOVA and t are "experimental" and regression and correlation "correlational" because historically that is how they were used – but these procedures are mathematically identical. A correlation computed on data from a controlled experiment supports causal inference; ANOVA on a survey does not.

5. Between-subjects or repeated measures – two sources of variance

In an experiment there are two ways to manipulate the predictor (Field, 2024, Section 1.6.3): different groups of participants in each condition – a between-subjects (independent) design; or the same participants in all conditions, one after another – a within-subjects (repeated measures) design. The choice determines the test: independent-samples t versus paired-samples t, one-way ANOVA versus repeated-measures ANOVA. And the logic behind it lies in the variance (section 1.6.4). In every experiment there are two sources of variance: systematic – what the researcher did in one condition and not in the other; and unsystematic – random factors present across conditions: differences in ability, time of day, mood. Statistical tests estimate how much of the variance is systematic and how much is not, and compare. In repeated measures the participant's stable characteristics – age, sex, IQ, motivation – are identical in both conditions, so the "noise" is small; between subjects these factors are free to vary between the groups, and the noise is large. So, other things being equal, a repeated-measures design is more sensitive to detecting an effect.

6. Randomization – why without it there is no experiment

In both designs the unsystematic variance should be minimized, but more important than that – to make sure that any systematic variance between the conditions stems from the manipulation alone. The tool is randomization (Field, 2024, Section 1.6.5), in two forms. In repeated measures the two main sources of systematic variance are practice effects – in the second condition the participant is already familiar with the situation and the measure – and boredom effects or fatigue. They cannot be eliminated, but they can be made non-systematic by counterbalancing: draw for each participant whether they go through condition 1 before 2 or the other way round. In Field's example: if all the students went through "no feedback" and then "harsh feedback", the improvement after the harsh feedback could simply stem from more statistics seminars. Between subjects the randomization is random assignment of participants to conditions: participants in different groups differ in IQ, in attention and in a thousand things, and random assignment ensures that these differences contribute to the unsystematic variance and not to the systematic one. Field's example: a workshop in analytical thinking against belief in fake news, whose experimental group was recruited at a meeting of conspiracy-theory enthusiasts and whose control group – at a sports club. The workshop "did not work" – and there is no way to know whether this is because analytical thinking does not help, or because those recruited to the workshop are more prone to unusual beliefs and find such thinking difficult. The groups differed not only in the workshop.

The randomization itself is produced by the Design of experiments page: random assignment or a random run order, blocks, a Latin square or a Williams square for counterbalancing the order, and also the required sample size – with a fixed seed, so that the randomization can be reproduced in R.

Two different kinds of randomization that should not be confused: random sampling – who from the population enters the study – concerns generalization from the sample to the population (the "Sample and sampling" page); random assignment – which participant goes to which condition – concerns causal inference. An experiment on a convenience sample allows causal inference about this sample without generalization; a survey on a probability sample allows generalization without causality.

7. And what to do in field research, where almost everything is correlational

Surveys, interviews, content analysis, administrative data – most applied social research cannot randomly assign people to conditions, so most of it is correlational, and causal inference in it is limited. That is not a flaw; it is a limitation that is reported. What can be done: measure the confounding variables that can be anticipated (length of residence, age, education) and enter them into the regression – "statistical control". It removes the share of the variables that were measured and only those; random assignment also takes care of what we did not think of. Repeated measurement over time – at least two waves – gives the temporal order that Hume's definition requires, but not the ruling out of other explanations. A before-after comparison with a comparison group that did not receive the program – close to an experiment to the extent that the groups are similar, and far from it to the extent that those who chose to take part differ from those who did not. And in all cases: phrase the conclusion in the language the design permits – "associated with", "predicts", "rose among" – and keep "caused" for where there was randomization.

Common mistakes

A mean of nominal codes. "The mean type of locality is 2.3" – arithmetic on names. For nominal variables count frequencies; SPSS will compute a mean for any number, and the responsibility lies with the researcher.

A Pearson correlation between two single rating items. A single item is an ordinal scale; Spearman. Pearson and the mean are reserved for multi-item scale scores – and report that this is what you did.

A "predictor" that is read as "influences". In a regression on survey data the predictor predicts; it does not influence until the design says so. The language in the discussion section derives from the design.

Forgetting the condition in which the cause is absent. "We gave everyone the program and belonging rose" – without a group that did not receive it, there is no knowing whether it would not have risen without it too. The control is the heart of the experiment, not an add-on.

A control group from another source. Experiment in one community and control in another – the groups differ in everything that distinguishes the communities. Random assignment from the same pool, or at least documented matching.

Repeated measures in a fixed order. Everyone went through condition A and then B – the difference between the conditions is mixed up with practice and fatigue. Randomize the order.

Thinking that a "correlational" design is the correlation test. The design is how you collected; the test is how you analyzed. ANOVA on a survey is correlational; a correlation on an experiment – experimental.

Confusing random sampling with random assignment. The first concerns whom the conclusion refers to; the second – whether it is causal. You can have one without the other.

Checklist – before building the data file

References

Field, A. (2024). Discovering statistics using IBM SPSS statistics (6th ed.). SAGE.