Generalizability beyond the experiment
A large participant sample does not remove uncertainty about the words, faces or situations selected.
The question
Does an effect generalize across people, stimuli, tasks and settings?
Yarkoni’s target article, published online in 2020 and in Behavioral and Brain Sciences in 2022, argues that many psychological conclusions exceed the variation represented in their statistical models.
What the claims mean
Narrow inference
A well-controlled study can establish a result for the particular operation tested. Its precision need not justify a broad statement about a construct.
Explicit generalization
Model and sample relevant variation in stimuli, settings and tasks as well as participants, or narrow the claim to the observed design.
Evidence to examine
Yarkoni uses methodological analysis to connect broad verbal conclusions with underspecified statistical inferences. This is a critique of inference practices; it is not a new prevalence estimate of how many psychological effects fail in daily life.
Shared ground and disagreement
What the accounts can share
Repeating the same narrow task with many people improves one part of precision. It does not automatically cover every task that a theoretical label could include.
What remains at issue
Which sources of variation belong in a given claim, and what design and model can support inference across them.
Limits and next tests
No study samples every possible context. Treating all contexts as interchangeable can also erase meaningful boundaries. Generalization requires a defensible target, not maximal complexity by default.
What would help distinguish the claims?
Write the intended conclusion first, identify the populations and operations it names, and sample or justify those dimensions. Evaluate prediction in genuinely new tasks and settings.
Textbook connection
ContextualOpenStax Psychology 2e: 2.2 Approaches to Research
The section provides background vocabulary. It does not cover the full debate or establish the status of later research.
Sources and access
- Yarkoni, T. (2020). The generalizability crisis. The Behavioral and Brain Sciences, 45, Article e1. https://doi.org/10.1017/s0140525x20001685Consulted sourceAbstract inspected
Primary bibliographic record and complete abstract retrieved from Europe PMC on October 21, 2026. Full paper methods and supplements were not inspected.