Replication and scientific credibility
Distinguish uncertainty about an effect from a claim that all psychology has failed.
The question
What should change when a finding does not reproduce in a new study?
Large collaborative replication projects changed discussion of reproducibility during the 2010s. Their scope and sampling frame matter when interpreting their results.
What the claims mean
Direct replication
Repeat a procedure closely to test whether the reported result recurs under similar conditions. A precise replication can challenge a claimed effect size.
Boundary-condition account
An effect may depend on a population or context. Such an explanation gains credibility from independently testable predictions, not from a reason invented after every failure.
Cumulative evidence
Synthesis considers estimates, precision, design quality and reporting practices across studies rather than counting significant results.
Evidence to examine
The Open Science Collaboration (2015) attempted replications of 100 studies from three psychology journals. Its results raised concerns about reproducibility in that selected body of work. The project was not a random audit of every field, decade or method in psychology.
Shared ground and disagreement
What the accounts can share
A single statistically significant result is insufficient to establish a dependable general claim. Failed replication is information, even when it does not identify the cause of disagreement.
What remains at issue
Which differences between studies are substantive moderators, and how much published estimates were inflated by selection or flexible analysis.
Limits and next tests
Failure to reject a null is not proof that an effect is exactly zero. Equally, an unspecified contextual explanation cannot protect a claim from every possible outcome.
What would help distinguish the claims?
Use adequately precise preregistered replications and prospectively test proposed moderators. Compare confidence intervals and practical effect sizes, with transparent exclusions and shared materials.
Textbook connection
ContextualOpenStax Psychology 2e: 2.2 Approaches to Research
The section provides background vocabulary. It does not cover the full debate or establish the status of later research.
Sources and access
- Open Science Collaboration. (2015). PSYCHOLOGY. estimating the reproducibility of psychological science. Science (New York, N.Y.), 349(6251), aac4716. https://doi.org/10.1126/science.aac4716Consulted sourceAbstract inspected
Primary bibliographic record and complete abstract retrieved from Europe PMC on October 21, 2026. Full paper methods and supplements were not inspected.