Preregistration and exploration
Preserve useful exploration while making the evidential status of an analysis visible.
The question
How should researchers distinguish planned tests from discoveries made while analyzing data?
Simmons, Nelson and Simonsohn (2011) demonstrated how undisclosed flexibility could inflate false-positive findings. Open-science reforms subsequently emphasized advance plans and transparent reporting.
What the claims mean
Confirmatory testing
A specific hypothesis and analysis are fixed before outcomes are known. This constrains opportunities to select an analysis because it produces a preferred answer.
Exploratory analysis
Inspecting unexpected patterns can generate better questions and expose model problems. Its results need to be described as discoveries rather than as predictions made in advance.
Transparent flexibility
A departure from a plan can be justified and reported. Preregistration is a record of commitments, not a prohibition on learning from data.
Evidence to examine
The 2011 paper combines simulations and experiments to demonstrate the consequences of flexible collection and analysis. It supports concern about selective reporting, not a numerical estimate of misconduct across the discipline.
Shared ground and disagreement
What the accounts can share
An honest record of what was planned helps readers evaluate evidence. An advance plan cannot repair an invalid measure, biased sample or inappropriate model.
What remains at issue
How detailed plans should be, how to handle justified deviations, and how publication systems should reward discovery without overstating confirmation.
Limits and next tests
A registered analysis can still be poorly chosen. An unregistered study is not automatically false. The question is how much flexibility was available and what inference the reported analysis warrants.
What would help distinguish the claims?
Publish time-stamped plans, explain deviations and use independent data for important discoveries. Evaluate reforms by their effects on estimation, transparency and correction rather than by registration counts alone.
Textbook connection
ContextualOpenStax Psychology 2e: 2.3 Analyzing Findings
The section provides background vocabulary. It does not cover the full debate or establish the status of later research.
Sources and access
- Simmons, J. P., Nelson, L. D., & Simonsohn, U. (2011b). False-positive psychology: Undisclosed flexibility in data collection and analysis allows presenting anything as significant. Psychological Science, 22(11), 1359–1366. https://doi.org/10.1177/0956797611417632Consulted sourceAbstract inspected
Primary bibliographic record and complete abstract retrieved from Europe PMC on October 21, 2026. Full paper methods and supplements were not inspected.