Where Evidence Ends and Priorities Begin
Two people can accept the same results and still disagree. Pulling their dispute apart shows which parts a further study could settle and which parts it could only inform.
Browse the directoryImagine a department that tried a new instructional program for one term and recorded scores before and after, alongside a comparison group. In this invented case, average scores rose more in the program than in the comparison group, the spread of scores widened, and the lowest scorers improved least. One instructor calls the program a success. Another calls it a failure. They agree on every number.
Their disagreement has several layers, and different evidence touches each. The first is what happened: how large was the average gain, and how much did different students gain? A well-designed comparison can estimate that. The second is why: what about the program helped some students more than others? Further study can help distinguish explanations. The third is meaning. Suppose the department scores essays with a rubric and calls the result writing quality. Two readers may mean different things by good writing, and a rubric can give consistent scores while missing part of what one of them cares about. That is a construct validity dispute, and evidence can help, for example by checking whether rubric scores agree with what experienced readers judge independently.
The fourth layer is about priorities: is a bigger average gain worth a wider gap? Data can inform that by showing who gains, who is left behind and by how much. The decision also requires an explicit argument about which outcomes deserve priority, and that argument can be reasoned and criticized even though data alone will not choose it. Two camps arguing from rival studies may in fact be arguing at different layers, which is a reason to ask which one they are on before weighing the studies.
Disagreement does not make the evidence equal. One side may simply have better evidence on the empirical layer. Reading an argument begins by sorting which layer each claim sits on and how strong its support is.
For the department, a comparison can estimate how large each group's gain was, and checks on the rubric can help show whether it tracks the skill it names. What remains is the argument about whether a larger average gain justifies a wider spread. The claim also stays about this one course until it has been tried elsewhere.