Discipline-Relative Evidence Metrics

Discipline-Relative Evidence Metrics

Discipline-relative evidence metrics is this publication's rule that a book's peer-reviewed share means nothing until you know which discipline the book belongs to. The strongest evidence in history is the archival document. The strongest evidence in clinical science is the randomised trial. So a low peer-reviewed share can sit on excellent evidence and a high one on weak evidence, and setting the two numbers side by side is an error rather than a comparison. It is an audit rule, not a research claim.

What it is

The rule came out of auditing two books in the same week and getting opposite readings from the same statistic.

A book of history with several hundred notes had about six per cent of them pointing at peer-reviewed work and roughly half pointing at primary and archival material, including several named repositories. By the arithmetic that circulates as a quality proxy, that is a poor apparatus. By the standards of its own discipline it is a strong one, because the archival document is to history what the trial is to medicine and the historian who has been into the archive has done the thing that counts.

A book of popular neuroscience had about eighty-six per cent peer-reviewed citations, and roughly three in ten of those slots were single-patient case reports, which is the weakest design in clinical medicine. By the same arithmetic that is an excellent apparatus. By the standards of its own discipline it is a thin one.

The first number is not a failure and the second is not a pass. What the pair shows is that the metric has no meaning detached from the field that produced it.

In effect

In practice the rule is three instructions. Report the peer-reviewed share with the discipline attached or do not report it at all. Report citation class and study design together, because a citation's venue and a citation's strength are different properties. And never set a history title's figure beside a clinical one, because the comparison is not a comparison.

The rule follows from things that are already standard elsewhere: the evidence hierarchies used in medicine, and ordinary historical method, in which primary sources outrank secondary literature. What it adds is the refusal to let a single headline number from one of those traditions be read as though it came from the other.

It is also a defence against a specific failure this publication watches for, which is treating the venue a claim was published in as a measure of how much weight the claim can carry.

What it does not say

It does not say that peer review is unimportant. It says the share of peer-reviewed citations is not a portable quality score.

It does not say the two books in the example are equally good. It says the statistic does not tell you which is, and that you have to open the notes and look at what is in them.

It does not apply only to books. Any apparatus figure quoted about any body of work carries the same problem.

And it is not a finding. Nothing here was measured or tested. It is a working rule derived from standard practice in two disciplines, and it is recorded as such.

Sources

  1. Coined in this vault in 2026 while auditing two books of very different disciplines in the same pass.
  2. Brydan, D. Smart. The low-share, strong-evidence case. 399 notes, about six per cent peer-reviewed and about fifty-two per cent primary and archival, including six named repositories. Read in the vault on 2026-09-27.
  3. Dingman, M. Bizarre. The high-share, weak-design case. About eighty-six per cent peer-reviewed, of which about twenty-nine per cent are single-patient case reports. Read in the vault on 2026-09-27.
  4. The underlying conventions are the clinical evidence hierarchies used in medicine and ordinary historical method, in which primary and archival sources outrank secondary literature.