WHEN THE INTEGRITY AUDIT FAILS ITS OWN AUDIT
Frank et al. published an extraordinary accusation in a journal called Research Integrity and Peer Review: hundreds of IHU-Marseille publications were said to raise ethical or regulatory concerns, and 81.3% of a 374-paper subgroup allegedly did not match the ethics approvals they cited.
There is now a rather serious problem. The paper cannot even agree with itself about its main result. In several places it reports: 304 / 374 = 81.3%. Elsewhere in the Results: 283 / 374 = 75.7%
Same denominator. Twenty-one papers have apparently vanished. And the obvious way to resolve such a discrepancy — going back to the published dataset — does not work.
The released CSV contains no proper article-level variable telling readers which papers were classified as mismatches, why they were classified that way, which legal rule was applied, who performed the assessment, whether a second reviewer checked it or how disagreements were resolved.
In other words: the headline result of a research-integrity paper is not reproducible from the data the authors released. That alone deserves an explanation.
But the problems do not stop there. Independent checking has already identified examples that appear difficult to reconcile with the published classifications:
a study reportedly labelled as having retrospective ethics approval although the approval preceded enrolment;
a review article sitting inside the supposed human-participant cohort;
a duplicated DOI with conflicting ethics information.
Again, these examples do not prove that all 853 records are wrong. But they destroy any reasonable assumption that hundreds of classifications can simply be accepted without an independent reliability audit. And there is another uncomfortable fact.
The peer reviewers warned about precisely these issues before publication. Lex Bouter repeatedly asked for a proper protocol, reproducible search procedures, details of the assessors, independent coding and access to the underlying evidence.
Eventually, the authors admitted: there was no predefined protocol.
Think about what that means. A multi-year campaign examined hundreds of papers, classified their ethical and legal status, contacted their editors, followed up editorial decisions, and then published those outcomes as evidence supporting the investigation. Yet the classification exercise itself was not governed from the beginning by a frozen protocol capable of independent replication.
There is also a statistical sleight of hand that deserves much more attention.
The famous 81% does not come from a random sample of IHU research. Approximately 69% of the 374-paper subgroup is tied to one single approval family, 09-022.
So the impressive-looking percentage is overwhelmingly driven by one large cluster. That may be a perfectly legitimate cluster to investigate. It is not the same thing as demonstrating that 81% of IHU publications generally had invalid ethics approvals.
There are also legal claims in the paper that appear remarkably categorical for an article making accusations of regulatory non-compliance. Research involving religious information is described as prohibited in France. Protocol changes are described in terms broader than the statutory concept of substantial modification. Historical versions of the Declaration of Helsinki appear to be interpreted through later standards. When legal interpretation determines whether a paper is labelled unethical or unlawful, such errors are not academic footnotes.
They change the classification itself. There is an important distinction to make here. Documented regulatory failures at the IHU exist. Denying them would be as scientifically irresponsible as exaggerating them. But genuine individual cases cannot be used as a licence to validate hundreds of additional classifications that readers cannot independently reproduce. And this is where the story becomes particularly ironic.
For years, the justification for reporting hundreds of IHU papers to editors has been scientific integrity.
The paper now offered as the scientific summary of that campaign itself contains: contradictory headline numbers, an unreproducible central result, identifiable classification errors, questionable legal assumptions, highly clustered sampling, no predefined protocol and no systematic published measure of inter-rater reliability.
That does not prove fraud. It does not prove that every allegation is false. It proves something far simpler: Frank et al. should now be subjected to exactly the same standard of scrutiny that its authors demanded for everybody else.
Before this paper is repeatedly cited as proof that 853 IHU publications were ethically or legally defective, its authors and its journal should answer one elementary question:
Show us the audit trail.
Show the 374 papers.
Show which 304 — or 283 — were mismatches.
Show the reason for each classification.
Show the applicable law.
Show the reviewers.
Show the disagreements.
Show the adjudications.
And let an independent team reproduce the result.
Until then, this paper may be a catalogue of allegations worthy of investigation.
It is not yet a reproducible demonstration of the sweeping conclusions being drawn from it.