What the audit measures, and how far to trust it.

1. The DOI resolves via OpenAlex, falling back to Crossref, PubMed, and Semantic Scholar for venues that deposit no abstract.
2. Six open-access routes are tried for full text (arXiv by DOI, the OpenAlex OA URL, Unpaywall, EuropePMC, Semantic Scholar, arXiv by title). When all six miss, the audit binds the abstract and says so above the verdicts.
3. That text goes to Claude Opus 5 with the v1.4 prompt, at temperature 0, returning strict JSON: a 13-item rubric, per-field confidence, and a quote for every verdict it can source one for.
4. The tested model is matched against a registry of 185named models carrying release dates, Arena Elo, Epoch AI's Capabilities Index, and Artificial Analysis's Intelligence Index, then compared against the frontier as it stood when the evaluation ran.

No database. Identical inputs are served from a 24-hour cache; nothing outlives it.

The comparison is against the model at the top of the leaderboard in the month the evaluation ran, not against the best model that exists today. A paper testing GPT-4 in March 2024 was at the frontier; the same paper testing GPT-4 in March 2025 was about a year behind it. Holding the comparison to the contemporaneous frontier is what separates a disclosure failure from the ordinary fact that time passes between running an evaluation and publishing it.

Where no evaluation date is disclosed, the audit imputes a window bounded below by the tested model's release and above by the publication date, and reports the gap as a range marked imputed rather than picking a point inside it. The range is genuinely wide: the frontier moves during the window, so the gap is smallest if the evaluation ran the moment the tested model shipped and largest if it ran the day before publication. Which end is nearer the truth is exactly what the paper failed to say, and that missing sentence is the finding.

• Epoch Capabilities Index. Epoch AI's cross-benchmark capability score, and the unit the gap-in-months conversion runs on. Epoch re-fits the index as models and benchmarks land; the registry adopts each fit whole rather than mixing vintages, which would not be comparable.
• Chatbot Arena Elo. Per-model ratings and the monthly frontier trajectory, from the public leaderboard. Elo lags release by the weeks a model spends accumulating votes, so a new flagship can sit just under the previous leader while already being ahead of it; the audit anchors capability on ECI for that reason and reports Elo alongside.
• Artificial Analysis Intelligence Index. A third scale, versioned by its publisher. Values are comparable within a version and not across one, so a model without a current-version score reports none rather than a stale-scale one.
• Release dates. Vendor announcements and system cards first; where a vendor published no date, the registry records the Arena first-seen date as an explicit upper bound and labels it as such.

Last refreshed 2026-07-26, current through Claude Opus 5 (2026-07-24).

The companion paper benchmarks four extractors on 450 gold-standard papers against dual-human-adjudicated labels, under one frozen prompt. Claude Opus scored highest of the four on both subjective judgements: Cohen's κ of 0.771 on capability framing and 0.793 on conclusion valence, with 0.854 on domain. Those are raw first-pass figures, before the canonicalisation step the corpus pipeline applies, and they are the honest ceiling to put on any single verdict here.

Two caveats worth stating plainly. Model-name extraction scores far lower raw (κ = 0.478) because papers name models loosely and the gain comes from canonicalisation rather than from reading. And the benchmark measured Opus 4.7; this tool now runs Opus 5, which has not been re-benchmarked against the gold set. Confidence is published per field on every card, and the quote under each verdict is there so a disagreement can be settled against the paper rather than against the model.

• It audits disclosure, not performance. Every verdict about capability is a verdict about what the paper reported, never about how the model would actually do on the task.
• Item 4, the statistical comparison method, is not scored automatically. Methods sections vary too much for the extraction to have earned that yet.
• Capability axes the leaderboards do not adjudicate — new modalities, agent harnesses, vertical fine-tunes — sit outside the comparison, and a specialised model strong in its domain will read as further from frontier than a domain-specific benchmark would place it.
• A single extraction pass carries real variance. Reading the quote is the check.

The rubric, the calibration, and the frontier registry all come from Frontier Lag: A Bibliometric Audit of Capability Misrepresentation in Academic AI Evaluation (Gringras and Salahshoor, 2026). The audit pipeline is published as the frontierlag package, archived at Zenodo. This site is maintained by David Gringras, a Frank Knox Fellow at Harvard with original training in medicine and law. Corrections and disagreements go to davidgringras@hsph.harvard.edu.