ARC-AGI's leaderboard on tensor.news is one ranked table. It is built from two different publisher feeds and eleven rows with no source at all. The two feeds' median scores sit 60.9 points apart, and they share zero model versions — so nothing in the record can check whether they measure the same thing.
The page grades ARC-AGI's harness comparability a perfect 100 and its FAQ says scores there "are reasonably apples-to-apples." Both are true statements about what the record checks, and neither covers the gap above. This is what the record can and cannot say about its own leaderboard.
All 89 ARC-AGI rows, from the ARC-AGI leaderboard:
| Feed | Rows | Median | Range |
|---|---|---|---|
https://arcprize.org/leaderboard |
54 | 90.50 | 4.38 – 98.50 |
ARC Prize Leaderboard |
24 | 29.65 | 0.00 – 98.00 |
| No source recorded | 11 | 54.33 | 11.00 – 86.50 |
The merged table is not an obvious seam. Gemini 3.8 Flash leads at 98.50 from the arcprize.org feed. Gemini 3.1 Pro sits at rank 7 with 98.00 from the other feed — a 0.5-point difference from the top, across two feeds. GPT-4.1 nano is last at 0.00, also from the ARC Prize Leaderboard feed. Rows from both feeds are interleaved throughout the table.
The 60.9-point median gap looks like a scale disagreement. It is not, or at least the record cannot show that it is. The two feeds are measuring different cohorts of models.
The arcprize.org feed's median evaluation date is 2026-07-09. The ARC Prize Leaderboard feed's is 2025-04-14. One feed covers the 2026 model generation; the other stops around early 2026. Compare the feeds only over the window where both are active, 2025-04-05 through 2026-02-19, and the median gap falls from 60.9 points to 20.1 points.
The same pattern holds across the record. On SimpleBench the raw median gap is 25.9 points; restricted to the shared evaluation window it is 9.5. On Terminal Bench it falls from 22.4 to 8.4. On WeirdML, from 4.9 to 1.9. On GSO-Bench, from 11.7 to 2.4.
A gap that shrinks when you hold the model cohort fixed is a gap about which models each publisher reported, not about how each publisher scores. Read the wrong way, "the feeds are 60 points apart" would be a false claim about ARC Prize's two boards. Read the right way, it says something the record is built to say and does not: the two feeds were never checked against each other.
The harness axis on every benchmark page is computed from two conditions, in the ainews enricher:
If neither mixes, the benchmark scores 100 and is labelled "consistent harness." ARC-AGI mixes neither, so it scores 100. The check is real and it does catch a genuine problem — GPQA diamond mixes Epoch's own runs with a vendor's self-reported table, and the axis drops to 50 there, "mixed harness — not directly comparable."
What the check does not test: whether two third-party feeds measure the same scale. ARC-AGI's two feeds are both third-party, so mixing third-party sources is invisible to it. Feed A and feed B are never compared against each other on the same model.
That comparison is impossible in this record. The feeds share zero model versions — not zero after normalization, zero as recorded. A holds Gemini 2.5 Pro at one version string; B holds it under a different one. A holds gpt-5.2-pro-2025-12-11_xhigh; B holds gpt-5.2-2025-12-11_xhigh. No overlap means no paired scores, and no paired scores means no calibration.
The site does disclose the feed on every row, and carries a caveat: "Leaderboard Reported scores — protocols may differ." That is the honest flag. It says the protocols may differ. It does not say how much they differ, because nothing in the record can say that.
This is not one benchmark with a bad merge. Twelve benchmarks in the record carry two or more source labels and no vendor self-report. On all twelve, the harness axis reads 100, "consistent harness." On all twelve, the feeds share zero model versions.
| Benchmark | Feeds | Rows | Median gap | Gap in shared window |
|---|---|---|---|---|
| ARC-AGI | 3 | 89 | 60.9 | 20.1 |
| MirrorCode | 3 | 15 | 55.5 | no shared window |
| The Agent Company | 2 | 13 | 26.0 | no shared window |
| SimpleBench | 2 | 78 | 25.9 | 9.5 |
| Terminal Bench | 2 | 32 | 22.4 | 8.4 |
| OSWorld | 2 | 8 | 22.4 | no shared window |
| GSO-Bench | 2 | 28 | 11.7 | 2.4 |
| Balrog | 2 | 29 | 11.1 | 9.6 |
| Aider polyglot | 2 | 44 | 6.5 | 4.2 |
| WeirdML | 2 | 116 | 4.9 | 1.9 |
| DeepResearch Bench | 2 | 24 | 3.8 | 4.2 |
| VPCT | 2 | 26 | 0.3 | no shared window |
Together those 12 hold 502 of the record's 2,841 scored rows, 17.7%. The other four with multiple sources are the ones where a vendor self-report is mixed in, and those the harness axis already catches.
Four benchmarks have no overlapping evaluation window at all — MirrorCode, The Agent Company, OSWorld, and VPCT. Their feeds never saw the same model on the same date. For those, even a cohort-controlled comparison is impossible, and the raw gap stands uncorrected.
The site builds head-to-head comparisons from the merged tables. On ARC-AGI it offers 2,154 model pairs where one model's score comes from one feed and the other's from a different feed. Across the whole record, 33,710 of 113,248 same-benchmark model pairs — 29.8% — draw their two scores from different feeds.
Every one of those comparisons declares a winner. The winner is the higher number, and the higher number may be from the feed with the higher scale.
Take Gemini 3.1 Pro at 98.00 from the ARC Prize Leaderboard feed against Gemini 3.8 Flash at 98.50 from arcprize.org. The site calls that a 0.5-point gap. On a single feed the two are also half a point apart, and both numbers happen to be near the ceiling of the benchmark, where the record itself warns that "small gaps at the top" deserve caution. The wider hazard is elsewhere: a model from the arcprize.org feed with an 85 and a model from the ARC Prize Leaderboard feed with a 92 are compared directly, and the record holds no row that would tell you whether those 92 and 85 are the same distance from zero.
That is the honest limit. The record does not know these two feeds agree. It does not know they disagree. It reports both, ranks them together, and grades its own consistency check 100 out of 100.
Nothing here needs a schema change or a new axis. One number is missing: for each pair of feeds on a benchmark, the median score difference over models both feeds measured, and the count of models both feeds measured. Where that count is zero — all twelve of these benchmarks — the record should say the feeds are uncalibrated rather than leave a clean 100 on the harness axis.
The unsourced rows are the second gap. Eleven of ARC-AGI's 89 rows carry no source at all, and none of them reach the top 20. They land between rank 40 and rank 81 in the merged table, which means they do not distort the top of the leaderboard, but they do occupy positions no publisher stands behind.
It does not show that ARC Prize's two boards disagree. Nothing in this record tests that, because the two feeds share no models.
It does not show that the ARC-AGI scores are wrong, or that Gemini 3.8 Flash's 98.5 is inflated. Every one of the 89 rows comes from a source-backed measurement with a date and a tier label, and the per-row feed is disclosed on the page.
It does not show that the harness axis is broken. It is scoped to mixing vendor self-reports with third parties and mixing optimized scaffolds with standard ones, and on that scope it is correct. GPQA diamond's 50 is the axis working. What ARC-AGI has is a gap outside the axis's scope, and a FAQ sentence that reads wider than the axis.
The record's own strongest signal points the same way. Of ARC-AGI's 89 rows, none is independently reproduced — every one is unverified or self-reported — and the benchmark's contamination score is 0 out of 100 because every dated row comes from a model released after the benchmark's 2019-11-05 release date. The gap above is a third thing the record is honest about not knowing, sitting in the same place as the first two.