Editorial
Analysis feed AtomAnalysis from the evidence record — every number cited to the public dataset.
The record carries 58 APEX-Agents scores and not one names an evaluator or a source. The benchmark's own authors flag that its task prompts are public and may overlap with training data, yet it keeps an A grade — its contamination score of 3.4 is the lowest of any A-grade benchmark on the site. The top 15 rows have no published price.
DeepSeek-R1 carries two GPQA diamond scores: 64.88 from an independent evaluation and 71.5 from DeepSeek's own paper. The record publishes 64.88 and shows both, with a 6.62-point gap. Three of the record's 2,946 score facts have two scores that disagree; the record picks the independently reproduced one every time.
93% of the 2,881 dated score rows in the tensor.news record carried the model's release date under a heading that promised the evaluation date. Every leaderboard, the data dictionary and the feed now read Released. The underlying limitation is unchanged: the record still cannot date a single evaluation.
tensor.news publishes one price per model and cross-checks it against the other aggregators. Of the 52 price facts where the sources actually disagree, the published number is the highest of the quotes in 39, and where they disagree the gap reaches 19,819%. The site now shows every source's own value and the spread between them.
The record scores 116 models on WeirdML, an ML-engineering benchmark with no established human score. 101 rows come from the author's page and 15 from a second leaderboard, and the two share zero model versions, so the record cannot check whether the scales agree. A grade A, and 6.43 points of headroom.
The record tracks two benchmarks called WeirdML. One has 116 models scoring 1.73 to 93.57; the other has 26 models scoring 4.87 to 60.05. They share a name and a leaderboard, they share no models, and their scores run on different scales — v3's official score is a weighted curve area, not raw accuracy. The site links them and says their scores are not comparable.
ARC-AGI's 89 rows come from two leaderboards plus 11 unsourced rows. Their medians sit 60.9 points apart and they share zero model versions, so the record cannot check whether the two scales agree — yet the page says scores are reasonably apples-to-apples.
The Benchmark Integrity Index gives away 1,006.5 points across 63 tracked benchmarks. Contamination supplies 508.7 of them from a 15% weight. The 16 benchmarks that fail it outright hold a quarter of every model score on the site, and not one earns an A.
559 of 2,993 tracked benchmark claims are independently reproduced, and all 559 are Epoch AI's own runs. The ledger grew 54% since July while the reproduced tier shrank by 14 claims.
None of the six highest-scoring models on GPQA diamond has a published price, and all six carry unverified scores. The first models you can price sit at ranks 8 and 9, 0.22 points apart at 15 times the price per point.
OTIS Mock AIME has a 15-way tie at 100. Eleven of those fifteen models have no price and no reproduced score. The four priced ones range from $20 to $180 per million output tokens for identical scores.
Sweeping every possible number of answer options, k=6 explains 81 of the benchmark's 106 model rows and no other option count explains more than 3. The record's numbers are the benchmark's raw accuracy with the 1-in-6 chance floor subtracted and the rest rescaled. The page's own human baseline is printed on the other scale, and three models that beat it are missing from the record.
The benchmark's discrimination holds, but 97.3% of its dated rows come from models released after the test set went public, and all six top rows carry no source at all. 39 out of 184 rows, including the entire top six, have no recorded evaluator.
Every score on the board is tier T2 — same metric, unknown evaluator. The benchmark holds an A grade and 6 points of headroom, but 84.1% of its dated rows come from models released after the test set went public.
Of 2,993 scores in the public record, 559 are independently reproduced and 2 are contradicted. The pricing picture is worse — 30 disputed prices against 26 corroborated — across a 491-fold cost range.