DeepSeek-R1 has two scores on GPQA diamond, and they do not agree.
| Score | Evaluator | Source | Status |
|---|---|---|---|
| 64.88 | Epoch AI | Epoch evaluations | independently reproduced |
| 71.5 | DeepSeek AI | the DeepSeek-R1 paper, arXiv:2501.12948, Table 4 | self-reported |
The tensor.news leaderboard publishes 64.88. The model ranks #105 of 179 on GPQA diamond, between DeepSeek-R1-Distill-Llama-70B at 65.2 above it and Claude Haiku 4.5 at 64.18 below it.
Had the record published the paper's number, DeepSeek-R1 would sit at 71.5 — somewhere around the 60th row of the same table. The gap between the two is 6.62 points, and it moves the model about 45 places.
The record's selection is not "lowest wins" and it is not "first found wins". It is a tier order, applied to every measurement on every leaderboard:
Within a tier, a non-optimized run is preferred over an optimized one, then the score that matches the cross-check consensus, then the more recent measurement.
On this benchmark the Epoch evaluation is tier 2 and the paper's number is tier 4, so the Epoch score wins — and the record publishes the lower of the two. That is what the rule is supposed to do, and it is worth being explicit about, because a vendor's own number is always the friendlier one.
DeepSeek-V3 has the same pattern and a wider gap: 43.8 from Epoch AI, independently reproduced, against 59.1 from the same DeepSeek-R1 paper, self-reported. A gap of 15.30 points — the widest in the record. The leaderboard publishes 43.8, and the model sits at #132 of 179.
Both of the paper's numbers come from the same table of the same paper. Both are the vendor grading its own model, on a benchmark whose questions were public three years before the paper was written.
| Tier | Meaning |
|---|---|
| 1 | corroborated — the sources agree |
| 2 | independently reproduced — a named third party ran it |
| 3 | a third party that is neither reproduced nor self-reported |
| 4 | self-reported — the vendor's own number |
Within a tier, a non-optimized run beats an optimized one, then the score matching the cross-check consensus, then the more recent measurement.
One more pair in the record has three scores and no evaluator for any of them: 2.18, 4.11 and 9, on HumanEval, from a model the record no longer carries as an entity of its own. No evaluator is named, no source is recorded, none is reproduced or self-reported.
This is the case the record handles least well, and it is worth saying so plainly: when
nothing is known about who produced a number, the record cannot know which of three numbers
is the real one. It publishes one, flags it unverified, and the range is shown.
Three of the record's 2,946 score facts carry more than one recorded score, and the three below are the ones where those scores disagree:
Three is a small number, and it is small for a specific reason: the record holds one measurement for almost every (model, benchmark) pair, because most benchmarks are scored by one evaluator. The pairs that carry two are the ones where a vendor published its own number and an independent harness later ran the same benchmark.
Two things, and both are now visible.
The model page's Measurements section lists every recorded score for a pair that has more
than one, each with its evaluator, its evidence tier, and a min–max range. For DeepSeek-R1 it
shows 64.88 from Epoch AI with a reproduced glyph and 71.5 from DeepSeek AI with a
self-reported glyph, under the heading range 64.88–71.5.
And the number that reaches the leaderboard is the one the tier rule picks. A reader can check that decision against the rule in the methodology, and can see both numbers behind it.
That is the difference between a record and a table. A table shows you 64.88 and leaves you to assume it is the only number. The record shows you 64.88, shows you the 71.5 it declined to publish, and tells you which rule chose between them.
Both DeepSeek numbers come from the same place: the DeepSeek-R1 paper's own results table, arXiv:2501.12948, Table 4. Epoch AI later ran the same model on the same benchmark under its own harness and got a lower number.
Two things could explain a 6.62-point gap, and the record does not claim to know which: the benchmark's multiple-choice answers may be scored differently between the paper's grading and Epoch's, and the prompts or the harness may differ enough to move a reasoning model. GPQA diamond is a 198-question set, so 6.62 points is about thirteen questions.
That gap is also the reason the record prefers the independent evaluation. Not because Epoch AI is right and the paper is wrong — the record has no way to settle that — but because an independent harness is not the vendor grading its own model, and the tier rule is about who ran it, not about which number is larger. On both DeepSeek pairs the independently reproduced score is the lower one. If the rule preferred the higher number, both models would move up the table by 6.62 and 15.30 points, and nobody checking the rule would notice.