The six models at the top of the GPQA diamond leaderboard have no published price. They also have no verified score. The leaderboard has 182 rows; the price table has 70 entries on this benchmark, and none of them belongs to the top six. A buyer who opens the leaderboard to find the strongest model and then tries to price it finds an empty cell at exactly the point where the decision gets expensive.
That is not a rounding error in the price table. The unpriced rows are the top of the distribution: 0 of 5 at ranks 1–5, 2 of 10 at ranks 1–10, 5 of 20 at ranks 1–20. By contrast, 65 of the 132 rows below rank 50 carry a price. The gap is worse at the front than the back, which is the opposite of what a missing-data problem usually looks like.
GPQA diamond is the hardest subset of Graduate-level Google-Proof Q&A: 198 questions written by PhD-level experts across biology, chemistry, and physics, scored as 4-option multiple-choice accuracy. It was released in November 2023 by researchers at NYU, Cohere, and Anthropic. PhD-level human experts score around 65% on it; non-experts with web access score about 34%. The design goal was to make questions that resist lookup, so that a correct answer reflects synthesis rather than retrieval.
The benchmark still separates models. Its discrimination score is 100 out of 100, with a 97.85-point spread between the weakest and strongest of the 182 scored models. If you want one number that orders models by graduate-level science reasoning inside a single evaluation harness, this remains a good one.
The six highest scores sit inside a 1.57-point band: Gemini 4 Carbon at 99.34, Gemini 4 Argon at 99.17, GPT-6 Astra at 98.29, Claude Sonnet 5.5 at 98.03, and Gemini 3.8 Flash and GPT-6.1 Sol both at 97.77. Only 0.7 points of headroom remain to the 100-point ceiling, and the saturation flag is on, with a 6-model cluster bunched at the top.
More important for a reader is where those six rows come from. All six carry a source recorded as unknown. None is marked independently reproduced, and none is marked self-reported either — they sit in the category this record treats as unverified, meaning no provenance is asserted for the measurement. The verified rows begin at rank 7 with Gemini 3.7 Flash at 96.98, from Epoch evaluations and independently reproduced. Rank 10 and rank 11 drop back into the unverified group.
So the leaderboard's first six rows and its first three verified rows are different sets of models. Anyone reading a model card that quotes "99.34 on GPQA diamond" is reading a number this record cannot trace to a source.
Ranking by the cost of a point rather than by the score itself makes the buyer's question answerable. The site's price-per-point measure divides a benchmark score by the output price per million tokens, so the number is points bought per $1M of output.
The first two models on the leaderboard with a published price are GPT-5.4 Pro at rank 8 (96.67) and Gemini 3.1 Pro at rank 9 (96.45). The gap between them is 0.22 points. Their output prices are $180 per million and $12 per million respectively, with input at $30 and $2.
That is 0.537 points per $1M of output for GPT-5.4 Pro against 8.038 for Gemini 3.1 Pro — a 14.97 times difference in what each dollar of output buys, for a score difference of 0.22 points. The blended three-to-one convention, which weights input at three times output, puts the same gap at $67.50 against $4.50 per million tokens.
Two caveats travel with that comparison. Gemini 3.1 Pro's price is recorded from a single source, while GPT-5.4 Pro's is corroborated across sources, so the price side of the ratio is not equally firm. Both scores are independently reproduced. And a price-per-point table over GPQA diamond inherits the benchmark's contamination profile, which is the subject of the next section.
| Model | Rank | GPQA diamond | Output $/M | Points per $1M output | Price sources |
|---|---|---|---|---|---|
| Gemini 3.5 Flash | 25 | 94.17 | $9 | 10.46 | corroborated |
| Gemini 3.1 Pro | 9 | 96.45 | $12 | 8.04 | single-source |
| GPT-5.6 Terra | 21 | 94.87 | $12 | 7.91 | single-source |
| GPT-5.4 | 22 | 94.86 | $15 | 6.32 | corroborated |
| GPT-5.6 Sol | 19 | 95.14 | $20 | 4.76 | single-source |
| GPT-5.5 | 15 | 95.84 | $30 | 3.19 | corroborated |
| GPT-5.5 Pro | 16 | 95.72 | $180 | 0.53 | corroborated |
| GPT-5.4 Pro | 8 | 96.67 | $180 | 0.54 | corroborated |
The table is the argument. Five models in the 94-to-96 band cost between $9 and $30 per million output tokens. The one other model in that band, GPT-5.5 Pro, costs $180 and scores 95.72. Buying the extra 0.22 points that separates GPT-5.4 Pro from Gemini 3.1 Pro costs fifteen times as much per point and buys a difference that a 198-question multiple-choice test cannot resolve reliably anyway.
GPQA diamond earns a Benchmark Integrity grade of B, at 73 out of 100, ranking 49th of the 63 tracked benchmarks. Two components explain that grade.
Contamination scores 2.7 out of 100. The test set is recorded as public, with a canary string but no secrecy, and the contamination window ratio is 0.973 — 97.3% of the dated rows come from models released after GPQA's 2023-11-20 release date, so nearly every model in the record had the opportunity to train on the questions. A score above 90 on this benchmark is an upper bound on reasoning, not a measurement of it.
The harness axis scores 50 out of 100, with the comparison recorded as "mixed harness — not directly comparable". The flag is set because the 184 rows come from different sources: 137 from Epoch evaluations, 8 from the DeepSeek-R1 paper's Tables 4 and 5, and 39 with no recorded source. Every row in the benchmark is single-source; no result has been cross-checked by a second evaluator. That is why the unverified top rows and the priced, verified rows at ranks 7 and 8 are not directly comparable to each other, whatever their scores.
GPQA diamond still orders models, and it still grades B for reasons that have nothing to do with the benchmark being broken. The problem is narrower and more practical than a broken benchmark: the front of the leaderboard is unpriceable and unverified, so the leaderboard cannot be used to build a shortlist on its own.
Use it to compare models that carry a reproduced score from the same evaluator, and treat the sub-96 band as the decision range, because inside the top ten the scores are separated by just over 3 points on a 198-item test. Do not use it to pick a price tier: 0 of the top six are priced, and the first priceable pair sits 0.22 points apart at 15 times the price per point. Pair it with a held-out benchmark and with the price record's own caveats — 35 of the 83 prices in this record are disputed, 26 corroborated, and 22 come from a single source.
What this number does not prove: that GPT-5.4 Pro reasons better than Gemini 3.1 Pro. It proves that one model holds a score 0.22 points higher on a benchmark whose test set has been public since 2023, and that the second model costs a fifteenth as much per point of that score.