GPQA diamond is a multiple-choice benchmark that tests graduate-level reasoning in biology, chemistry, and physics. It was released in November 2023 by researchers at NYU, Cohere, and Anthropic. The explicit design goal was to create questions that are difficult for humans even when they have unrestricted web access, making it a proxy for deep domain expertise rather than surface-level retrieval. The metric is straightforward: 4-option accuracy, scored over the 198-item Diamond subset that PhD-level experts agreed on and non-experts failed.
What the benchmark actually measures is narrow but well-defined. Each question demands synthesis of graduate-level concepts; the "Google-proof" framing means that simple search-and-extract strategies fail. In practice, the benchmark evaluates a model's ability to chain facts, reason through counterfactuals, and identify correct answers when all answer choices appear plausible to a non-expert. That design keeps the signal high for advanced reasoning, which is why the discrimination component of its Benchmark Integrity Index stands at 100. The spread between the weakest and strongest of the 182 scored models is 97.85 points, confirming that the benchmark still separates capability levels cleanly.
In July this page reported that GPQA diamond was not saturated, with 7.2 points of headroom and a top score of 92.8. That is no longer true. The record now holds 184 scored rows instead of 122, the top score is 99.34, and only 0.7 points separate the leader from a perfect score. The saturation flag is now on, with a clustering ratio of 0.033 across a 6-model cluster and a ceiling proximity of 0.993.
The leaderboard's front also changed shape. The top six rows are Gemini 4 Carbon at 99.34, Gemini 4 Argon at 99.17, GPT-6 Astra at 98.29, Claude Sonnet 5.5 at 98.03, and Gemini 3.8 Flash and GPT-6.1 Sol both at 97.77 — all inside a 1.57-point band. Every one of those six rows carries a source recorded as unknown, no evaluator, and no evidence claim in either direction. They are what this record calls unverified: provenance is not asserted, not that the numbers are wrong.
Below them the picture changes sharply. Rank 7 is Gemini 3.7 Flash at 96.98, from an Epoch evaluation and independently reproduced. Rank 8 is GPT-5.4 Pro at 96.67, also from an Epoch evaluation and reproduced. Rank 9 is Gemini 3.1 Pro at 96.45, likewise reproduced. Rank 10 and rank 11 return to unattributed rows: GPT-6 Sol and GPT-6.1 Terra, both at 96.28.
So the first six rows and the first three rows with a recorded evaluator are different sets of models. On this benchmark, the strongest published numbers and the strongest traceable numbers do not overlap.
| Rank | Model | Score | Source | Evidence |
|---|---|---|---|---|
| 1 | Gemini 4 Carbon | 99.34 | unknown | unverified |
| 2 | Gemini 4 Argon | 99.17 | unknown | unverified |
| 3 | GPT-6 Astra | 98.29 | unknown | unverified |
| 4 | Claude Sonnet 5.5 | 98.03 | unknown | unverified |
| 5 | Gemini 3.8 Flash | 97.77 | unknown | unverified |
| 6 | GPT-6.1 Sol | 97.77 | unknown | unverified |
| 7 | Gemini 3.7 Flash | 96.98 | Epoch evaluations | reproduced |
| 8 | GPT-5.4 Pro | 96.67 | Epoch evaluations | reproduced |
| 9 | Gemini 3.1 Pro | 96.45 | Epoch evaluations | reproduced |
| 10 | GPT-6 Sol | 96.28 | unknown | unverified |
| 11 | GPT-6.1 Terra | 96.28 | unknown | unverified |
Across all 184 rows the evidence splits 137 independently reproduced, 8 self-reported, and 39 with neither. The source mix is 137 Epoch evaluations, 39 with no recorded source, and 8 rows from the DeepSeek-R1 paper's Tables 4 and 5, evaluated by DeepSeek AI. Every row is single-source. The harness axis scores 50 out of 100 with the comparison recorded as "mixed harness — not directly comparable", and the flag is set because those three sources do not share a harness.
Contamination is the weakest component of the benchmark's overall B grade, and it scores 2.7 out of 100. The test set is explicitly public. The contamination window ratio is 0.973 — 97.3% of the dated rows come from models released after GPQA's own 2023-11-20 release date, so nearly every model in the record had the opportunity to train on the questions.
This contamination score means two separate things. First, the risk itself is severe: the questions and answers have been public since November 2023, and every model released since then is a plausible beneficiary. Second, the contamination axis is measuring something real rather than being a placeholder — the window ratio moved from 0.943 in July to 0.973 now, tracking the arrival of newer models on a benchmark whose test set was already public. A high score on this benchmark is an upper bound on reasoning, not a measurement of it.
The age axis says the same thing from a different angle. The benchmark is 1,091 days old as of this refresh, and its age component scores 60.2 out of 100. As models get stronger and the community optimizes for this specific metric, the benchmark shifts from measurement toward target.
GPQA diamond holds a Benchmark Integrity grade of B at 73 out of 100, ranking 49th of the 63 tracked benchmarks. The five components:
| Component | Score | What it says |
|---|---|---|
| Discrimination | 100 | 97.85-point spread across 182 models |
| Saturation | 96.7 | Flag on; 0.7 points of headroom; 6-model cluster |
| Contamination | 2.7 | Public test set; 97.3% of dated rows postdate the release |
| Harness | 50 | Mixed harness — not directly comparable |
| Age | 60.2 | 1,091 days old |
The grade has not moved much since July, but the reason behind it has. In July the weakest component was contamination at 5.7, with saturation still generous at 97.2 on a much wider spread. Today the contamination number is worse, the headroom is gone, and the harness is no longer consistent. Same letter, different shape.
GPQA diamond still orders models, and its discrimination score of 100 says the ordering is real: a 97.85-point spread cannot be noise. But the benchmark's value as a decision input has narrowed, and the page that used to read "unsaturated, consistent harness, all top rows from one evaluator" now reads differently.
Use it to compare models that carry a reproduced score from the same evaluator, and treat the sub-96 band as the decision range. Inside the top ten the scores are separated by just over 3 points on a 198-item test, so the ranking is a tie in practice. Do not read the top six rows as measurements: they carry no source, no evaluator, and no reproduction. And do not read any score above 90 as a pure reasoning number, because 97.3% of the dated rows come from models that could have seen the public test set.
What this number does not prove: that Gemini 4 Carbon reasons better than Gemini 3.7 Flash. It proves that one model holds a score 2.36 points higher on a benchmark whose test set has been public for 1,091 days, and that the higher score is the one with no recorded evaluator behind it.