SimpleQA Verified
Integrity rank #28 of 65 · 81 models scored · top score 76 · Gemini 4 Argon
LLM short-form factuality on questions with single, indisputable answers (de-noised, de-duplicated, topically balanced from OpenAI's original SimpleQA).
Strongest on discrimination, weakest on contamination resistance. Caveat: the public test set does not rule out training-data contamination.
Reported scores — protocols may differ. How we compare
| # | Model | Score | Evidence | Measured |
|---|---|---|---|---|
| 1 | Gemini 4 Argon | 76 | unverifiedT2 | 2026-09-30 |
| 2 | GPT-6 Astra | 75.6 | unverifiedT2 | 2026-09-03 |
| 3 | GPT-6.1 Sol | 73.9 | unverifiedT2 | 2026-09-29 |
| 4 | Gemini 3.1 Pro | 73.5 | unverifiedT2 | 2026-02-19 |
| 5 | Claude Opus 5.5 | 72.2 | unverifiedT2 | 2026-09-22 |
| 6 | Claude Fable 5.1 | 70.8 | unverifiedT2 | 2026-09-01 |
| 7 | Claude Fable 5 | 70.7 | unverifiedT2 | 2026-06-09 |
| 8 | Gemini 3.8 Flash | 69.7 | unverifiedT2 | 2026-09-02 |
| 9 | GPT-5.6 Sol | 69.7 | unverifiedT2 | 2026-07-09 |
| 10 | Gemini 3.7 Flash | 69.2 | unverifiedT2 | 2026-08-13 |
| 11 | Gemini 3 Flash | 66.8 | unverifiedT2 | 2025-12-17 |
| 12 | Gemini 3.5 Flash | 66.2 | unverifiedT2 | 2026-05-19 |
| 13 | Gemini 3.6 Flash | 66.2 | unverifiedT2 | 2026-07-21 |
| 14 | GPT-5.5 | 63 | unverifiedT2 | 2026-04-23 |
| 15 | GPT-6 Sol | 60.7 | unverifiedT2 | 2026-09-22 |
| 16 | Muse Spark 1.2 | 60.3 | unverifiedT2 | 2026-08-05 |
| 17 | Claude Opus 5 | 59.9 | unverifiedT2 | 2026-07-24 |
| 18 | Muse Spark 1.1 | 57.79 | unverifiedT2 | 2026-07-09 |
| 19 | Grok 4.7 | 56 | unverifiedT2 | 2026-09-21 |
| 20 | Qwen3.7-Max | 55.77 | unverifiedT2 | 2026-05-19 |
| 21 | Claude Opus 4.8 | 53 | unverifiedT2 | 2026-05-28 |
| 22 | DeepSeek V4 Pro 0813 | 52.91 | unverifiedT2 | 2026-08-13 |
| 23 | Qwen 3.6 Max (Preview) | 52 | unverifiedT2 | 2026-04-20 |
| 24 | Claude Opus 4.7 | 51.7 | unverifiedT2 | 2026-04-16 |
| 25 | Kimi K3 | 50.6 | unverifiedT2 | 2026-07-16 |
| 26 | GPT-5 | 50.1 | unverifiedT2 | 2025-08-07 |
| 27 | o3 | 49.4 | unverifiedT2 | 2025-04-16 |
| 28 | Grok 4.6 | 49.3 | unverifiedT2 | 2026-08-12 |
| 29 | Qwen3-Max | 48.75 | unverifiedT2 | 2025-09-24 |
| 30 | Grok 4.5 | 48.3 | unverifiedT2 | 2026-07-08 |
| 31 | GPT-5.1 | 48 | unverifiedT2 | 2025-11-13 |
| 32 | Qwen3.8 Max (0902) | 47.3 | unverifiedT2 | 2026-09-01 |
| 33 | Claude Opus 4.6 | 47 | unverifiedT2 | 2026-02-05 |
| 34 | DeepSeek-V4-Pro | 46.99 | unverifiedT2 | 2026-04-24 |
| 35 | Claude Sonnet 5.5 | 46.5 | unverifiedT2 | 2026-09-28 |
| 36 | GPT-5.4 Pro | 46.3 | unverifiedT2 | 2026-03-05 |
| 37 | Qwen 3.8 Max | 45.8 | unverifiedT2 | 2026-08-02 |
| 38 | Claude Opus 4.5 | 45.7 | unverifiedT2 | 2025-11-24 |
| 39 | GPT-5.4 | 45.1 | unverifiedT2 | 2026-03-05 |
| 40 | Qwen 3.6 Plus | 44.14 | unverifiedT2 | 2026-03-31 |
| 41 | GPT-5.6 Terra | 43.2 | unverifiedT2 | 2026-07-09 |
| 42 | DeepSeek V4.1 Flash | 42.9 | unverifiedT2 | 2026-09-09 |
| 43 | GPT-6 Luna | 41.4 | unverifiedT2 | 2026-09-22 |
| 44 | o1 | 41.1 | unverifiedT2 | 2024-12-17 |
| 45 | GLM-5.3 | 41 | unverifiedT2 | 2026-08-14 |
| 46 | GPT-5.6 Luna | 41 | unverifiedT2 | 2026-07-09 |
| 47 | Qwen3-235B-A22B-Thinking (Jul 2025) | 40.44 | unverifiedT2 | 2025-07-25 |
| 48 | Inkling | 40.3 | unverifiedT2 | 2026-07-15 |
| 49 | GPT-5.2 | 37.1 | unverifiedT2 | 2025-12-11 |
| 50 | Kimi K2.7 Code | 36.5 | unverifiedT2 | 2026-06-12 |
| 51 | Claude Sonnet 4.6 | 35.5 | unverifiedT2 | 2026-02-17 |
| 52 | Kimi K2.6 | 34.9 | unverifiedT2 | 2026-04-20 |
| 53 | Kimi K2.5 | 34.3 | unverifiedT2 | 2026-01-27 |
| 54 | GLM-5.2 | 34.2 | unverifiedT2 | 2026-06-16 |
| 55 | GLM-5.1 | 34 | unverifiedT2 | 2026-04-07 |
| 56 | Claude Sonnet 5 | 33.7 | unverifiedT2 | 2026-06-30 |
| 57 | DeepSeek V4 Flash 0731 | 33.63 | unverifiedT2 | 2026-07-31 |
| 58 | Grok 4.3 Beta | 33.2 | unverifiedT2 | 2026-04-17 |
| 59 | GLM-4.7 | 32.2 | unverifiedT2 | 2025-12-22 |
| 60 | GPT-4.1 | 31.1 | unverifiedT2 | 2025-04-14 |
| 61 | Claude Sonnet 4.5 | 30.7 | unverifiedT2 | 2025-09-29 |
| 62 | Grok 4.20 | 30.2 | unverifiedT2 | 2026-02-17 |
| 63 | GPT-5.4 Mini | 29.4 | unverifiedT2 | 2026-03-17 |
| 64 | GPT-4o (Aug 2024) | 26 | unverifiedT2 | 2024-08-06 |
| 65 | Qwen 3.5 Plus (hosted 397B-A17B) | 25.35 | unverifiedT2 | 2026-02-16 |
| 66 | Claude Haiku 5.5 | 23.8 | unverifiedT2 | 2026-10-07 |
| 67 | GPT-5 mini | 21.6 | unverifiedT2 | 2025-08-07 |
| 68 | Qwen 3.5 Flash (hosted 35B-A3B) | 20.32 | unverifiedT2 | 2026-02-25 |
| 69 | Mistral Large 4 | 20 | unverifiedT2 | 2026-10-06 |
| 70 | o4-mini | 19.6 | unverifiedT2 | 2025-04-16 |
| 71 | Inkling-Small | 19.1 | unverifiedT2 | 2026-07-15 |
| 72 | Qwen 3.6 Flash | 15.92 | unverifiedT2 | 2026-04-27 |
| 73 | o3-mini | 15.3 | unverifiedT2 | 2025-01-31 |
| 74 | Claude Haiku 4.5 | 13.2 | unverifiedT2 | 2025-10-15 |
| 75 | GPT-4.1 mini | 12.7 | unverifiedT2 | 2025-04-14 |
| 76 | Claude 3 Opus | 12.6 | unverifiedT2 | 2024-02-29 |
| 77 | GPT-5 nano | 11.7 | unverifiedT2 | 2025-08-07 |
| 78 | GPT-5.4 Nano | 11.7 | unverifiedT2 | 2026-03-17 |
| 79 | Gemma 4 31B IT | 10.4 | unverifiedT2 | 2026-04-02 |
| 80 | GPT-4o mini | 8.3 | unverifiedT2 | 2024-07-18 |
| 81 | GPT-4.1 nano | 6 | unverifiedT2 | 2025-04-14 |
Continue from this leaderboard into the tightest head-to-head reads.
Weighted composite. Each dimension opens to the source-backed sub-signals it is computed from.
Claims drawn from cited facts, not live model generation.
This benchmark tests short-form factual answers to questions with a single unambiguous answer.
1 cited fact
This benchmark has scores from 81 tracked models, with a recorded score range spanning 70.0.
2 cited facts
Its Benchmark Integrity Index score is 87, with grade A and rank 28 out of 65 tracked benchmarks.
4 cited facts
It is not flagged as saturated. The top recorded score is 76.0, with 24.0 points of headroom. The computed top-score cluster contains 2 models.
4 cited facts
The harness classification is consistent harness. Test-set privacy is recorded as unknown. The recorded contamination-window ratio is 0.827.
3 cited facts
SimpleQA Verified: SimpleQA Verified as reported in Epoch AI's Capabilities Index CSV.
Gemini 4 Argon leads SimpleQA Verified at 76. The full leaderboard above lists every recorded measurement, not just the headline number.
81 models have recorded scores on SimpleQA Verified, spanning a score spread of 70.
tensor.news grades SimpleQA Verified A for integrity (score 87/100), ranking #28 of 65 benchmarks we assess across discrimination, saturation, contamination resistance, harness comparability, and freshness.
No — SimpleQA Verified still has headroom and continues to discriminate between models rather than bunching them at the ceiling.
Its test set is unknown, so contamination risk is on the table: a high score may partly reflect training-data overlap rather than capability.
Scores are reported under a consistent harness, so comparisons on SimpleQA Verified are reasonably apples-to-apples.
Every SimpleQA Verified measurement is source-backed and tagged reproduced, self-reported, or unverified; the underlying record cites arxiv.org/abs/2509.07968.