Google DeepMind·released 2025-11-18
Scored on 23 benchmarks, leading 3. 30% of its scores are independently reproduced — the rest are self-reported or unverified.
Leads on: Balrog, FrontierMath-Tier-4-2025-07-01-Private, VPCT
Every score, ranked, colored by evidence status.
| Benchmark | Score | Evidence status | Measured |
|---|---|---|---|
| OTIS Mock AIME 2024-2025 | 91.38accuracy (%) | reproduced· optimized | 2025-11-18 |
| GPQA diamond | 90.15accuracy (%) | reproduced· optimized | 2025-11-18 |
| VPCT | 86.5accuracy (%) | unverified | 2025-11-18 |
| GeoBench | 84accuracy (%) | unverified | 2025-11-18 |
| ARC-AGI | 75accuracy (%) | unverified· optimized | 2025-11-18 |
| SWE-Bench verified | 72.93accuracy (%) | reproduced· optimized | 2025-11-18 |
| SimpleQA Verified | 72.9accuracy (%) | reproduced | 2025-11-18 |
| SimpleBench | 71.68accuracy (%) | unverified | 2025-11-18 |
| WeirdML | 69.93accuracy (%) | unverified | 2025-11-18 |
| Terminal Bench | 69.4accuracy (%) | unverified· optimized | 2025-11-18 |
| FrontierMath-2025-02-28-Private | 65.96accuracy (%) | reproduced· optimized | 2025-11-18 |
| Balrog | 58.1accuracy (%) | unverified | 2025-11-18 |
| GDPval | 40.3accuracy (%) | unverified | 2025-11-18 |
| HLE | 34.37accuracy (%) | unverified | 2025-11-18 |
| FrontierMath-Tier-4-2025-07-01-Private | 31.25accuracy (%) | reproduced | 2025-11-18 |
| ARC-AGI-2 | 31.11accuracy (%) | unverified | 2025-11-18 |
| Chess Puzzles | 31accuracy (%) | reproduced | 2025-11-18 |
| GSO-Bench | 18.6accuracy (%) | unverified | 2025-11-18 |
| APEX-Agents | 18.4accuracy (%) | unverified | 2025-11-18 |
| PostTrainBench | 18.12accuracy (%) | unverified | 2025-11-18 |
| CL-bench | 15.8accuracy (%) | unverified | 2025-11-18 |
| CritPt | 6.9accuracy (%) | unverified | 2025-11-18 |
| Remote Labor Index | 1.25accuracy (%) | unverified | 2025-11-18 |