Anthropic·released 2023-07-11
Scored on 5 benchmarks. 60% of its scores are independently reproduced — the rest are self-reported or unverified.
Every score, ranked, colored by evidence status.
| Benchmark | Score | Evidence status | Measured |
|---|---|---|---|
| TriviaQA | 87.5accuracy (%) | self-reported· optimized | 2023-07-11 |
| MMLU | 71.33accuracy (%) | self-reported· optimized | 2023-07-11 |
| GPQA diamond | 12.88accuracy (%) | reproduced· optimized | 2023-07-11 |
| MATH level 5 | 11.73accuracy (%) | reproduced· optimized | 2023-07-11 |
| OTIS Mock AIME 2024-2025 | 2.4accuracy (%) | reproduced· optimized | 2023-07-11 |