Anthropic·released 2024-10-221 source
Claude 3.5 Sonnet (October 2024) benchmark scores: 19 benchmarks tracked. 26% of its scores are independently reproduced — the rest are self-reported or unverified.
Leads on 0 of 19 benchmarks·best result 83.07 on MMLU (unverified)·5 of 19 independently reproduced
| Benchmark | Score | Evidence status | Measured |
|---|---|---|---|
| MMLU | 83.07accuracy (%) | unverified· optimizedT1 | 2024-10-22Stanford CRFM Leaderboard |
| Lech Mazur Writing | 80.3accuracy (%) | unverifiedT1 | 2024-10-22lechmazur/writing Github repository |
| GeoBench | 62accuracy (%) | unverifiedT1 | 2024-10-22GeoBench leaderboard |
| MATH level 5 | 56.95accuracy (%) | reproduced· optimizedT1 | 2024-10-22Epoch AI |
| Aider polyglot | 51.6accuracy (%) | unverified· optimizedT1 | 2024-10-22https://aider.chat/docs/leaderboards/ |
| CadEval | 48accuracy (%) | unverifiedT1 | 2024-10-22CadEval Dashboard |
| DTBench | 46.4accuracy (%) | unverifiedT1 | 2024-10-22https://conceptualreasoning.ai/dtbench |
| METR Time Horizons | 45.24accuracy (%) | self-reportedT1 | 2024-10-22https://metr.org/blog/2025-03-19-measuring-ai-ability-to-complete-long-tasks/ |
| GPQA diamond | 40.4accuracy (%) | reproduced· optimizedT1 | 2024-10-22Epoch AI |
| WeirdML | 39.97accuracy (%) | unverifiedT1 | 2024-10-22https://htihle.github.io/weirdml.html |
| Balrog | 32.6accuracy (%) | unverifiedT1 | 2024-10-22https://balrogai.com/ |
| SimpleBench | 29.68accuracy (%) | unverifiedT1 | 2024-10-22SimpleBench Leaderboard |
| The Agent Company | 24accuracy (%) | unverifiedT1 | 2024-10-22TheAgentCompany experiment results github |
| OTIS Mock AIME 2024-2025 | 8.38accuracy (%) | reproduced· optimizedT1 | 2024-10-22Epoch AI |
| GSO-Bench | 4.6accuracy (%) | unverifiedT1 | 2024-10-22GSO Leaderboard |
| FrontierMath-2025-02-28-Private | 3.63accuracy (%) | reproduced· optimizedT1 | 2024-10-22Epoch AI |
| FrontierMath-Tier-4-2025-07-01-Private | 0accuracy (%) | reproducedT1 | 2024-10-22Epoch AI |
| VPCT | 0accuracy (%) | unverifiedT1 | 2024-10-22VPCT leaderboard |
| HLE | 0accuracy (%) | unverifiedT2 | 2024-10-22 |
Claims drawn from cited facts, not live model generation.
This model was developed by Anthropic and released in 2024, placing it in the current generation of frontier language models from that lab. Because no parameter count is available, its scale cannot be quantified beyond its recent vintage — a 2024 release date implies access to contemporary training methods and architecture, though the exact capacity and its accompanying efficiency or cost trade-offs remain unstated.
3 cited facts
This model is scored on 19 tracked benchmarks, and it currently holds no top score on any of them. Its record carries independent verification: outside evaluation backs the reported scores up, so they can be treated as reproduced results rather than vendor claims.
3 cited facts
On average this model trails the state-of-the-art leader by 41.12 points across its benchmarks under the disclosed harness — a wide gap that reads as well behind the front of the pack, not merely a rounding-level shortfall. Consistently, it holds the top score on 0 benchmarks, so there is no category where it currently leads; the standing is one of being behind rather than at parity or ahead.
3 cited facts
1 facts cross-checked across data sources: 1 single-source.
Source facts, citations, and refresh stamp for this record.
Sources: epoch_benchmarks
Claude 3.5 Sonnet (October 2024) is an AI model developed by Anthropic, released 2024-10-22. Its benchmark record below tags every score as independently reproduced, self-reported, or unverified.
Claude 3.5 Sonnet (October 2024) has recorded scores on 19 benchmarks, each shown with its evidence status.
Claude 3.5 Sonnet (October 2024) has recorded scores on 19 benchmarks — Lech Mazur Writing, GPQA diamond, MATH level 5, OTIS Mock AIME 2024-2025, DTBench, WeirdML, and 13 more. The full table above shows each score with its evidence status.
5 of 19 recorded scores (26%) are independently reproduced rather than self-reported by the lab.
Claude 3.5 Sonnet (October 2024) has 19 tracked claims: 5 independently reproduced, 1 self-reported, 13 unverified.