Anthropic·released 2024-10-221 source
Claude 3.5 Haiku benchmark scores: 12 benchmarks tracked. 33% of its scores are independently reproduced — the rest are self-reported or unverified.
Leads on 0 of 12 benchmarks·best result 73.5 on Lech Mazur Writing (unverified)·4 of 12 independently reproduced
| Benchmark | Score | Evidence status | Measured |
|---|---|---|---|
| Lech Mazur Writing | 73.5accuracy (%) | unverifiedT1 | 2024-10-22lechmazur/writing Github repository |
| MMLU | 65.73accuracy (%) | unverified· optimizedT1 | 2024-10-22Stanford CRFM Leaderboard |
| MATH level 5 | 46.36accuracy (%) | reproduced· optimizedT1 | 2024-10-22Epoch AI |
| GeoBench | 34accuracy (%) | unverifiedT1 | 2024-10-22GeoBench leaderboard |
| CadEval | 32accuracy (%) | unverifiedT1 | 2024-10-22CadEval Dashboard |
| WeirdML | 30.73accuracy (%) | unverifiedT1 | 2024-10-22https://htihle.github.io/weirdml.html |
| Aider polyglot | 28accuracy (%) | unverified· optimizedT1 | 2024-10-22https://aider.chat/docs/leaderboards/ |
| DTBench | 27.77accuracy (%) | unverifiedT1 | 2024-10-22https://conceptualreasoning.ai/dtbench |
| Balrog | 19.3accuracy (%) | unverifiedT1 | 2024-10-22https://balrogai.com/ |
| GPQA diamond | 17.51accuracy (%) | reproduced· optimizedT1 | 2024-10-22Epoch AI |
| OTIS Mock AIME 2024-2025 | 4.21accuracy (%) | reproduced· optimizedT1 | 2024-10-22Epoch AI |
| FrontierMath-2025-02-28-Private | 0.6accuracy (%) | reproduced· optimizedT1 | 2024-10-22Epoch AI |
Claims drawn from cited facts, not live model generation.
This model was developed by Anthropic and released in 2024.
2 cited facts
This model is scored on 12 tracked benchmarks, and it holds no current top score on any of them. Its results have been independently reproduced, meaning outside evaluation backs the scores up.
3 cited facts
Across its benchmarks, it trails the state-of-the-art leader by an average of 54.14 points, a score gap under a disclosed harness rather than a capability claim, which reads as well behind the leaders. It holds the top score on 0 benchmarks, meaning it has no category leadership yet.
2 cited facts
1 facts cross-checked across data sources: 1 single-source.
Source facts, citations, and refresh stamp for this record.
Sources: epoch_benchmarks
Claude 3.5 Haiku is an AI model developed by Anthropic, released 2024-10-22. Its benchmark record below tags every score as independently reproduced, self-reported, or unverified.
Claude 3.5 Haiku has recorded scores on 12 benchmarks, each shown with its evidence status.
Claude 3.5 Haiku has recorded scores on 12 benchmarks — Lech Mazur Writing, GPQA diamond, MATH level 5, OTIS Mock AIME 2024-2025, DTBench, WeirdML, and 6 more. The full table above shows each score with its evidence status.
4 of 12 recorded scores (33%) are independently reproduced rather than self-reported by the lab.
Claude 3.5 Haiku has 12 tracked claims: 4 independently reproduced, 8 unverified.