Meta AI·released 2024-07-231 source
Llama 3.1-70B benchmark scores: 9 benchmarks tracked. 33% of its scores are independently reproduced — the rest are self-reported or unverified.
Leads on 0 of 9 benchmarks·best result 73.47 on MMLU (unverified)·3 of 9 independently reproduced
| Benchmark | Score | Evidence status | Measured |
|---|---|---|---|
| MMLU | 73.47accuracy (%) | unverified· optimizedT1 | 2024-07-23Stanford CRFM Leaderboard |
| MATH level 5 | 36.68accuracy (%) | reproduced· optimizedT1 | 2024-07-23Epoch AI |
| DTBench | 33.33accuracy (%) | unverifiedT1 | 2024-07-23https://conceptualreasoning.ai/dtbench |
| Balrog | 27.9accuracy (%) | unverifiedT1 | 2024-07-23https://balrogai.com/ |
| GPQA diamond | 25.59accuracy (%) | reproduced· optimizedT1 | 2024-07-23Epoch AI |
| LMCA | 17.47accuracy (%) | unverifiedT1 | 2024-07-23https://conceptualreasoning.ai/lmca |
| WeirdML | 8.97accuracy (%) | unverifiedT1 | 2024-07-23https://htihle.github.io/weirdml.html |
| The Agent Company | 6.9accuracy (%) | unverifiedT1 | 2024-07-23TheAgentCompany experiment results github |
| OTIS Mock AIME 2024-2025 | 3.51accuracy (%) | reproduced· optimizedT1 | 2024-07-23Epoch AI |
Claims drawn from cited facts, not live model generation.
Developed by Meta AI and released in 2024, this model is a large-scale system with 70 billion parameters. That scale implies more headroom for capability but higher training and inference costs than smaller, more efficient alternatives.
4 cited facts
This model is scored on nine tracked benchmarks, but it holds no current top score on any of them. Its benchmark record has been independently reproduced, meaning outside evaluation backs up the reported scores rather than leaving them as unverified vendor claims.
3 cited facts
On average across its benchmarks this model trails the state-of-the-art score by 56.5 points, a wide gap under the disclosed harness that reads as well behind the leaders rather than at or near the frontier. Consistently, it holds the top score on none of those benchmarks (sota leader count of 0), so there is no category leadership here yet; these are score gaps under a single harness, not capability claims.
2 cited facts
2 facts cross-checked across data sources: 2 single-source.
Source facts, citations, and refresh stamp for this record.
Sources: curated_param_counts, epoch_benchmarks
Llama 3.1-70B is an AI model developed by Meta AI, released 2024-07-23. Its benchmark record below tags every score as independently reproduced, self-reported, or unverified.
Llama 3.1-70B has recorded scores on 9 benchmarks, each shown with its evidence status.
Llama 3.1-70B has recorded scores on 9 benchmarks — GPQA diamond, MATH level 5, OTIS Mock AIME 2024-2025, DTBench, WeirdML, MMLU, and 3 more. The full table above shows each score with its evidence status.
3 of 9 recorded scores (33%) are independently reproduced rather than self-reported by the lab.
Llama 3.1-70B has 9 tracked claims: 3 independently reproduced, 6 unverified.