Meta AI·released 2024-07-231 source
Llama 3.1-405B benchmark scores: 15 benchmarks tracked, leading 2. 20% of its scores are independently reproduced — the rest are self-reported or unverified.
Leads on 2 of 15 benchmarks·best result 93.73 on ARC AI2 (self-reported)·3 of 15 independently reproduced
Head-to-headLlama 3.1-405B vs Qwen2.5-72BLeads on: ARC AI2, Winogrande
| Benchmark | Score | Evidence status | Measured |
|---|---|---|---|
| ARC AI2 | 93.73accuracy (%) | self-reported· optimizedT1 | 2024-07-23DeepSeek-V3 Technical Report |
| HellaSwag | 85.6accuracy (%) | self-reported· optimizedT1 | 2024-07-23DeepSeek-V3 Technical Report |
| TriviaQA | 82.7accuracy (%) | self-reported· optimizedT1 | 2024-07-23DeepSeek-V3 Technical Report |
| MMLU | 79.33accuracy (%) | unverified· optimizedT1 | 2024-07-23Stanford CRFM Leaderboard |
| Winogrande | 78.4accuracy (%) | unverified· optimizedT1 | 2024-07-23The Llama 3 Herd of Models |
| BBH | 77.2accuracy (%) | self-reported· optimizedT1 | 2024-07-23DeepSeek-V3 Technical Report |
| PIQA | 71.8accuracy (%) | self-reported· optimizedT1 | 2024-07-23DeepSeek-V3 Technical Report |
| MATH level 5 | 49.77accuracy (%) | reproduced· optimizedT1 | 2024-07-23Epoch AI |
| DTBench | 35.63accuracy (%) | unverifiedT1 | 2024-07-23https://conceptualreasoning.ai/dtbench |
| GPQA diamond | 34.55accuracy (%) | reproduced· optimizedT1 | 2024-07-23Epoch AI |
| WeirdML | 21.38accuracy (%) | unverifiedT1 | 2024-07-23https://htihle.github.io/weirdml.html |
| OTIS Mock AIME 2024-2025 | 9.63accuracy (%) | reproduced· optimizedT1 | 2024-07-23Epoch AI |
| SimpleBench | 7.6accuracy (%) | unverifiedT1 | 2024-07-23SimpleBench Leaderboard |
| Cybench | 7.5accuracy (%) | unverified· optimizedT1 | 2024-07-23Cybench leaderboard |
| The Agent Company | 7.4accuracy (%) | unverifiedT1 | 2024-07-23TheAgentCompany experiment results github |
Claims drawn from cited facts, not live model generation.
Developed by Meta AI, this model was released in 2024. With 405 billion parameters, this model is frontier-scale, implying more capability headroom but also higher training and inference costs.
3 cited facts
This model is scored on 15 tracked benchmarks and currently holds 2 top scores. Its record is independently reproduced, meaning outside evaluation backs up the reported scores.
3 cited facts
Under a disclosed harness, this model trails the SOTA leader by an average of 37.13 points—a wide gap that reads as well behind the leaders. It does hold the top score on 2 benchmarks, indicating genuine category leadership on those specific tasks.
2 cited facts
2 facts cross-checked across data sources: 2 single-source.
Source facts, citations, and refresh stamp for this record.
Sources: curated_capability_claims, curated_param_counts, epoch_benchmarks
Llama 3.1-405B is an AI model developed by Meta AI, released 2024-07-23. Its benchmark record below tags every score as independently reproduced, self-reported, or unverified.
Llama 3.1-405B has recorded scores on 15 benchmarks, each shown with its evidence status.
Llama 3.1-405B has recorded scores on 15 benchmarks — PIQA, Winogrande, GPQA diamond, MATH level 5, OTIS Mock AIME 2024-2025, TriviaQA, and 9 more. The full table above shows each score with its evidence status.
3 of 15 recorded scores (20%) are independently reproduced rather than self-reported by the lab.
Llama 3.1-405B has 15 tracked claims: 3 independently reproduced, 5 self-reported, 7 unverified.