Meta AI·released 2024-04-18
Scored on 9 benchmarks. 33% of its scores are independently reproduced — the rest are self-reported or unverified.
Every score, ranked, colored by evidence status.
| Benchmark | Score | Evidence status | Measured |
|---|---|---|---|
| ARC AI2 | 77.07accuracy (%) | self-reported· optimized | 2024-04-18 |
| OpenBookQA | 76.8accuracy (%) | self-reported· optimized | 2024-04-18 |
| TriviaQA | 67.7accuracy (%) | self-reported· optimized | 2024-04-18 |
| MMLU | 58.4accuracy (%) | unverified· optimized | 2024-04-18 |
| Winogrande | 51.4accuracy (%) | unverified· optimized | 2024-04-18 |
| ANLI | 35.95accuracy (%) | self-reported· optimized | 2024-04-18 |
| MATH level 5 | 6.13accuracy (%) | reproduced· optimized | 2024-04-18 |
| GPQA diamond | 1.43accuracy (%) | reproduced· optimized | 2024-04-18 |
| OTIS Mock AIME 2024-2025 | 0.73accuracy (%) | reproduced· optimized | 2024-04-18 |