Mistral AI·released 2023-12-11
Scored on 11 benchmarks. 18% of its scores are independently reproduced — the rest are self-reported or unverified.
Every score, ranked, colored by evidence status.
| Benchmark | Score | Evidence status | Measured |
|---|---|---|---|
| ARC AI2 | 83.07accuracy (%) | unverified· optimized | 2023-12-11 |
| HellaSwag | 82.27accuracy (%) | unverified· optimized | 2023-12-11 |
| TriviaQA | 82.2accuracy (%) | self-reported· optimized | 2023-12-11 |
| OpenBookQA | 81.07accuracy (%) | self-reported· optimized | 2023-12-11 |
| GSM8K | 74.4accuracy (%) | unverified· optimized | 2023-12-11 |
| PIQA | 67.2accuracy (%) | unverified· optimized | 2023-12-11 |
| MMLU | 60.8accuracy (%) | unverified· optimized | 2023-12-11 |
| Winogrande | 54.4accuracy (%) | unverified· optimized | 2023-12-11 |
| ANLI | 32.8accuracy (%) | self-reported· optimized | 2023-12-11 |
| MATH level 5 | 9.95accuracy (%) | reproduced· optimized | 2023-12-11 |
| GPQA diamond | 7.45accuracy (%) | reproduced· optimized | 2023-12-11 |