Mistral AI·released 2024-07-18
Scored on 5 benchmarks. 40% of its scores are independently reproduced — the rest are self-reported or unverified.
Every score, ranked, colored by evidence status.
| Benchmark | Score | Evidence status | Measured |
|---|---|---|---|
| GSM8K | 84.2accuracy (%) | self-reported· optimized | 2024-07-18 |
| PIQA | 67accuracy (%) | self-reported· optimized | 2024-07-18 |
| Balrog | 17.6accuracy (%) | unverified | 2024-07-18 |
| MATH level 5 | 10.83accuracy (%) | reproduced· optimized | 2024-07-18 |
| GPQA diamond | 6.52accuracy (%) | reproduced· optimized | 2024-07-18 |