Microsoft·released 2024-04-23
Scored on 10 benchmarks. 20% of its scores are independently reproduced — the rest are self-reported or unverified.
Every score, ranked, colored by evidence status.
| Benchmark | Score | Evidence status | Measured |
|---|---|---|---|
| ARC AI2 | 88.8accuracy (%) | self-reported· optimized | 2024-04-23 |
| OpenBookQA | 83.2accuracy (%) | self-reported· optimized | 2024-04-23 |
| HellaSwag | 76.53accuracy (%) | self-reported· optimized | 2024-04-23 |
| BBH | 75.2accuracy (%) | self-reported· optimized | 2024-04-23 |
| TriviaQA | 73.9accuracy (%) | self-reported· optimized | 2024-04-23 |
| MMLU | 70.67accuracy (%) | unverified· optimized | 2024-04-23 |
| Winogrande | 63accuracy (%) | self-reported· optimized | 2024-04-23 |
| ANLI | 33.7accuracy (%) | self-reported· optimized | 2024-04-23 |
| MATH level 5 | 17.56accuracy (%) | reproduced· optimized | 2024-04-23 |
| GPQA diamond | 3.45accuracy (%) | reproduced· optimized | 2024-04-23 |