Mistral AI·released 2024-04-171 source
Mixtral 8x22B benchmark scores: 6 benchmarks tracked. 33% of its scores are independently reproduced — the rest are self-reported or unverified.
Leads on 0 of 6 benchmarks·best result 70.4 on MMLU (unverified)·2 of 6 independently reproduced
| Benchmark | Score | Evidence status | Measured |
|---|---|---|---|
| MMLU | 70.4accuracy (%) | unverified· optimizedT1 | 2024-04-17Stanford CRFM Leaderboard |
| DTBench | 25.23accuracy (%) | unverifiedT1 | 2024-04-17https://conceptualreasoning.ai/dtbench |
| MATH level 5 | 24.24accuracy (%) | reproduced· optimizedT1 | 2024-04-17Epoch AI |
| GPQA diamond | 12.08accuracy (%) | reproduced· optimizedT1 | 2024-04-17Epoch AI |
| Cybench | 7.5accuracy (%) | unverified· optimizedT1 | 2024-04-17Cybench leaderboard |
| WeirdML | 3.17accuracy (%) | unverifiedT1 | 2024-04-17https://htihle.github.io/weirdml.html |
Claims drawn from cited facts, not live model generation.
This model originates from Mistral AI, the developer behind it. Its vintage places it in 2024, marking it as a recent release rather than an older generation.
2 cited facts
This model is scored on 6 tracked benchmarks, and it holds no current top score on any of them. Its results have been independently reproduced, meaning outside evaluation backs the scores up.
3 cited facts
Across its benchmarks the model trails the leading score by an average of 69.54 points, a score gap measured under the disclosed harness rather than a statement about capability; a margin this wide reads as well behind the leaders rather than merely competitive. Consistently, it holds the top score on 0 of its benchmarks, which means it has no category leadership to point to on any of them.
2 cited facts
1 facts cross-checked across data sources: 1 single-source.
Source facts, citations, and refresh stamp for this record.
Sources: epoch_benchmarks
Mixtral 8x22B is an AI model developed by Mistral AI, released 2024-04-17. Its benchmark record below tags every score as independently reproduced, self-reported, or unverified.
Mixtral 8x22B has recorded scores on 6 benchmarks, each shown with its evidence status.
Mixtral 8x22B has recorded scores on 6 benchmarks — GPQA diamond, MATH level 5, DTBench, WeirdML, MMLU, Cybench. The full table above shows each score with its evidence status.
2 of 6 recorded scores (33%) are independently reproduced rather than self-reported by the lab.
Mixtral 8x22B has 6 tracked claims: 2 independently reproduced, 4 unverified.