Mistral AI·released 2024-07-181 source
Mistral NeMo benchmark scores: 6 benchmarks tracked. 33% of its scores are independently reproduced — the rest are self-reported or unverified.
Leads on 0 of 6 benchmarks·best result 84.2 on GSM8K (self-reported)·2 of 6 independently reproduced·$0.15/$0.15 per M tokens
Source: models.dev, OpenRouter · See every model’s pricing →
| Benchmark | Score | Evidence status | Measured |
|---|---|---|---|
| GSM8K | 84.2accuracy (%) | self-reported· optimizedT1 | 2024-07-18Phi-3 Technical Report: A Highly Capable Language Model Locally on Your Phone |
| PIQA | 67accuracy (%) | self-reported· optimizedT1 | 2024-07-18Phi-3 Technical Report: A Highly Capable Language Model Locally on Your Phone |
| Balrog | 17.6accuracy (%) | unverifiedT1 | 2024-07-18https://balrogai.com/ |
| DTBench | 14.35accuracy (%) | unverifiedT1 | 2024-07-18https://conceptualreasoning.ai/dtbench |
| MATH level 5 | 10.83accuracy (%) | reproduced· optimizedT1 | 2024-07-18Epoch AI |
| GPQA diamond | 6.52accuracy (%) | reproduced· optimizedT1 | 2024-07-18Epoch AI |
Claims drawn from cited facts, not live model generation.
This model was developed by Mistral AI and released in 2024.
2 cited facts
This model is scored on 6 tracked benchmarks. It holds no current top score on any of them. Its record can be trusted: the scores have been independently reproduced by outside evaluation.
3 cited facts
On its benchmarks this model trails the leading score by an average of 53.09 points, a wide gap under the disclosed harness that reads as well behind the front of the pack rather than merely a couple of points off the frontier. It holds the top score on none of its benchmarks, meaning there is no category in which it currently shows genuine first-place leadership.
2 cited facts
6 facts cross-checked across data sources: 1 corroborated, 2 single-source, 3 disputed.
Source facts, citations, and refresh stamp for this record.
Sources: epoch_benchmarks, modelsdev_models, openrouter_models
Mistral NeMo is an AI model developed by Mistral AI, released 2024-07-18. Its benchmark record below tags every score as independently reproduced, self-reported, or unverified.
Mistral NeMo has recorded scores on 6 benchmarks, each shown with its evidence status.
Mistral NeMo has recorded scores on 6 benchmarks — PIQA, GPQA diamond, MATH level 5, DTBench, Balrog, GSM8K. The full table above shows each score with its evidence status.
2 of 6 recorded scores (33%) are independently reproduced rather than self-reported by the lab.
Mistral NeMo has 6 tracked claims: 2 independently reproduced, 2 self-reported, 2 unverified.
Listed API pricing: $0.15 per million input tokens, $0.15 per million output tokens (prices disputed across sources). See the pricing block for the full breakdown.