OpenAI·released 2024-07-181 source
GPT-4o mini benchmark scores: 20 benchmarks tracked. 20% of its scores are independently reproduced — the rest are self-reported or unverified.
Leads on 0 of 20 benchmarks·best result 91.3 on GSM8K (self-reported)·4 of 20 independently reproduced·$0.15/$0.6 per M tokens
Source: models.dev, OpenRouter · See every model’s pricing →
| Benchmark | Score | Evidence status | Measured |
|---|---|---|---|
| GSM8K | 91.3accuracy (%) | self-reported· optimizedT1 | 2024-07-18Phi-3 Technical Report: A Highly Capable Language Model Locally on Your Phone |
| PIQA | 77.4accuracy (%) | self-reported· optimizedT1 | 2024-07-18Phi-3 Technical Report: A Highly Capable Language Model Locally on Your Phone |
| MMLU | 75.73accuracy (%) | self-reported· optimizedT1 | 2024-07-18Phi-4 Technical Report |
| Lech Mazur Writing | 67.2accuracy (%) | unverifiedT1 | 2024-07-18lechmazur/writing Github repository |
| GeoBench | 64accuracy (%) | unverifiedT1 | 2024-07-18GeoBench leaderboard |
| MATH level 5 | 52.63accuracy (%) | reproduced· optimizedT1 | 2024-07-18Epoch AI |
| DTBench | 24accuracy (%) | unverifiedT1 | 2024-07-18https://conceptualreasoning.ai/dtbench |
| Balrog | 17.4accuracy (%) | unverifiedT1 | 2024-07-18https://balrogai.com/ |
| GPQA diamond | 16.96accuracy (%) | reproduced· optimizedT1 | 2024-07-18Epoch AI |
| LMCA | 12.2accuracy (%) | unverifiedT1 | 2024-07-18https://conceptualreasoning.ai/lmca |
| WeirdML | 11.76accuracy (%) | unverifiedT1 | 2024-07-18https://htihle.github.io/weirdml.html |
| SimpleQA Verified | 8.3accuracy (%) | unverifiedT2 | 2024-07-18 |
| OTIS Mock AIME 2024-2025 | 6.85accuracy (%) | reproduced· optimizedT1 | 2024-07-18Epoch AI |
| Aider polyglot | 3.6accuracy (%) | unverified· optimizedT1 | 2024-07-18https://aider.chat/docs/leaderboards/ |
| Mystery Game Puzzles | 3.06accuracy (%) | unverifiedT2 | 2024-07-18 |
| VPCT | 1accuracy (%) | unverifiedT1 | 2024-07-18VPCT leaderboard |
| FrontierMath-Tiers-1-3-v2-Private | 0.7accuracy (%) | unverifiedT2 | 2024-07-18 |
| Chess Puzzles | 0accuracy (%) | reproducedT1 | 2024-07-18Epoch AI |
| SimpleBench | 0accuracy (%) | unverifiedT1 | 2024-07-18SimpleBench Leaderboard |
| ARC-AGI-2 | 0accuracy (%) | unverifiedT2 | 2024-07-18 |
Claims drawn from cited facts, not live model generation.
This model was developed by OpenAI and released in 2024, making it a recent arrival from an established AI lab. As a 2024 release from OpenAI, it belongs to the current generation of that developer's models, though no parameter count is offered here to establish its scale.
4 cited facts
This model is scored on 20 tracked benchmarks, but it holds no current top score on any of them. Its benchmark record has been independently reproduced, meaning outside evaluation backs the scores up rather than leaving them as vendor-claimed figures.
3 cited facts
On average this model trails the best reported score by 59.02 points across its benchmarks, a score gap measured under a disclosed harness rather than a capability claim; a gap of that width reads plainly as well behind the leaders. It holds the top score on 0 benchmarks, so there is no category leadership here yet — it sits outside the front of the pack on every benchmark counted.
2 cited facts
6 facts cross-checked across data sources: 3 corroborated, 2 single-source, 1 disputed.
Source facts, citations, and refresh stamp for this record.
Sources: epoch_benchmarks, modelsdev_models, openrouter_models
GPT-4o mini is an AI model developed by OpenAI, released 2024-07-18. Its benchmark record below tags every score as independently reproduced, self-reported, or unverified.
GPT-4o mini has recorded scores on 20 benchmarks, each shown with its evidence status.
GPT-4o mini has recorded scores on 20 benchmarks — Lech Mazur Writing, PIQA, GPQA diamond, MATH level 5, OTIS Mock AIME 2024-2025, DTBench, and 14 more. The full table above shows each score with its evidence status.
4 of 20 recorded scores (20%) are independently reproduced rather than self-reported by the lab.
GPT-4o mini has 20 tracked claims: 4 independently reproduced, 3 self-reported, 13 unverified.
Listed API pricing: $0.15 per million input tokens, $0.6 per million output tokens. See the pricing block for the full breakdown.