Google DeepMind·released 2024-05-231 source
Gemini 1.5 Flash (May 2024) benchmark scores: 6 benchmarks tracked. 50% of its scores are independently reproduced — the rest are self-reported or unverified.
Leads on 0 of 6 benchmarks·best result 82.4 on GSM8K (self-reported)·3 of 6 independently reproduced
| Benchmark | Score | Evidence status | Measured |
|---|---|---|---|
| GSM8K | 82.4accuracy (%) | self-reported· optimizedT1 | 2024-05-23Phi-3 Technical Report: A Highly Capable Language Model Locally on Your Phone |
| MMLU | 70.53accuracy (%) | unverified· optimizedT1 | 2024-05-23Stanford CRFM Leaderboard |
| MATH level 5 | 25.12accuracy (%) | reproduced· optimizedT1 | 2024-05-23Epoch AI |
| GPQA diamond | 20.5accuracy (%) | reproduced· optimizedT1 | 2024-05-23Epoch AI |
| DTBench | 18.97accuracy (%) | unverifiedT1 | 2024-05-23https://conceptualreasoning.ai/dtbench |
| OTIS Mock AIME 2024-2025 | 3.79accuracy (%) | reproduced· optimizedT1 | 2024-05-23Epoch AI |
Claims drawn from cited facts, not live model generation.
This model was developed by Google DeepMind and released in 2024, placing it within that year's generation of models. No parameter count is disclosed here, so its scale cannot be stated and only the 2024 vintage and the developer's identity frame what this model is.
4 cited facts
This model is scored on 6 tracked benchmarks. It holds no current top score on any of them. The record can be trusted: the scores have been independently reproduced by outside evaluation.
3 cited facts
On average, this model trails the SOTA leader by 57.44 points across its benchmarks, a wide gap that places it well behind the leaders rather than at the frontier. It holds the top score on none of the benchmarks, so there is no category leadership yet.
2 cited facts
1 facts cross-checked across data sources: 1 single-source.
Source facts, citations, and refresh stamp for this record.
Sources: epoch_benchmarks
Gemini 1.5 Flash (May 2024) is an AI model developed by Google DeepMind, released 2024-05-23. Its benchmark record below tags every score as independently reproduced, self-reported, or unverified.
Gemini 1.5 Flash (May 2024) has recorded scores on 6 benchmarks, each shown with its evidence status.
Gemini 1.5 Flash (May 2024) has recorded scores on 6 benchmarks — GPQA diamond, MATH level 5, OTIS Mock AIME 2024-2025, DTBench, MMLU, GSM8K. The full table above shows each score with its evidence status.
3 of 6 recorded scores (50%) are independently reproduced rather than self-reported by the lab.
Gemini 1.5 Flash (May 2024) has 6 tracked claims: 3 independently reproduced, 1 self-reported, 2 unverified.