Google DeepMind·released 2024-05-141 source
Gemini 1.5 Pro (May 2024) benchmark scores: 6 benchmarks tracked, leading 1. 50% of its scores are independently reproduced — the rest are self-reported or unverified.
Leads on 1 of 6 benchmarks·best result 85.6 on BBH (unverified)·3 of 6 independently reproduced
Head-to-headGemini 1.5 Pro (May 2024) vs DeepSeek-V3Leads on: BBH
| Benchmark | Score | Evidence status | Measured |
|---|---|---|---|
| BBH | 85.6accuracy (%) | unverified· optimizedT1 | 2024-05-14Gemini 1.5 report |
| MMLU | 81.2accuracy (%) | unverified· optimizedT1 | 2024-05-14Gemini 1.5 Report |
| MATH level 5 | 40.75accuracy (%) | reproduced· optimizedT1 | 2024-05-14Epoch AI |
| GPQA diamond | 27.82accuracy (%) | reproduced· optimizedT1 | 2024-05-14Epoch AI |
| DTBench | 27.45accuracy (%) | unverifiedT1 | 2024-05-14https://conceptualreasoning.ai/dtbench |
| OTIS Mock AIME 2024-2025 | 6.71accuracy (%) | reproduced· optimizedT1 | 2024-05-14Epoch AI |
Claims drawn from cited facts, not live model generation.
This model was developed by Google DeepMind and was released in 2024.
2 cited facts
This model is scored on 6 tracked benchmarks. It currently holds the top score on 1 of them. Its benchmark record is independently reproduced, meaning outside evaluation backs up the scores.
3 cited facts
Across the benchmarks with a disclosed harness, this model trails the leading score by 48.34 points on average, a wide gap that places it well behind the front of the pack. It holds the top score on 1 benchmark, so its overall standing is that of a trailing system rather than a category leader.
2 cited facts
1 facts cross-checked across data sources: 1 single-source.
Source facts, citations, and refresh stamp for this record.
Sources: epoch_benchmarks
Gemini 1.5 Pro (May 2024) is an AI model developed by Google DeepMind, released 2024-05-14. Its benchmark record below tags every score as independently reproduced, self-reported, or unverified.
Gemini 1.5 Pro (May 2024) has recorded scores on 6 benchmarks, each shown with its evidence status.
Gemini 1.5 Pro (May 2024) has recorded scores on 6 benchmarks — GPQA diamond, MATH level 5, OTIS Mock AIME 2024-2025, DTBench, MMLU, BBH. The full table above shows each score with its evidence status.
3 of 6 recorded scores (50%) are independently reproduced rather than self-reported by the lab.
Gemini 1.5 Pro (May 2024) has 6 tracked claims: 3 independently reproduced, 3 unverified.