Verdict
Gemini 1.5 Pro (Sept 2024) leads 4–1 across 5 shared benchmarks.
DeepSeek-V3 1 · Gemini 1.5 Pro (Sept 2024) 4 · higher isn't always better — check the caveats below.
Reported scores — protocols may differ. How we compare
Math: Gemini 1.5 Pro (Sept 2024) leads 2–0 across 2 shared math benchmarks (largest gap: OTIS Mock AIME 2024-2025, 22.98 vs 15.75 — single evaluator).
Reasoning: Gemini 1.5 Pro (Sept 2024) leads 2–0 across 2 shared reasoning benchmarks (largest gap: SimpleBench, 12.52 vs 2.68 — single evaluator).
Vendor claims — compare against the measured scores above. A claim is what a developer says about its own model, not an independent measurement.
Coding
“DeepSeek-V3 achieves the best performance on most benchmarks, especially on math and code tasks.”
General
“At an economical cost of only 2.664M H800 GPU hours, we complete the pre-training of DeepSeek-V3 on 14.8T tokens, producing the currently strongest open-source base model.”
“DeepSeek-V3 outperforms other open-source models and achieves performance comparable to leading closed-source models.”
Reasoning
“Our pipeline elegantly incorporates the verification and reflection patterns of R1 into DeepSeek-V3 and notably improves its reasoning performance.”
No vendor claims on file for Gemini 1.5 Pro (Sept 2024).
Gemini 1.5 Pro (Sept 2024) leads 4–1 across 5 shared benchmarks. DeepSeek-V3 leads 1 benchmark and Gemini 1.5 Pro (Sept 2024) leads 4. Higher isn't always better — see the integrity caveats on each benchmark.
DeepSeek-V3 and Gemini 1.5 Pro (Sept 2024) have 5 benchmarks in common in our data — those are the only rows where a direct, apples-to-apples comparison is drawn.
Gemini 1.5 Pro (Sept 2024) leads math 2–0. Gemini 1.5 Pro (Sept 2024) leads reasoning 2–0.