Verdict
DeepSeek-R1 leads 5–0 across 5 shared benchmarks.
DeepSeek-R1 5 · Llama 3.1-405B 0 · higher isn't always better — check the caveats below.
Reported scores — protocols may differ. How we compare
Math: DeepSeek-R1 leads 2–0 across 2 shared math benchmarks (largest gap: OTIS Mock AIME 2024-2025, 53.29 vs 9.63 — single evaluator).
Reasoning: DeepSeek-R1 leads 2–0 across 2 shared reasoning benchmarks (largest gap: GPQA diamond, 62.29 vs 34.55 — single evaluator).
Vendor claims — compare against the measured scores above. A claim is what a developer says about its own model, not an independent measurement.
Reasoning
“After these steps, we obtained a checkpoint referred to as DeepSeek-R1, which achieves performance on par with OpenAI-o1-1217.”
“For education-oriented knowledge benchmarks such as MMLU, MMLU-Pro, and GPQA Diamond, DeepSeek-R1 demonstrates superior performance compared to DeepSeek-V3.”
Context Handling
“These are multilingual and have a significantly longer context length of 128K, state-of-the-art tool use, and overall stronger reasoning capabilities.”
General
“Llama 3.1 405B is in a class of its own, with unmatched flexibility, control, and state-of-the-art capabilities that rival the best closed source models.”
“Llama 3.1 405B is the first openly available model that rivals the top AI models when it comes to state-of-the-art capabilities in general knowledge, steerability, math, tool use”
DeepSeek-R1 leads 5–0 across 5 shared benchmarks. DeepSeek-R1 leads 5 benchmarks and Llama 3.1-405B leads 0. Higher isn't always better — see the integrity caveats on each benchmark.
DeepSeek-R1 and Llama 3.1-405B have 5 benchmarks in common in our data — those are the only rows where a direct, apples-to-apples comparison is drawn.
DeepSeek-R1 leads math 2–0. DeepSeek-R1 leads reasoning 2–0.