Verdict
DeepSeek-V3 leads 6–4 across 11 shared benchmarks.
DeepSeek-V3 6 · Llama 3.1-405B 4 · 1 tied · higher isn't always better — check the caveats below.
Reported scores — protocols may differ. How we compare
Reasoning: DeepSeek-V3 leads 2–1 across 4 shared reasoning benchmarks (largest gap: GPQA diamond, 42.05 vs 34.55 — single evaluator).
Commonsense Reasoning: Llama 3.1-405B leads 3–0 across 3 shared commonsense reasoning benchmarks (largest gap: Winogrande, 78.4 vs 70.4 — single evaluator).
Knowledge: DeepSeek-V3 leads 2–0 across 2 shared knowledge benchmarks (largest gap: MMLU, 82.93 vs 79.33 — single evaluator).
Math: DeepSeek-V3 leads 2–0 across 2 shared math benchmarks (largest gap: MATH level 5, 64.85 vs 49.77 — single evaluator).
Vendor claims — compare against the measured scores above. A claim is what a developer says about its own model, not an independent measurement.
Coding
“DeepSeek-V3 achieves the best performance on most benchmarks, especially on math and code tasks.”
General
“At an economical cost of only 2.664M H800 GPU hours, we complete the pre-training of DeepSeek-V3 on 14.8T tokens, producing the currently strongest open-source base model.”
“DeepSeek-V3 outperforms other open-source models and achieves performance comparable to leading closed-source models.”
Reasoning
“Our pipeline elegantly incorporates the verification and reflection patterns of R1 into DeepSeek-V3 and notably improves its reasoning performance.”
Context Handling
“These are multilingual and have a significantly longer context length of 128K, state-of-the-art tool use, and overall stronger reasoning capabilities.”
General
“Llama 3.1 405B is in a class of its own, with unmatched flexibility, control, and state-of-the-art capabilities that rival the best closed source models.”
“Llama 3.1 405B is the first openly available model that rivals the top AI models when it comes to state-of-the-art capabilities in general knowledge, steerability, math, tool use”
DeepSeek-V3 leads 6–4 across 11 shared benchmarks. DeepSeek-V3 leads 6 benchmarks and Llama 3.1-405B leads 4, with 1 tied. Higher isn't always better — see the integrity caveats on each benchmark.
DeepSeek-V3 and Llama 3.1-405B have 11 benchmarks in common in our data — those are the only rows where a direct, apples-to-apples comparison is drawn.
Llama 3.1-405B leads commonsense reasoning 3–0. DeepSeek-V3 leads knowledge 2–0.