Verdict
Llama 3.1-405B leads 8–0 across 8 shared benchmarks.
Llama 3.1-405B 8 · Mixtral 8x7B 0 · higher isn't always better — check the caveats below.
Reported scores — protocols may differ. How we compare
Commonsense Reasoning: Llama 3.1-405B leads 3–0 across 3 shared commonsense reasoning benchmarks (largest gap: Winogrande, 78.4 vs 54.4 — single evaluator).
Knowledge: Llama 3.1-405B leads 2–0 across 2 shared knowledge benchmarks (largest gap: MMLU, 79.33 vs 60.8 — single evaluator).
Reasoning: Llama 3.1-405B leads 2–0 across 2 shared reasoning benchmarks (largest gap: GPQA diamond, 34.55 vs 7.45 — single evaluator).
Vendor claims — compare against the measured scores above. A claim is what a developer says about its own model, not an independent measurement.
Context Handling
“These are multilingual and have a significantly longer context length of 128K, state-of-the-art tool use, and overall stronger reasoning capabilities.”
General
“Llama 3.1 405B is in a class of its own, with unmatched flexibility, control, and state-of-the-art capabilities that rival the best closed source models.”
“Llama 3.1 405B is the first openly available model that rivals the top AI models when it comes to state-of-the-art capabilities in general knowledge, steerability, math, tool use”
Coding
“It shows strong performance in code generation.”
Context Handling
“It gracefully handles a context of 32k tokens.”
General
“It handles English, French, Italian, German and Spanish.”
Reasoning
“Mixtral matches or outperforms Llama 2 70B, as well as GPT3.5, on most benchmarks.”
Speed / Latency
“Mixtral outperforms Llama 2 70B on most benchmarks with 6x faster inference.”
Llama 3.1-405B leads 8–0 across 8 shared benchmarks. Llama 3.1-405B leads 8 benchmarks and Mixtral 8x7B leads 0. Higher isn't always better — see the integrity caveats on each benchmark.
Llama 3.1-405B and Mixtral 8x7B have 8 benchmarks in common in our data — those are the only rows where a direct, apples-to-apples comparison is drawn.
Llama 3.1-405B leads commonsense reasoning 3–0. Llama 3.1-405B leads knowledge 2–0.