Verdict
phi-3-small 7.4B leads 5–2 across 7 shared benchmarks.
Mixtral 8x7B 2 · phi-3-small 7.4B 5 · higher isn't always better — check the caveats below.
Reported scores — protocols may differ. How we compare
Commonsense Reasoning: Evenly split 1–1 across 2 shared commonsense reasoning benchmarks (largest gap: HellaSwag, 82.27 vs 69.33 — single evaluator).
Knowledge: Evenly split 1–1 across 2 shared knowledge benchmarks (largest gap: TriviaQA, 82.2 vs 58.1 — single evaluator).
Reasoning: phi-3-small 7.4B leads 2–0 across 2 shared reasoning benchmarks (largest gap: ARC AI2, 87.6 vs 83.07 — single evaluator).
Vendor claims — compare against the measured scores above. A claim is what a developer says about its own model, not an independent measurement.
Coding
“It shows strong performance in code generation.”
Context Handling
“It gracefully handles a context of 32k tokens.”
General
“It handles English, French, Italian, German and Spanish.”
Reasoning
“Mixtral matches or outperforms Llama 2 70B, as well as GPT3.5, on most benchmarks.”
Speed / Latency
“Mixtral outperforms Llama 2 70B on most benchmarks with 6x faster inference.”
General
“Phi-3-Small-8K-Instruct is a 7B parameters, lightweight, state-of-the-art open model”
Reasoning
“Phi-3-Small-8K-Instruct showcased a robust and state-of-the-art performance among models of the same-size and next-size-up”
phi-3-small 7.4B leads 5–2 across 7 shared benchmarks. Mixtral 8x7B leads 2 benchmarks and phi-3-small 7.4B leads 5. Higher isn't always better — see the integrity caveats on each benchmark.
Mixtral 8x7B and phi-3-small 7.4B have 7 benchmarks in common in our data — those are the only rows where a direct, apples-to-apples comparison is drawn.
phi-3-small 7.4B leads reasoning 2–0.