Claude 3.5 Sonnet (October 2024) vs GPT-4o (Nov 2024)

Verdict

Claude 3.5 Sonnet (October 2024) leads 9–4 across 14 shared benchmarks.

Claude 3.5 Sonnet (October 2024) 9 · GPT-4o (Nov 2024) 4 · 1 tied · higher isn't always better — check the caveats below.

Per-benchmark head-to-head

Reported scores — protocols may differ. How we compare

Sort

Claude 3.5 Sonnet (October 2024)GPT-4o (Nov 2024)

51.6
AAider polyglot
18.2
3.63
AFrontierMath-2025-02-28-Private
0.6
62
AGeoBench
71
40.4
BGPQA diamond
30.51
4.6
AGSO-Bench
0
0
AHLE
0
80.3
BLech Mazur Writing
81.8
56.95
BMATH level 5
49.77
52.7
AMETR Time Horizons
40.76
83.07
CMMLU
84.13
8.38
AOTIS Mock AIME 2024-2025
6.16
24
AThe Agent Company
8.6
0
AVPCT
10
39.97
AWeirdML
25.12

Capabilities

Math

Math: Claude 3.5 Sonnet (October 2024) leads 3–0 across 3 shared math benchmarks (largest gap: MATH level 5, 56.95 vs 49.77 — single evaluator).

6 reproduced

Reasoning

Reasoning: Claude 3.5 Sonnet (October 2024) leads 1–0 across 2 shared reasoning benchmarks (largest gap: GPQA diamond, 40.4 vs 30.51 — single evaluator).

2 reproduced2 unverified

A higher score does not always mean a better model. Each number is task performance under a disclosed harness. Of the 28 measurements across 14 shared benchmarks, 28 single-evaluator or undisclosed-protocol reports; 29% are independently reproduced. Check any benchmark's integrity grade before reading a win as decisive.

Frequently asked questions

Claude 3.5 Sonnet (October 2024) leads 9–4 across 14 shared benchmarks. Claude 3.5 Sonnet (October 2024) leads 9 benchmarks and GPT-4o (Nov 2024) leads 4, with 1 tied. Higher isn't always better — see the integrity caveats on each benchmark.

Claude 3.5 Sonnet (October 2024) and GPT-4o (Nov 2024) have 14 benchmarks in common in our data — those are the only rows where a direct, apples-to-apples comparison is drawn.

Claude 3.5 Sonnet (October 2024) leads math 3–0. Claude 3.5 Sonnet (October 2024) leads reasoning 1–0.

Follow the record

These models

Same tier