Epoch AI Chess Puzzles (ECI internal benchmark)
Integrity rank #2 of 61 · 81 models scored · top score 62.12 · GPT-5.5 Pro
Spatial reasoning and planning via best-move selection
Strongest on discrimination, weakest on saturation headroom.
Reported scores — protocols may differ. How we compare
| # | Model | Score | Evidence | Measured |
|---|---|---|---|---|
| 1 | GPT-5.5 Pro | 62.12 | reproducedT1 | 2026-04-23 |
| 2 | GPT-5.6 Sol | 62.12 | reproducedT1 | 2026-07-09 |
| 3 | GPT-5.4 Pro | 56.44 | reproducedT1 | 2026-03-05 |
| 4 | Gemini 3.1 Pro | 52.65 | reproducedT1 | 2026-02-19 |
| 5 | GPT-5.5 | 51.6 | reproducedT1 | 2026-04-23 |
| 6 | GPT-5.6 Terra | 51.6 | reproducedT1 | 2026-07-09 |
| 7 | Gemini 3.5 Flash | 47.39 | reproducedT1 | 2026-05-19 |
| 8 | GPT-5.2 | 46.34 | reproducedT1 | 2025-12-11 |
| 9 | Gemini 3.7 Flash | 44.23 | reproducedT1 | 2026-08-13 |
| 10 | GPT-5.4 | 41.08 | reproducedT1 | 2026-03-05 |
| 11 | Gemini 3.6 Flash | 40.03 | reproducedT1 | 2026-07-21 |
| 12 | Claude Opus 5 | 38.97 | reproducedT1 | 2026-07-24 |
| 13 | Claude Fable 5 | 37.92 | reproducedT1 | 2026-06-09 |
| 14 | Gemini 3 Flash | 36.87 | reproducedT1 | 2025-12-17 |
| 15 | Grok 4.6 | 36.87 | reproducedT1 | 2026-08-12 |
| 16 | GPT-5.6 Luna | 36.87 | reproducedT1 | 2026-07-09 |
| 17 | Kimi K3 | 35.82 | reproducedT1 | 2026-07-16 |
| 18 | o3 | 34.76 | reproducedT1 | 2024-12-20 |
| 19 | GPT-5 | 33.71 | reproducedT1 | 2025-08-07 |
| 20 | Grok 4.5 | 32.66 | reproducedT1 | 2026-07-08 |
| 21 | Claude Sonnet 5 | 31.61 | reproducedT1 | 2026-06-30 |
| 22 | Claude Opus 4.8 | 30.56 | reproducedT1 | 2026-05-28 |
| 23 | DeepSeek V4 Flash 0731 | 29.5 | reproducedT1 | 2026-07-31 |
| 24 | GPT-5.1 | 28.45 | reproducedT1 | 2025-11-13 |
| 25 | Gemini 3 Pro | 27.4 | reproducedT1 | 2025-11-18 |
| 26 | Claude Opus 4.7 | 26.35 | reproducedT1 | 2026-04-16 |
| 27 | GPT-5 mini | 26.35 | reproducedT1 | 2025-08-07 |
| 28 | GPT-5.4 Nano | 26.35 | reproducedT1 | 2026-03-17 |
| 29 | Qwen 3.8 Max | 25.29 | reproducedT1 | 2026-07-19 |
| 30 | Grok 4 | 24.24 | reproducedT1 | 2025-07-09 |
| 31 | GPT-5 nano | 23.19 | reproducedT1 | 2025-08-07 |
| 32 | Kimi K2.6 | 22.14 | reproducedT1 | 2026-04-20 |
| 33 | Qwen 3.6 35B-A3B | 22.14 | reproducedT1 | 2026-04-14 |
| 34 | o4-mini | 22.14 | reproducedT1 | 2025-04-16 |
| 35 | Gemini 3.1 Flash-Lite | 21.09 | reproducedT1 | 2026-03-03 |
| 36 | Grok 4.3 Beta | 21.09 | reproducedT1 | 2026-04-17 |
| 37 | Grok 4.20 | 20.03 | reproducedT1 | 2026-02-17 |
| 38 | Qwen3.7-Plus | 20.03 | reproducedT1 | 2026-06-02 |
| 39 | GPT-5.4 Mini | 20.03 | reproducedT1 | 2026-03-17 |
| 40 | Gemini 3.5 Flash-Lite | 17.93 | reproducedT1 | 2026-07-21 |
| 41 | Qwen 3.5 Plus (hosted 397B-A17B) | 17.93 | reproducedT1 | 2026-02-16 |
| 42 | GLM-5.2 | 16.88 | reproducedT1 | 2026-06-16 |
| 43 | Inkling | 16.88 | reproducedT1 | 2026-07-15 |
| 44 | Kimi K2.7 Code | 16.88 | reproducedT1 | 2026-06-12 |
| 45 | Qwen 3.5 Flash (hosted 35B-A3B) | 16.88 | reproducedT1 | 2026-02-25 |
| 46 | gpt-oss-120b | 15.82 | reproducedT1 | 2025-08-05 |
| 47 | DeepSeek-V4-Pro | 15.82 | reproducedT1 | 2026-04-24 |
| 48 | Gemini 2.5 Pro (Jun 2025) | 15.82 | reproducedT1 | 2025-06-05 |
| 49 | Kimi K2 Thinking | 15.82 | reproducedT1 | 2025-11-06 |
| 50 | Qwen 3.6 Flash | 15.82 | reproducedT1 | 2026-04-26 |
| 51 | Qwen 3.6 Max (Preview) | 15.82 | reproducedT1 | 2026-04-20 |
| 52 | Qwen3.7-Max | 14.77 | reproducedT1 | 2026-05-19 |
| 53 | GLM-5.1 | 14.77 | reproducedT1 | 2026-04-07 |
| 54 | Inkling-Small | 13.72 | reproducedT1 | 2026-07-30 |
| 55 | Qwen 3.6 Plus | 12.67 | reproducedT1 | 2026-04-01 |
| 56 | Claude Opus 4.6 | 12.67 | reproducedT1 | 2026-02-05 |
| 57 | o3-mini | 12.67 | reproducedT1 | 2025-01-31 |
| 58 | o1 | 10.56 | reproducedT1 | 2024-12-05 |
| 59 | DeepSeek-V3.2 | 9.51 | reproducedT1 | 2025-12-01 |
| 60 | MiniMax-M3 | 9.51 | reproducedT1 | 2026-06-01 |
| 61 | Claude Sonnet 4.6 | 8.46 | reproducedT1 | 2026-02-17 |
| 62 | GPT-4o (Aug 2024) | 8.46 | reproducedT1 | 2024-05-13 |
| 63 | Kimi K2.5 | 7.41 | reproducedT1 | 2026-02-02 |
| 64 | Claude Opus 4.5 | 7.41 | reproducedT1 | 2025-11-24 |
| 65 | Qwen3-235B-A22B-Thinking (Jul 2025) | 7.41 | reproducedT1 | 2025-07-25 |
| 66 | Claude Sonnet 4.5 | 7.41 | reproducedT1 | 2025-09-29 |
| 67 | GPT-5.5 Instant | 7.41 | reproducedT1 | 2026-05-05 |
| 68 | GLM-5 | 5.3 | reproducedT1 | 2026-02-11 |
| 69 | Claude Haiku 4.5 | 3.2 | reproducedT1 | 2025-10-15 |
| 70 | Claude Opus 4.1 | 2.15 | reproducedT1 | 2025-08-05 |
| 71 | GPT-4.1 mini | 2.15 | reproducedT1 | 2025-04-14 |
| 72 | GLM-4.7 | 1.09 | reproducedT1 | 2025-12-22 |
| 73 | GPT-4 Turbo (Apr 2024) | 1.09 | reproducedT1 | 2024-04-09 |
| 74 | GPT-4.1 | 1.09 | reproducedT1 | 2025-04-14 |
| 75 | Claude 3 Opus | 0.04 | reproducedT1 | 2024-03-04 |
| 76 | Gemma 4 31B IT | 0.04 | reproducedT1 | 2026-04-02 |
| 77 | Gemma 3 27B | 0 | reproducedT1 | 2025-03-12 |
| 78 | Qwen3-Max | 0 | reproducedT1 | 2025-09-05 |
| 79 | GPT-3.5 Turbo (Jan 2024) | 0 | reproducedT1 | 2023-06-13 |
| 80 | GPT-4 (Jun 2023) | 0 | reproducedT1 | 2023-06-13 |
| 81 | GPT-4o mini | 0 | reproducedT1 | 2024-07-18 |
Continue from this leaderboard into the tightest head-to-head reads.
Weighted composite. Each dimension opens to the source-backed sub-signals it is computed from.
Claims drawn from cited facts, not live model generation.
This benchmark scores 81 models, with a spread of 62.12 points between the best and worst results. This wide spread supports real separation between models, so rank differences likely reflect genuine capability gaps rather than noise.
3 cited facts
The benchmark carries an integrity score of 100, an A integrity grade, and an integrity rank of 2 among 61 benchmarks. The weakest component in the integrity breakdown is saturation, which means the ceiling is crowded for high-performing models. This score reflects the benchmark's health as a discriminator of frontier models under a disclosed harness.
6 cited facts
This benchmark is not saturated: the top score is 62.12, leaving 37.88 points of headroom to the ceiling. Two models cluster near the top, within a couple of points, so there is still genuine room to separate the field at the top rather than treating small differences as meaningful.
4 cited facts
Scores were produced under a consistent harness, so task performance is directly comparable across runs of this benchmark. Because the test set is held-out and scores are not posted to a public leaderboard, high scores warrant less contamination scrutiny than public-set results, though exact-string matching means format errors are scored as wrong. The benchmark is contamination-resistant by design: positions were generated programmatically and do not appear in other sources, with no known leakage as of its release.
4 cited facts
This benchmark measures spatial reasoning and planning through best-move selection on novel chess positions. It was built by generating novel positions from random opening play followed by engine-rated moves, keeping only positions where the engine identifies a single clear best move. Its core assumption is that matching the engine's best move on unfamiliar positions proxies spatial reasoning and planning while avoiding memorization of known puzzles. The sharpest caveat is that the item count is small, giving high variance; 'best move' is engine-defined, so a correct reasoner can be penalized by parsing errors, and no human baseline is documented.
5 cited facts
Chess Puzzles (Epoch AI Chess Puzzles (ECI internal benchmark)): An Epoch AI internal benchmark of 100 programmatically generated chess positions (given as FEN); the model must output the single Stockfish-best move, scored as exact-match accuracy. Feeds quick Epoch Capabilities Index (ECI) estimates; the top frontier score was under 40% at its 2025-12 release.
GPT-5.5 Pro leads Chess Puzzles at 62.12 — independently reproduced. The full leaderboard above lists every recorded measurement, not just the headline number.
81 models have recorded scores on Chess Puzzles, spanning a score spread of 62.12.
tensor.news grades Chess Puzzles A for integrity (score 100/100), ranking #2 of 61 benchmarks we assess across discrimination, saturation, contamination resistance, harness comparability, and freshness.
No — Chess Puzzles still has headroom and continues to discriminate between models rather than bunching them at the ceiling.
Its test set is held-out, which lowers — but does not eliminate — the risk that training data leaked into the questions.
Scores are reported under a consistent harness, so comparisons on Chess Puzzles are reasonably apples-to-apples.
Every Chess Puzzles measurement is source-backed and tagged reproduced, self-reported, or unverified; the underlying record cites epoch.ai/benchmarks/chess-puzzles.