Epoch AI Chess Puzzles (ECI internal benchmark)
Integrity rank #1 of 65 · 135 models scored · top score 72.64 · GPT-6 Astra
Spatial reasoning and planning via best-move selection
Strongest on discrimination, weakest on saturation headroom.
Reported scores — protocols may differ. How we compare
| # | Model | Score | Evidence | Measured |
|---|---|---|---|---|
| 1 | GPT-6 Astra | 72.64 | unverifiedT2 | 2026-09-03 |
| 2 | GPT-6.1 Sol | 66.33 | unverifiedT2 | 2026-09-29 |
| 3 | GPT-5.6 Sol | 66.33 | unverifiedT2 | 2026-07-09 |
| 4 | Claude Opus 5.5 | 65.28 | unverifiedT2 | 2026-09-22 |
| 5 | GPT-5.5 Pro | 62.12 | unverifiedT2 | 2026-04-23 |
| 6 | GPT-5.5 | 61.07 | unverifiedT2 | 2026-04-23 |
| 7 | GPT-5.2 Pro | 57.91 | unverifiedT2 | 2025-12-11 |
| 8 | Gemini 4 Argon | 56.86 | unverifiedT2 | 2026-09-30 |
| 9 | Claude Sonnet 5.5 | 56.86 | unverifiedT2 | 2026-09-28 |
| 10 | GPT-5.4 Pro | 56.86 | unverifiedT2 | 2026-03-05 |
| 11 | Gemini 3.8 Flash | 55.81 | unverifiedT2 | 2026-09-02 |
| 12 | GPT-6 Sol | 53.7 | unverifiedT2 | 2026-09-22 |
| 13 | Gemini 3.1 Pro | 52.65 | reproducedT1 | 2026-02-19 |
| 14 | Claude Haiku 5.5 | 52.65 | unverifiedT2 | 2026-10-07 |
| 15 | Claude Opus 4.8 | 49.49 | unverifiedT2 | 2026-05-28 |
| 16 | Claude Fable 5.1 | 45.29 | unverifiedT2 | 2026-09-01 |
| 17 | DeepSeek V4.1 Flash | 45.29 | unverifiedT2 | 2026-09-09 |
| 18 | GPT-5 Pro | 45.29 | unverifiedT2 | 2025-10-07 |
| 19 | GPT-5.2 | 45.29 | unverifiedT2 | 2025-12-11 |
| 20 | GPT-5.4 | 45.29 | unverifiedT2 | 2026-03-05 |
| 21 | Gemini 3.7 Flash | 44.23 | reproducedT1 | 2026-08-13 |
| 22 | Claude Fable 5 | 44.23 | unverifiedT2 | 2026-06-09 |
| 23 | GPT-5.6 Terra | 44.23 | unverifiedT2 | 2026-07-09 |
| 24 | Qwen 3.8 Max | 43.18 | unverifiedT2 | 2026-08-02 |
| 25 | DeepSeek V4 Pro 0813 | 43.18 | unverifiedT2 | 2026-08-13 |
| 26 | Gemini 3.5 Flash | 42.13 | unverifiedT2 | 2026-05-19 |
| 27 | Claude Sonnet 5 | 42.13 | unverifiedT2 | 2026-06-30 |
| 28 | GPT-5.4 Nano | 42.13 | unverifiedT2 | 2026-03-17 |
| 29 | Claude Opus 5 | 40.03 | unverifiedT2 | 2026-07-24 |
| 30 | GPT-5.6 Luna | 40.03 | unverifiedT2 | 2026-07-09 |
| 31 | Gemini 3.6 Flash | 38.97 | unverifiedT2 | 2026-07-21 |
| 32 | GPT-5.1 | 38.97 | unverifiedT2 | 2025-11-13 |
| 33 | Grok 4.7 | 37.92 | unverifiedT2 | 2026-09-21 |
| 34 | Claude Opus 4.7 | 37.92 | unverifiedT2 | 2026-04-16 |
| 35 | Gemini 3 Flash | 35.82 | unverifiedT2 | 2025-12-17 |
| 36 | Kimi K3 | 33.71 | unverifiedT2 | 2026-07-16 |
| 37 | Muse Spark 1.3 | 33.71 | unverifiedT2 | 2026-09-02 |
| 38 | Qwen3.8 Max (0902) | 33.71 | unverifiedT2 | 2026-09-01 |
| 39 | o3 | 33.71 | unverifiedT2 | 2025-04-16 |
| 40 | GLM-5.3 | 33.71 | unverifiedT2 | 2026-08-14 |
| 41 | GPT-6 Luna | 32.66 | unverifiedT2 | 2026-09-22 |
| 42 | GPT-5 | 32.66 | unverifiedT2 | 2025-08-07 |
| 43 | Grok 4.5 | 31.61 | unverifiedT2 | 2026-07-08 |
| 44 | Grok 4.6 | 29.5 | unverifiedT2 | 2026-08-12 |
| 45 | DeepSeek V4 Flash 0731 | 28.45 | unverifiedT2 | 2026-07-31 |
| 46 | Gemini 3 Pro | 27.4 | reproducedT1 | 2025-11-18 |
| 47 | Grok 4.20 | 25.29 | unverifiedT2 | 2026-02-17 |
| 48 | Grok 4 | 24.24 | reproducedT1 | 2025-07-09 |
| 49 | Qwen 3.6 35B-A3B | 22.14 | reproducedT1 | 2026-04-14 |
| 50 | Qwen3.7-Max | 22.14 | unverifiedT2 | 2026-05-19 |
| 51 | Gemini 3.1 Flash-Lite | 21.09 | reproducedT1 | 2026-03-03 |
| 52 | Qwen3.7-Plus | 20.03 | reproducedT1 | 2026-06-02 |
| 53 | GPT-5 mini | 20.03 | unverifiedT2 | 2025-08-07 |
| 54 | Qwen3.7 Flash | 18.98 | unverifiedT2 | 2026-07-27 |
| 55 | Claude Sonnet 4.6 | 18.98 | unverifiedT2 | 2026-02-17 |
| 56 | GPT-5.4 Mini | 18.98 | unverifiedT2 | 2026-03-17 |
| 57 | GLM-5.2 | 17.93 | unverifiedT2 | 2026-06-16 |
| 58 | Qwen3.6 27B | 17.93 | unverifiedT2 | 2026-04-22 |
| 59 | o4-mini | 17.93 | unverifiedT2 | 2025-04-16 |
| 60 | GPT-5 nano | 17.93 | unverifiedT2 | 2025-08-07 |
| 61 | Gemini 3.5 Flash-Lite | 16.88 | unverifiedT2 | 2026-07-21 |
| 62 | Inkling | 16.88 | reproducedT1 | 2026-07-15 |
| 63 | Claude Opus 4.6 | 16.88 | unverifiedT2 | 2026-02-05 |
| 64 | GLM-4.7 | 16.88 | unverifiedT2 | 2025-12-22 |
| 65 | GLM-5.3-Flash | 16.88 | unverifiedT2 | 2026-08-20 |
| 66 | gpt-oss-120b | 15.82 | reproducedT1 | 2025-08-05 |
| 67 | Kimi K2.6 | 15.82 | unverifiedT2 | 2026-04-20 |
| 68 | Gemini 2.5 Pro (Jun 2025) | 15.82 | reproducedT1 | 2025-06-05 |
| 69 | Kimi K2 Thinking | 15.82 | reproducedT1 | 2025-11-06 |
| 70 | Qwen 3.6 Max (Preview) | 15.82 | reproducedT1 | 2026-04-20 |
| 71 | GLM-5.1 | 15.82 | unverifiedT2 | 2026-04-07 |
| 72 | DeepSeek-V4-Pro | 14.77 | unverifiedT2 | 2026-04-24 |
| 73 | Grok 4.3 Beta | 13.72 | unverifiedT2 | 2026-04-17 |
| 74 | Inkling-Small | 13.72 | reproducedT1 | 2026-07-15 |
| 75 | o3-mini | 10.56 | unverifiedT2 | 2025-01-31 |
| 76 | DeepSeek-V3.2 | 9.51 | reproducedT1 | 2025-12-01 |
| 77 | Kimi K2.7 Code | 9.51 | unverifiedT2 | 2026-06-12 |
| 78 | Qwen 3.5 Flash (hosted 35B-A3B) | 9.51 | unverifiedT2 | 2026-02-25 |
| 79 | Qwen 3.6 Plus | 9.51 | unverifiedT2 | 2026-03-31 |
| 80 | Claude Opus 4.5 | 9.51 | unverifiedT2 | 2025-11-24 |
| 81 | MiniMax-M3 | 8.46 | unverifiedT2 | 2026-06-01 |
| 82 | Qwen3.5 397B-A17B | 8.46 | unverifiedT2 | 2026-02-13 |
| 83 | Kimi K2.5 | 7.41 | reproducedT1 | 2026-01-27 |
| 84 | Nemotron 3 Ultra | 7.41 | unverifiedT2 | 2026-06-04 |
| 85 | Qwen 3.6 Flash | 7.41 | unverifiedT2 | 2026-04-27 |
| 86 | Qwen3-235B-A22B-Thinking (Jul 2025) | 7.41 | reproducedT1 | 2025-07-25 |
| 87 | Qwen3.5-9B | 7.41 | unverifiedT2 | 2026-02-24 |
| 88 | GPT-5.5 Instant | 7.41 | reproducedT1 | 2026-05-05 |
| 89 | GLM-5 | 5.3 | reproducedT1 | 2026-02-11 |
| 90 | Qwen3.5-35B-A3B | 5.3 | unverifiedT2 | 2026-02-24 |
| 91 | o1 | 5.3 | unverifiedT2 | 2024-12-17 |
| 92 | Claude Sonnet 4.5 | 5.3 | unverifiedT2 | 2025-09-29 |
| 93 | Claude Haiku 4.5 | 3.2 | unverifiedT2 | 2025-10-15 |
| 94 | Qwen 3.5 Plus (hosted 397B-A17B) | 3.2 | unverifiedT2 | 2026-02-16 |
| 95 | Qwen3-30B-A3B-Thinking (Jul 2025) | 3.2 | unverifiedT2 | 2025-07-30 |
| 96 | Claude Opus 4.1 | 2.15 | reproducedT1 | 2025-08-05 |
| 97 | Gemma 4 26B A4B | 1.09 | unverifiedT2 | 2026-04-02 |
| 98 | GPT-4 Turbo (Apr 2024) | 1.09 | reproducedT1 | 2024-04-09 |
| 99 | GPT-4o (Aug 2024) | 1.09 | unverifiedT2 | 2024-08-06 |
| 100 | Gemma 4 31B IT | 0.04 | reproducedT1 | 2026-04-02 |
| 101 | QwQ-32B | 0.04 | unverifiedT2 | 2025-03-05 |
| 102 | Qwen3-32B | 0.04 | unverifiedT2 | 2025-04-29 |
| 103 | Qwen3-8B | 0.04 | unverifiedT2 | 2025-04-28 |
| 104 | gpt-oss-20b | 0 | unverifiedT2 | 2025-08-05 |
| 105 | Llama 3.1-8B | 0 | unverifiedT2 | 2024-07-23 |
| 106 | Llama 2-7B | 0 | unverifiedT2 | 2023-07-18 |
| 107 | Phi-4 | 0 | unverifiedT2 | 2024-12-12 |
| 108 | Gemma 3 12B | 0 | unverifiedT2 | 2025-03-12 |
| 109 | Gemma 3 27B | 0 | reproducedT1 | 2025-03-12 |
| 110 | Gemma 3 4B | 0 | unverifiedT2 | 2025-03-12 |
| 111 | Llama 2-13B | 0 | unverifiedT2 | 2023-07-18 |
| 112 | Llama 3-8B | 0 | unverifiedT2 | 2024-04-18 |
| 113 | Llama 3.2 1B | 0 | unverifiedT2 | 2024-09-24 |
| 114 | Magistral Small 1.2 | 0 | unverifiedT2 | 2025-09-18 |
| 115 | Mistral 7B v0.3 | 0 | unverifiedT2 | 2024-05-27 |
| 116 | Mistral Small 3 | 0 | unverifiedT2 | 2025-01-30 |
| 117 | Mistral Small 3.1 | 0 | unverifiedT2 | 2025-03-17 |
| 118 | Mistral Small 3.2 | 0 | unverifiedT2 | 2025-06-20 |
| 119 | Qwen2.5-32B | 0 | unverifiedT2 | 2024-09-17 |
| 120 | Qwen2.5-7B | 0 | unverifiedT2 | 2024-09-19 |
| 121 | Qwen3-14B | 0 | unverifiedT2 | 2025-04-29 |
| 122 | Qwen3-30B-A3B | 0 | unverifiedT2 | 2025-04-29 |
| 123 | Qwen3-30B-A3B-Instruct (Jul 2025) | 0 | unverifiedT2 | 2025-07-29 |
| 124 | Qwen3-Max | 0 | reproducedT1 | 2025-09-24 |
| 125 | phi-3-mini 3.8B | 0 | unverifiedT2 | 2024-04-23 |
| 126 | DeepSeek LLM 67B | 0 | unverifiedT2 | 2023-11-29 |
| 127 | DeepSeek-R1-Distill-Qwen-14B | 0 | unverifiedT2 | 2025-01-20 |
| 128 | DeepSeek-R1-Distill-Qwen-32B | 0 | unverifiedT2 | 2025-01-20 |
| 129 | GPT-3.5 Turbo (Jan 2024) | 0 | reproducedT1 | 2024-01-25 |
| 130 | GPT-4 (Jun 2023) | 0 | reproducedT1 | 2023-06-13 |
| 131 | GPT-4.1 | 0 | unverifiedT2 | 2025-04-14 |
| 132 | GPT-4.1 mini | 0 | unverifiedT2 | 2025-04-14 |
| 133 | GPT-4.1 nano | 0 | unverifiedT2 | 2025-04-14 |
| 134 | GPT-4o mini | 0 | reproducedT1 | 2024-07-18 |
| 135 | Claude 3 Opus | 0 | unverifiedT2 | 2024-02-29 |
Continue from this leaderboard into the tightest head-to-head reads.
Weighted composite. Each dimension opens to the source-backed sub-signals it is computed from.
Claims drawn from cited facts, not live model generation.
This benchmark tests spatial reasoning and planning through best-move selection.
1 cited fact
This benchmark has scores from 135 tracked models, with a recorded score range spanning 72.64.
2 cited facts
Its Benchmark Integrity Index score is 100, with grade A and rank 1 out of 65 tracked benchmarks.
4 cited facts
It is not flagged as saturated. The top recorded score is 72.64, with 27.36 points of headroom. The computed top-score cluster contains 1 models.
4 cited facts
The harness classification is consistent harness. Test-set privacy is recorded as held-out.
2 cited facts
Chess Puzzles (Epoch AI Chess Puzzles (ECI internal benchmark)): An Epoch AI internal benchmark of 100 programmatically generated chess positions (given as FEN); the model must output the single Stockfish-best move, scored as exact-match accuracy. Feeds quick Epoch Capabilities Index (ECI) estimates; the top frontier score was under 40% at its 2025-12 release.
GPT-6 Astra leads Chess Puzzles at 72.64. The full leaderboard above lists every recorded measurement, not just the headline number.
135 models have recorded scores on Chess Puzzles, spanning a score spread of 72.64.
tensor.news grades Chess Puzzles A for integrity (score 100/100), ranking #1 of 65 benchmarks we assess across discrimination, saturation, contamination resistance, harness comparability, and freshness.
No — Chess Puzzles still has headroom and continues to discriminate between models rather than bunching them at the ceiling.
Its test set is held-out, which lowers — but does not eliminate — the risk that training data leaked into the questions.
Scores are reported under a consistent harness, so comparisons on Chess Puzzles are reasonably apples-to-apples.
Every Chess Puzzles measurement is source-backed and tagged reproduced, self-reported, or unverified; the underlying record cites epoch.ai/benchmarks/chess-puzzles.