OTIS Mock AIME 2024-2025
Integrity rank #38 of 65 · 180 models scored · top score 100 · GPT-6 Sol
Competition-math reasoning on novel (un-leaked) AIME-style integer-answer problems crafted by elite olympiad students.
Strongest on discrimination, weakest on contamination resistance. Caveat: the public test set does not rule out training-data contamination.
Reported scores — protocols may differ. How we compare
| # | Model | Score | Evidence | Measured |
|---|---|---|---|---|
| 1 | GPT-6 Sol | 100 | unverifiedT2 | 2026-09-22 |
| 2 | GPT-6.1 Sol | 100 | unverifiedT2 | 2026-09-29 |
| 3 | Gemini 4 Argon | 100 | unverifiedT2 | 2026-09-30 |
| 4 | Claude Fable 5 | 100 | reproduced· optimizedT1 | 2026-06-09 |
| 5 | Claude Fable 5.1 | 100 | unverifiedT2 | 2026-09-01 |
| 6 | Qwen3.8 Max (0902) | 100 | unverifiedT2 | 2026-09-01 |
| 7 | Claude Opus 5.5 | 100 | unverifiedT2 | 2026-09-22 |
| 8 | Claude Sonnet 5.5 | 100 | unverifiedT2 | 2026-09-28 |
| 9 | GPT-5.5 | 100 | reproduced· optimizedT1 | 2026-04-23 |
| 10 | GPT-5.5 Pro | 100 | reproduced· optimizedT1 | 2026-04-23 |
| 11 | GPT-5.6 Sol | 100 | reproduced· optimizedT1 | 2026-07-09 |
| 12 | GPT-6 Astra | 100 | unverifiedT2 | 2026-09-03 |
| 13 | GPT-5.6 Terra | 99.72 | reproduced· optimizedT1 | 2026-07-09 |
| 14 | Qwen 3.8 Max | 99.44 | reproduced· optimizedT1 | 2026-08-02 |
| 15 | Grok 4.6 | 99.17 | reproduced· optimizedT1 | 2026-08-12 |
| 16 | Muse Spark 1.3 | 99.17 | unverifiedT2 | 2026-09-02 |
| 17 | GPT-6 Luna | 98.89 | unverifiedT2 | 2026-09-22 |
| 18 | Gemini 3.8 Flash | 98.89 | unverifiedT2 | 2026-09-02 |
| 19 | Claude Haiku 5.5 | 98.89 | unverifiedT2 | 2026-10-07 |
| 20 | Claude Opus 5 | 98.89 | reproduced· optimizedT1 | 2026-07-24 |
| 21 | DeepSeek V4 Pro 0813 | 98.61 | reproduced· optimizedT1 | 2026-08-13 |
| 22 | Mistral Large 4 | 98.33 | unverifiedT2 | 2026-10-06 |
| 23 | Claude Opus 4.8 | 98.33 | reproduced· optimizedT1 | 2026-05-28 |
| 24 | DeepSeek V4.1 Flash | 98.33 | unverifiedT2 | 2026-09-09 |
| 25 | GPT-5.6 Luna | 98.33 | reproduced· optimizedT1 | 2026-07-09 |
| 26 | Grok 4.7 | 98.05 | unverifiedT2 | 2026-09-21 |
| 27 | Claude Opus 4.7 | 97.8 | reproduced· optimizedT1 | 2026-04-16 |
| 28 | Grok 4.5 | 97.78 | reproduced· optimizedT1 | 2026-07-08 |
| 29 | GPT-5.4 | 97.78 | reproduced· optimizedT1 | 2026-03-05 |
| 30 | Gemini 3.7 Flash | 97.22 | reproduced· optimizedT1 | 2026-08-13 |
| 31 | Kimi K3 | 97.22 | reproduced· optimizedT1 | 2026-07-16 |
| 32 | DeepSeek-V4-Pro | 96.66 | reproduced· optimizedT1 | 2026-04-24 |
| 33 | Kimi K2.6 | 96.11 | reproduced· optimizedT1 | 2026-04-20 |
| 34 | GPT-5.2 | 96.11 | reproduced· optimizedT1 | 2025-12-11 |
| 35 | Gemini 3.1 Pro | 95.6 | reproduced· optimizedT1 | 2026-02-19 |
| 36 | Gemini 3 Flash | 95.55 | reproduced· optimizedT1 | 2025-12-17 |
| 37 | Gemini 3.5 Flash | 95.55 | reproduced· optimizedT1 | 2026-05-19 |
| 38 | Kimi K2.7 Code | 95.55 | reproduced· optimizedT1 | 2026-06-12 |
| 39 | Qwen3.7-Max | 95.55 | reproduced· optimizedT1 | 2026-05-19 |
| 40 | Claude Sonnet 5 | 94.72 | reproduced· optimizedT1 | 2026-06-30 |
| 41 | Claude Opus 4.6 | 94.44 | reproduced· optimizedT1 | 2026-02-05 |
| 42 | DeepSeek V4 Flash 0731 | 94.44 | reproduced· optimizedT1 | 2026-07-31 |
| 43 | Gemini 3.6 Flash | 94.16 | reproduced· optimizedT1 | 2026-07-21 |
| 44 | GLM-5.3-Flash | 93.88 | unverifiedT2 | 2026-08-20 |
| 45 | Grok 4.3 Beta | 93.33 | reproduced· optimizedT1 | 2026-04-17 |
| 46 | Qwen 3.6 Plus | 93.33 | reproduced· optimizedT1 | 2026-03-31 |
| 47 | Qwen3.7-Plus | 93.33 | reproduced· optimizedT1 | 2026-06-02 |
| 48 | GLM-5.1 | 93.33 | reproduced· optimizedT1 | 2026-04-07 |
| 49 | Grok 4.20 | 92.21 | reproduced· optimizedT1 | 2026-02-17 |
| 50 | Kimi K2.5 | 92.19 | reproduced· optimizedT1 | 2026-01-27 |
| 51 | Gemini 3 Pro | 91.38 | reproduced· optimizedT1 | 2025-11-18 |
| 52 | GPT-5 | 91.38 | reproduced· optimizedT1 | 2025-08-07 |
| 53 | Qwen 3.6 Max (Preview) | 91.1 | reproduced· optimizedT1 | 2026-04-20 |
| 54 | Qwen3.6 27B | 91.1 | unverifiedT2 | 2026-04-22 |
| 55 | GLM-5.3 | 91.1 | unverifiedT2 | 2026-08-14 |
| 56 | Inkling-Small | 89.99 | reproduced· optimizedT1 | 2026-07-15 |
| 57 | Muse Spark | 88.89 | reproduced· optimizedT1 | 2026-04-08 |
| 58 | gpt-oss-120b | 88.88 | reproduced· optimizedT1 | 2025-08-05 |
| 59 | Inkling | 88.88 | reproduced· optimizedT1 | 2026-07-15 |
| 60 | Qwen3.5 397B-A17B | 88.88 | unverifiedT2 | 2026-02-13 |
| 61 | GPT-5.4 Mini | 88.88 | reproduced· optimizedT1 | 2026-03-17 |
| 62 | GPT-5.1 | 88.6 | reproduced· optimizedT1 | 2025-11-13 |
| 63 | DeepSeek-V3.2 | 87.81 | reproduced· optimizedT1 | 2025-12-01 |
| 64 | GPT-5.4 Nano | 87.77 | reproduced· optimizedT1 | 2026-03-17 |
| 65 | Nemotron 3 Ultra | 86.65 | unverifiedT2 | 2026-06-04 |
| 66 | Qwen 3.5 Plus (hosted 397B-A17B) | 86.65 | reproduced· optimizedT1 | 2026-02-16 |
| 67 | Qwen 3.6 35B-A3B | 86.65 | reproduced· optimizedT1 | 2026-04-14 |
| 68 | Qwen3-235B-A22B-Thinking (Jul 2025) | 86.65 | reproduced· optimizedT1 | 2025-07-25 |
| 69 | Qwen3.7 Flash | 86.65 | unverifiedT2 | 2026-07-27 |
| 70 | GPT-5 mini | 86.65 | reproduced· optimizedT1 | 2025-08-07 |
| 71 | GLM-5.2 | 86.38 | reproduced· optimizedT1 | 2026-06-16 |
| 72 | Claude Opus 4.5 | 86.1 | reproduced· optimizedT1 | 2025-11-24 |
| 73 | Claude Sonnet 4.6 | 85.78 | reproduced· optimizedT1 | 2026-02-17 |
| 74 | Gemini 2.5 Pro (Jun 2025) | 84.71 | reproduced· optimizedT1 | 2025-06-05 |
| 75 | Qwen 3.5 Flash (hosted 35B-A3B) | 84.43 | reproduced· optimizedT1 | 2026-02-25 |
| 76 | Qwen 3.6 Flash | 84.43 | reproduced· optimizedT1 | 2026-04-27 |
| 77 | o3 | 84.43 | reproduced· optimizedT1 | 2025-04-16 |
| 78 | Grok 4 | 83.98 | reproduced· optimizedT1 | 2025-07-09 |
| 79 | GLM-4.7 | 83.32 | reproduced· optimizedT1 | 2025-12-22 |
| 80 | Kimi K2 Thinking | 83.04 | reproduced· optimizedT1 | 2025-11-06 |
| 81 | Gemma 4 26B A4B | 82.2 | unverifiedT2 | 2026-04-02 |
| 82 | o4-mini | 81.65 | reproduced· optimizedT1 | 2025-04-16 |
| 83 | GPT-5 nano | 81.09 | reproduced· optimizedT1 | 2025-08-07 |
| 84 | GLM-5 | 79.98 | reproduced· optimizedT1 | 2026-02-11 |
| 85 | Gemini 3.1 Flash-Lite | 79.98 | reproduced· optimizedT1 | 2026-03-03 |
| 86 | Grok-3 mini | 77.76 | reproduced· optimizedT1 | 2025-04-09 |
| 87 | Claude Sonnet 4.5 | 77.76 | reproduced· optimizedT1 | 2025-09-29 |
| 88 | o3-mini | 76.92 | reproduced· optimizedT1 | 2025-01-31 |
| 89 | Gemma 4 31B IT | 73.31 | reproduced· optimizedT1 | 2026-04-02 |
| 90 | Qwen3-Max | 73.31 | reproduced· optimizedT1 | 2025-09-24 |
| 91 | o1 | 73.31 | reproduced· optimizedT1 | 2024-12-17 |
| 92 | Gemini 2.5 Flash (Apr 2025) | 73.03 | reproduced· optimizedT1 | 2025-04-17 |
| 93 | Gemini 3.5 Flash-Lite | 71.08 | reproduced· optimizedT1 | 2026-07-21 |
| 94 | MiniMax-M3 | 71.08 | unverifiedT2 | 2026-06-01 |
| 95 | Claude Sonnet 4 | 71.08 | reproduced· optimizedT1 | 2025-05-22 |
| 96 | Gemini 2.5 Flash (May 2025) | 70.8 | reproduced· optimizedT1 | 2025-05-20 |
| 97 | Qwen3-30B-A3B-Thinking (Jul 2025) | 70.25 | unverifiedT2 | 2025-07-30 |
| 98 | Qwen3.5-35B-A3B | 69.97 | unverifiedT2 | 2026-02-24 |
| 99 | Claude Opus 4.1 | 68.86 | reproduced· optimizedT1 | 2025-08-05 |
| 100 | GPT-5.5 Instant | 68.02 | reproduced· optimizedT1 | 2026-05-05 |
| 101 | Qwen3-32B | 66.91 | unverifiedT2 | 2025-04-29 |
| 102 | Claude Haiku 4.5 | 66.63 | reproduced· optimizedT1 | 2025-10-15 |
| 103 | Qwen3-14B | 66.36 | unverifiedT2 | 2025-04-29 |
| 104 | DeepSeek-R1 (May 2025) | 66.36 | reproduced· optimizedT1 | 2025-05-28 |
| 105 | gpt-oss-20b | 65.24 | unverifiedT2 | 2025-08-05 |
| 106 | Claude Opus 4 | 64.41 | reproduced· optimizedT1 | 2025-05-22 |
| 107 | Qwen3-30B-A3B | 62.74 | unverifiedT2 | 2025-04-29 |
| 108 | Qwen3-30B-A3B-Instruct (Jul 2025) | 62.18 | unverifiedT2 | 2025-07-29 |
| 109 | Qwen3.5-9B | 61.63 | unverifiedT2 | 2026-02-24 |
| 110 | QwQ-32B | 59.13 | unverifiedT2 | 2025-03-05 |
| 111 | Gemini 2.0 Flash Thinking (Jan 2025) | 57.74 | reproduced· optimizedT1 | 2025-01-21 |
| 112 | Claude 3.7 Sonnet | 57.74 | reproduced· optimizedT1 | 2025-02-24 |
| 113 | Qwen3-8B | 56.07 | unverifiedT2 | 2025-04-28 |
| 114 | Grok 3 | 55.51 | reproduced· optimizedT1 | 2025-04-09 |
| 115 | DeepSeek-R1-Distill-Qwen-32B | 55.51 | unverifiedT2 | 2025-01-20 |
| 116 | DeepSeek-R1 | 53.29 | reproduced· optimizedT1 | 2025-01-20 |
| 117 | DeepSeek-R1-Distill-Qwen-14B | 50.51 | unverifiedT2 | 2025-01-20 |
| 118 | o1-mini | 46.89 | reproduced· optimizedT1 | 2024-09-12 |
| 119 | GPT-4.1 mini | 44.67 | reproduced· optimizedT1 | 2025-04-14 |
| 120 | GPT-4.1 | 38.27 | reproduced· optimizedT1 | 2025-04-14 |
| 121 | DeepSeek-V3 (Mar 2025) | 37.72 | reproduced· optimizedT1 | 2025-03-24 |
| 122 | GPT-4.5 | 37.72 | reproduced· optimizedT1 | 2025-02-27 |
| 123 | Mistral Medium 3 | 32.15 | reproduced· optimizedT1 | 2025-05-07 |
| 124 | Gemini 2.0 Flash (Feb 2025) | 31.04 | reproduced· optimizedT1 | 2025-02-05 |
| 125 | o1-preview | 31.04 | reproduced· optimizedT1 | 2024-09-12 |
| 126 | Mistral Small 3.2 | 30.21 | unverifiedT2 | 2025-06-20 |
| 127 | Magistral Small 1.0 | 29.93 | reproduced· optimizedT1 | 2025-06-10 |
| 128 | GPT-4.1 nano | 28.82 | reproduced· optimizedT1 | 2025-04-14 |
| 129 | Magistral Small 1.2 | 27.98 | unverifiedT2 | 2025-09-18 |
| 130 | Gemini 1.5 Pro (Sept 2024) | 22.98 | reproduced· optimizedT1 | 2024-09-24 |
| 131 | Gemma 3 27B | 22.42 | unverifiedT2 | 2025-03-12 |
| 132 | Llama 4 Maverick | 20.48 | reproduced· optimizedT1 | 2025-04-05 |
| 133 | Gemma 3 12B | 16.58 | unverifiedT2 | 2025-03-12 |
| 134 | Gemini 1.5 Flash (Sep 2024) | 16.17 | reproduced· optimizedT1 | 2024-09-24 |
| 135 | Qwen2.5-Max | 16.03 | reproduced· optimizedT1 | 2025-01-25 |
| 136 | DeepSeek-V3 | 15.75 | reproduced· optimizedT1 | 2024-12-26 |
| 137 | Phi-4 | 13.66 | reproduced· optimizedT1 | 2024-12-12 |
| 138 | Grok-2 (Dec 2024) | 11.44 | reproduced· optimizedT1 | 2024-12-12 |
| 139 | Llama 3.1-405B | 9.63 | reproduced· optimizedT1 | 2024-07-23 |
| 140 | Claude 3.5 Sonnet (October 2024) | 8.38 | reproduced· optimizedT1 | 2024-10-22 |
| 141 | Mistral Large 2 (Jul 2024) | 8.38 | reproduced· optimizedT1 | 2024-07-24 |
| 142 | Qwen2.5-72B | 7.96 | reproduced· optimizedT1 | 2024-09-19 |
| 143 | Llama 4 Scout | 7.69 | reproduced· optimizedT1 | 2025-04-05 |
| 144 | Mistral Large 2 (Nov 2024) | 7.69 | reproduced· optimizedT1 | 2024-11-18 |
| 145 | Gemma 3 4B | 7.41 | unverifiedT2 | 2025-03-12 |
| 146 | Qwen2.5-32B | 7.27 | unverifiedT2 | 2024-09-17 |
| 147 | GPT-4o mini | 6.85 | reproduced· optimizedT1 | 2024-07-18 |
| 148 | Gemini 1.5 Pro (May 2024) | 6.71 | reproduced· optimizedT1 | 2024-05-14 |
| 149 | Mistral Small 3 | 6.57 | unverifiedT2 | 2025-01-30 |
| 150 | GPT-4 Turbo (Apr 2024) | 6.57 | reproduced· optimizedT1 | 2024-04-09 |
| 151 | Claude 3.5 Sonnet | 6.43 | reproduced· optimizedT1 | 2024-06-20 |
| 152 | GPT-4o (Aug 2024) | 6.3 | reproduced· optimizedT1 | 2024-08-06 |
| 153 | GPT-4o (May 2024) | 6.16 | reproduced· optimizedT1 | 2024-05-13 |
| 154 | GPT-4o (Nov 2024) | 6.16 | reproduced· optimizedT1 | 2024-11-20 |
| 155 | Mistral Small 3.1 | 5.74 | unverifiedT2 | 2025-03-17 |
| 156 | Llama 3.3 70B | 5.04 | reproduced· optimizedT1 | 2024-12-06 |
| 157 | Claude 3 Opus | 4.63 | reproduced· optimizedT1 | 2024-02-29 |
| 158 | Claude 3.5 Haiku | 4.21 | reproduced· optimizedT1 | 2024-10-22 |
| 159 | Llama 3-70B | 4.21 | reproduced· optimizedT1 | 2024-04-18 |
| 160 | Gemini 1.5 Flash (May 2024) | 3.79 | reproduced· optimizedT1 | 2024-05-23 |
| 161 | Llama 3.1-70B | 3.51 | reproduced· optimizedT1 | 2024-07-23 |
| 162 | Llama 3.2 90B | 2.54 | reproduced· optimizedT1 | 2024-09-24 |
| 163 | Claude 3 Sonnet | 2.4 | reproduced· optimizedT1 | 2024-02-29 |
| 164 | Qwen2.5-7B | 2.4 | unverifiedT2 | 2024-09-19 |
| 165 | Claude 2 | 2.4 | reproduced· optimizedT1 | 2023-07-11 |
| 166 | GPT-3.5 Turbo (Jan 2024) | 2.12 | reproduced· optimizedT1 | 2024-01-25 |
| 167 | Llama 3-8B | 1.85 | unverifiedT2 | 2024-04-18 |
| 168 | Mistral Large | 1.85 | reproduced· optimizedT1 | 2024-02-26 |
| 169 | Claude 2.1 | 1.85 | reproduced· optimizedT1 | 2023-11-21 |
| 170 | Claude 3 Haiku | 1.71 | reproduced· optimizedT1 | 2024-03-07 |
| 171 | Llama 3.1-8B | 1.57 | unverifiedT2 | 2024-07-23 |
| 172 | Gemma 2 27B | 1.29 | reproduced· optimizedT1 | 2024-06-24 |
| 173 | Gemini 1.0 Pro | 1.01 | reproduced· optimizedT1 | 2023-12-13 |
| 174 | GPT-4 (Jun 2023) | 1.01 | reproduced· optimizedT1 | 2023-06-13 |
| 175 | DeepSeek LLM 67B | 0.73 | unverifiedT2 | 2023-11-29 |
| 176 | Gemma 2 9B | 0.46 | reproduced· optimizedT1 | 2024-06-24 |
| 177 | Llama 3.2 1B | 0.46 | unverifiedT2 | 2024-09-24 |
| 178 | GPT-4 (Mar 2023) | 0.46 | reproduced· optimizedT1 | 2023-03-14 |
| 179 | Mistral 7B v0.3 | 0.18 | unverifiedT2 | 2024-05-27 |
| 180 | Llama 2-70B | 0 | reproduced· optimizedT1 | 2023-07-18 |
Continue from this leaderboard into the tightest head-to-head reads.
Weighted composite. Each dimension opens to the source-backed sub-signals it is computed from.
Claims drawn from cited facts, not live model generation.
This benchmark tests competition mathematics using novel integer-answer problems written by olympiad students.
1 cited fact
This benchmark has scores from 180 tracked models, with a recorded score range spanning 100.0.
2 cited facts
Its Benchmark Integrity Index score is 83, with grade B and rank 38 out of 65 tracked benchmarks.
4 cited facts
It is flagged as saturated. The top recorded score is 100.0, with 0.0 points of headroom. The computed top-score cluster contains 26 models.
4 cited facts
The harness classification is consistent harness. Test-set privacy is recorded as unknown. The recorded contamination-window ratio is 0.75.
3 cited facts
OTIS Mock AIME 2024-2025: OTIS Mock AIME 2024-2025 as reported in Epoch AI's Capabilities Index CSV.
GPT-6 Sol leads OTIS Mock AIME 2024-2025 at 100. The full leaderboard above lists every recorded measurement, not just the headline number.
180 models have recorded scores on OTIS Mock AIME 2024-2025, spanning a score spread of 100.
tensor.news grades OTIS Mock AIME 2024-2025 B for integrity (score 83/100), ranking #38 of 65 benchmarks we assess across discrimination, saturation, contamination resistance, harness comparability, and freshness.
Yes — top scores are clustering near the ceiling (saturation ratio 0.14), so OTIS Mock AIME 2024-2025 no longer separates leading models well. Treat small gaps at the top with caution.
Its test set is unknown, so contamination risk is on the table: a high score may partly reflect training-data overlap rather than capability.
Scores are reported under a consistent harness, so comparisons on OTIS Mock AIME 2024-2025 are reasonably apples-to-apples.
Every OTIS Mock AIME 2024-2025 measurement is source-backed and tagged reproduced, self-reported, or unverified; the underlying record cites epoch.ai/benchmarks/otis-mock-aime-2024-2025.