OTIS Mock AIME 2024-2025
Integrity rank #34 of 61 · 136 models scored · top score 100 · Claude Fable 5
unknown
Strongest on discrimination, weakest on contamination resistance. Caveat: the public test set does not rule out training-data contamination.
Reported scores — protocols may differ. How we compare
| # | Model | Score | Evidence | Measured |
|---|---|---|---|---|
| 1 | Claude Fable 5 | 100 | reproduced· optimizedT1 | 2026-06-09 |
| 2 | GPT-5.5 | 100 | reproduced· optimizedT1 | 2026-04-23 |
| 3 | GPT-5.5 Pro | 100 | reproduced· optimizedT1 | 2026-04-23 |
| 4 | GPT-5.6 Sol | 100 | reproduced· optimizedT1 | 2026-07-09 |
| 5 | GPT-5.6 Terra | 99.72 | reproduced· optimizedT1 | 2026-07-09 |
| 6 | Qwen 3.8 Max | 99.44 | reproduced· optimizedT1 | 2026-07-19 |
| 7 | Grok 4.6 | 99.17 | reproduced· optimizedT1 | 2026-08-12 |
| 8 | Claude Opus 5 | 98.89 | reproduced· optimizedT1 | 2026-07-24 |
| 9 | Claude Opus 4.8 | 98.33 | reproduced· optimizedT1 | 2026-05-28 |
| 10 | GPT-5.6 Luna | 98.33 | reproduced· optimizedT1 | 2026-07-09 |
| 11 | Claude Opus 4.7 | 97.8 | reproduced· optimizedT1 | 2026-04-16 |
| 12 | Grok 4.5 | 97.78 | reproduced· optimizedT1 | 2026-07-08 |
| 13 | GPT-5.4 | 97.78 | reproduced· optimizedT1 | 2026-03-05 |
| 14 | Gemini 3.7 Flash | 97.22 | reproduced· optimizedT1 | 2026-08-13 |
| 15 | Kimi K3 | 97.22 | reproduced· optimizedT1 | 2026-07-16 |
| 16 | DeepSeek-V4-Pro | 96.66 | reproduced· optimizedT1 | 2026-04-24 |
| 17 | Kimi K2.6 | 96.11 | reproduced· optimizedT1 | 2026-04-20 |
| 18 | GPT-5.2 | 96.11 | reproduced· optimizedT1 | 2025-12-11 |
| 19 | Gemini 3.1 Pro | 95.6 | reproduced· optimizedT1 | 2026-02-19 |
| 20 | Gemini 3 Flash | 95.55 | reproduced· optimizedT1 | 2025-12-17 |
| 21 | Gemini 3.5 Flash | 95.55 | reproduced· optimizedT1 | 2026-05-19 |
| 22 | Kimi K2.7 Code | 95.55 | reproduced· optimizedT1 | 2026-06-12 |
| 23 | Qwen3.7-Max | 95.55 | reproduced· optimizedT1 | 2026-05-19 |
| 24 | Claude Sonnet 5 | 94.72 | reproduced· optimizedT1 | 2026-06-30 |
| 25 | Claude Opus 4.6 | 94.44 | reproduced· optimizedT1 | 2026-02-05 |
| 26 | DeepSeek V4 Flash 0731 | 94.44 | reproduced· optimizedT1 | 2026-07-31 |
| 27 | Gemini 3.6 Flash | 94.16 | reproduced· optimizedT1 | 2026-07-21 |
| 28 | Grok 4.3 Beta | 93.33 | reproduced· optimizedT1 | 2026-04-17 |
| 29 | Qwen 3.6 Plus | 93.33 | reproduced· optimizedT1 | 2026-04-01 |
| 30 | Qwen3.7-Plus | 93.33 | reproduced· optimizedT1 | 2026-06-02 |
| 31 | GLM-5.1 | 93.33 | reproduced· optimizedT1 | 2026-04-07 |
| 32 | Grok 4.20 | 92.21 | reproduced· optimizedT1 | 2026-02-17 |
| 33 | Kimi K2.5 | 92.19 | reproduced· optimizedT1 | 2026-02-02 |
| 34 | Gemini 3 Pro | 91.38 | reproduced· optimizedT1 | 2025-11-18 |
| 35 | GPT-5 | 91.38 | reproduced· optimizedT1 | 2025-08-07 |
| 36 | Qwen 3.6 Max (Preview) | 91.1 | reproduced· optimizedT1 | 2026-04-20 |
| 37 | Inkling-Small | 89.99 | reproduced· optimizedT1 | 2026-07-30 |
| 38 | Muse Spark | 88.89 | reproduced· optimizedT1 | 2026-04-08 |
| 39 | gpt-oss-120b | 88.88 | reproduced· optimizedT1 | 2025-08-05 |
| 40 | Inkling | 88.88 | reproduced· optimizedT1 | 2026-07-15 |
| 41 | GPT-5.4 Mini | 88.88 | reproduced· optimizedT1 | 2026-03-17 |
| 42 | GPT-5.1 | 88.6 | reproduced· optimizedT1 | 2025-11-13 |
| 43 | DeepSeek-V3.2 | 87.81 | reproduced· optimizedT1 | 2025-12-01 |
| 44 | GPT-5.4 Nano | 87.77 | reproduced· optimizedT1 | 2026-03-17 |
| 45 | Qwen 3.5 Plus (hosted 397B-A17B) | 86.65 | reproduced· optimizedT1 | 2026-02-16 |
| 46 | Qwen 3.6 35B-A3B | 86.65 | reproduced· optimizedT1 | 2026-04-14 |
| 47 | Qwen3-235B-A22B-Thinking (Jul 2025) | 86.65 | reproduced· optimizedT1 | 2025-07-25 |
| 48 | GPT-5 mini | 86.65 | reproduced· optimizedT1 | 2025-08-07 |
| 49 | GLM-5.2 | 86.38 | reproduced· optimizedT1 | 2026-06-16 |
| 50 | Claude Opus 4.5 | 86.1 | reproduced· optimizedT1 | 2025-11-24 |
| 51 | Claude Sonnet 4.6 | 85.78 | reproduced· optimizedT1 | 2026-02-17 |
| 52 | Gemini 2.5 Pro (Jun 2025) | 84.71 | reproduced· optimizedT1 | 2025-06-05 |
| 53 | Qwen 3.5 Flash (hosted 35B-A3B) | 84.43 | reproduced· optimizedT1 | 2026-02-25 |
| 54 | Qwen 3.6 Flash | 84.43 | reproduced· optimizedT1 | 2026-04-26 |
| 55 | o3 | 84.43 | reproduced· optimizedT1 | 2024-12-20 |
| 56 | Grok 4 | 83.98 | reproduced· optimizedT1 | 2025-07-09 |
| 57 | GLM-4.7 | 83.32 | reproduced· optimizedT1 | 2025-12-22 |
| 58 | Kimi K2 Thinking | 83.04 | reproduced· optimizedT1 | 2025-11-06 |
| 59 | o4-mini | 81.65 | reproduced· optimizedT1 | 2025-04-16 |
| 60 | GPT-5 nano | 81.09 | reproduced· optimizedT1 | 2025-08-07 |
| 61 | GLM-5 | 79.98 | reproduced· optimizedT1 | 2026-02-11 |
| 62 | Gemini 3.1 Flash-Lite | 79.98 | reproduced· optimizedT1 | 2026-03-03 |
| 63 | Grok-3 mini | 77.76 | reproduced· optimizedT1 | 2025-02-19 |
| 64 | Claude Sonnet 4.5 | 77.76 | reproduced· optimizedT1 | 2025-09-29 |
| 65 | o3-mini | 76.92 | reproduced· optimizedT1 | 2025-01-31 |
| 66 | Gemma 4 31B IT | 73.31 | reproduced· optimizedT1 | 2026-04-02 |
| 67 | Qwen3-Max | 73.31 | reproduced· optimizedT1 | 2025-09-05 |
| 68 | o1 | 73.31 | reproduced· optimizedT1 | 2024-12-05 |
| 69 | Gemini 2.5 Flash (Apr 2025) | 73.03 | reproduced· optimizedT1 | 2025-04-17 |
| 70 | Gemini 3.5 Flash-Lite | 71.08 | reproduced· optimizedT1 | 2026-07-21 |
| 71 | MiniMax-M3 | 71.08 | reproduced· optimizedT1 | 2026-06-01 |
| 72 | Claude Sonnet 4 | 71.08 | reproduced· optimizedT1 | 2025-05-22 |
| 73 | Gemini 2.5 Flash (May 2025) | 70.8 | reproduced· optimizedT1 | 2025-05-20 |
| 74 | Claude Opus 4.1 | 68.86 | reproduced· optimizedT1 | 2025-08-05 |
| 75 | GPT-5.5 Instant | 68.02 | reproduced· optimizedT1 | 2026-05-05 |
| 76 | Claude Haiku 4.5 | 66.63 | reproduced· optimizedT1 | 2025-10-15 |
| 77 | DeepSeek-R1 (May 2025) | 66.36 | reproduced· optimizedT1 | 2025-05-28 |
| 78 | Claude Opus 4 | 64.41 | reproduced· optimizedT1 | 2025-05-22 |
| 79 | Claude 3.7 Sonnet | 57.74 | reproduced· optimizedT1 | 2025-02-24 |
| 80 | Gemini 2.0 Flash Thinking (Jan 2025) | 57.74 | reproduced· optimizedT1 | 2025-01-21 |
| 81 | Grok 3 | 55.51 | reproduced· optimizedT1 | 2025-02-17 |
| 82 | gpt-oss-20b | 53.84 | reproduced· optimizedT1 | 2025-08-05 |
| 83 | DeepSeek-R1 | 53.29 | reproduced· optimizedT1 | 2025-01-20 |
| 84 | o1-mini | 46.89 | reproduced· optimizedT1 | 2024-09-12 |
| 85 | GPT-4.1 mini | 44.67 | reproduced· optimizedT1 | 2025-04-14 |
| 86 | GPT-4.1 | 38.27 | reproduced· optimizedT1 | 2025-04-14 |
| 87 | DeepSeek-V3 (Mar 2025) | 37.72 | reproduced· optimizedT1 | 2025-03-24 |
| 88 | GPT-4.5 | 37.72 | reproduced· optimizedT1 | 2025-02-27 |
| 89 | Mistral Medium 3 | 32.15 | reproduced· optimizedT1 | 2025-05-07 |
| 90 | o1-preview | 31.04 | reproduced· optimizedT1 | 2024-09-12 |
| 91 | Gemini 2.0 Flash (Feb 2025) | 31.04 | reproduced· optimizedT1 | 2024-12-11 |
| 92 | Magistral Small 1.0 | 29.93 | reproduced· optimizedT1 | 2025-06-10 |
| 93 | GPT-4.1 nano | 28.82 | reproduced· optimizedT1 | 2025-04-14 |
| 94 | Gemini 1.5 Pro (Sept 2024) | 22.98 | reproduced· optimizedT1 | 2024-09-24 |
| 95 | Gemma 3 27B | 22.14 | reproduced· optimizedT1 | 2025-03-12 |
| 96 | Llama 4 Maverick | 20.48 | reproduced· optimizedT1 | 2025-04-05 |
| 97 | Gemini 1.5 Flash (Sep 2024) | 16.17 | reproduced· optimizedT1 | 2024-05-10 |
| 98 | Qwen2.5-Max | 16.03 | reproduced· optimizedT1 | 2025-01-25 |
| 99 | DeepSeek-V3 | 15.75 | reproduced· optimizedT1 | 2024-12-24 |
| 100 | Phi-4 | 13.66 | reproduced· optimizedT1 | 2024-12-12 |
| 101 | Grok-2 (Dec 2024) | 11.44 | reproduced· optimizedT1 | 2024-08-13 |
| 102 | Llama 3.1-405B | 9.63 | reproduced· optimizedT1 | 2024-07-23 |
| 103 | Claude 3.5 Sonnet (October 2024) | 8.38 | reproduced· optimizedT1 | 2024-10-22 |
| 104 | Mistral Large 2 (Jul 2024) | 8.38 | reproduced· optimizedT1 | 2024-07-24 |
| 105 | Qwen2.5-72B | 7.96 | reproduced· optimizedT1 | 2024-09-19 |
| 106 | Llama 4 Scout | 7.69 | reproduced· optimizedT1 | 2025-04-05 |
| 107 | Mistral Large 2 (Nov 2024) | 7.69 | reproduced· optimizedT1 | 2024-07-24 |
| 108 | GPT-4o mini | 6.85 | reproduced· optimizedT1 | 2024-07-18 |
| 109 | Gemini 1.5 Pro (May 2024) | 6.71 | reproduced· optimizedT1 | 2024-05-14 |
| 110 | GPT-4 Turbo (Apr 2024) | 6.57 | reproduced· optimizedT1 | 2024-04-09 |
| 111 | Claude 3.5 Sonnet | 6.43 | reproduced· optimizedT1 | 2024-06-20 |
| 112 | GPT-4o (Aug 2024) | 6.3 | reproduced· optimizedT1 | 2024-05-13 |
| 113 | GPT-4o (May 2024) | 6.16 | reproduced· optimizedT1 | 2024-05-13 |
| 114 | GPT-4o (Nov 2024) | 6.16 | reproduced· optimizedT1 | 2024-05-13 |
| 115 | Mistral Small 3.1 | 5.74 | reproduced· optimizedT1 | 2025-03-17 |
| 116 | Llama 3.3 70B | 5.04 | reproduced· optimizedT1 | 2024-12-06 |
| 117 | Claude 3 Opus | 4.63 | reproduced· optimizedT1 | 2024-03-04 |
| 118 | Claude 3.5 Haiku | 4.21 | reproduced· optimizedT1 | 2024-10-22 |
| 119 | Llama 3-70B | 4.21 | reproduced· optimizedT1 | 2024-04-18 |
| 120 | Gemini 1.5 Flash (May 2024) | 3.79 | reproduced· optimizedT1 | 2024-05-10 |
| 121 | Llama 3.1-70B | 3.51 | reproduced· optimizedT1 | 2024-07-23 |
| 122 | Llama 3.2 90B | 2.54 | reproduced· optimizedT1 | 2024-09-24 |
| 123 | Llama 3.1-8B | 2.4 | reproduced· optimizedT1 | 2024-07-23 |
| 124 | Claude 3 Sonnet | 2.4 | reproduced· optimizedT1 | 2024-03-04 |
| 125 | Claude 2 | 2.4 | reproduced· optimizedT1 | 2023-07-11 |
| 126 | GPT-3.5 Turbo (Jan 2024) | 2.12 | reproduced· optimizedT1 | 2023-06-13 |
| 127 | Mistral Large | 1.85 | reproduced· optimizedT1 | 2024-02-26 |
| 128 | Claude 2.1 | 1.85 | reproduced· optimizedT1 | 2023-11-21 |
| 129 | Claude 3 Haiku | 1.71 | reproduced· optimizedT1 | 2024-03-04 |
| 130 | Gemma 2 27B | 1.29 | reproduced· optimizedT1 | 2024-06-24 |
| 131 | GPT-4 (Jun 2023) | 1.01 | reproduced· optimizedT1 | 2023-06-13 |
| 132 | Gemini 1.0 Pro | 1.01 | reproduced· optimizedT1 | 2023-12-06 |
| 133 | Llama 3-8B | 0.73 | reproduced· optimizedT1 | 2024-04-18 |
| 134 | Gemma 2 9B | 0.46 | reproduced· optimizedT1 | 2024-06-24 |
| 135 | GPT-4 (Mar 2023) | 0.46 | reproduced· optimizedT1 | 2023-03-15 |
| 136 | Llama 2-70B | 0 | reproduced· optimizedT1 | 2023-07-18 |
Continue from this leaderboard into the tightest head-to-head reads.
Weighted composite. Each dimension opens to the source-backed sub-signals it is computed from.
Claims drawn from cited facts, not live model generation.
This benchmark evaluates 136 models, and the difference between its best and worst scores is 100.0 points. A spread that wide supports genuine separation between models, whereas a narrower spread would indicate that close scores may not reflect a real capability gap.
3 cited facts
The benchmark receives an integrity score of 86, corresponding to an integrity grade of A, and is ranked 34th among 61 benchmarks, indicating its health as a discriminator of frontier models under a consistent harness. Contamination is the single weakest component of its integrity breakdown, and that contamination narrows how much to trust top scores.
5 cited facts
This benchmark is saturated. The top score is 100.0, leaving no headroom, and ten models cluster within a couple of points of the top. Small differences among the leading models should therefore not be read as meaningful capability gaps.
5 cited facts
Scores represent task performance under a consistent harness, so results are directly comparable within this benchmark. The test set privacy is unknown, meaning it is not clear whether high scores warrant extra scrutiny for possible contamination; if the test set were public, high scores would deserve more scrutiny than a held-out set. Contamination history is unknown, and the disclosed harness records model version, optimized flag, score source, and benchmark release date, providing qualitative context for interpreting results.
4 cited facts
The methodology of this benchmark is largely undocumented: what it measures is not stated. How the test was built is not documented. The assumptions it relies on are not documented. The sharpest caveat is that its limitations are undisclosed, so a reader cannot know where it is most likely to mislead.
4 cited facts
OTIS Mock AIME 2024-2025: OTIS Mock AIME 2024-2025 as reported in Epoch AI's Capabilities Index CSV.
Claude Fable 5 leads OTIS Mock AIME 2024-2025 at 100 — independently reproduced. The full leaderboard above lists every recorded measurement, not just the headline number.
136 models have recorded scores on OTIS Mock AIME 2024-2025, spanning a score spread of 100.
tensor.news grades OTIS Mock AIME 2024-2025 A for integrity (score 86/100), ranking #34 of 61 benchmarks we assess across discrimination, saturation, contamination resistance, harness comparability, and freshness.
Yes — top scores are clustering near the ceiling (saturation ratio 0.07), so OTIS Mock AIME 2024-2025 no longer separates leading models well. Treat small gaps at the top with caution.
Its test set is unknown, so contamination risk is on the table: a high score may partly reflect training-data overlap rather than capability.
Scores are reported under a consistent harness, so comparisons on OTIS Mock AIME 2024-2025 are reasonably apples-to-apples.
Every OTIS Mock AIME 2024-2025 measurement is source-backed and tagged reproduced, self-reported, or unverified; the underlying record cites epoch.ai/data/eci_benchmarks.csv.