DeepSWE
Integrity rank #36 of 65 · 26 models scored · top score 74.12 · GPT-6 Astra
unknown
Strongest on discrimination, weakest on contamination resistance. Caveat: the public test set does not rule out training-data contamination.
Reported scores — protocols may differ. How we compare
| # | Model | Score | Evidence | Measured |
|---|---|---|---|---|
| 1 | GPT-6 Astra | 74.12 | unverifiedT1 | 2026-09-03 |
| 2 | Gemini 3.8 Flash | 73.83 | unverifiedT1 | 2026-09-02 |
| 3 | Claude Opus 5 | 73.65 | unverifiedT1 | 2026-07-24 |
| 4 | GPT-5.6 Sol | 72.67 | unverifiedT1 | 2026-07-09 |
| 5 | Claude Fable 5 | 69.91 | unverifiedT1 | 2026-06-09 |
| 6 | GPT-5.6 Terra | 69.62 | unverifiedT1 | 2026-07-09 |
| 7 | GLM-5.3 | 68.96 | unverifiedT1 | 2026-08-14 |
| 8 | Kimi K3 | 68.51 | unverifiedT1 | 2026-07-16 |
| 9 | Grok 4.6 | 67.48 | unverifiedT1 | 2026-08-12 |
| 10 | GPT-5.6 Luna | 67.19 | unverifiedT1 | 2026-07-09 |
| 11 | GPT-5.5 | 67.04 | unverifiedT1 | 2026-04-23 |
| 12 | Gemini 3.7 Flash | 65.49 | unverifiedT1 | 2026-08-13 |
| 13 | GLM-5.3-Flash | 63.39 | unverifiedT1 | 2026-08-20 |
| 14 | Claude Opus 4.8 | 58.97 | unverifiedT1 | 2026-05-28 |
| 15 | Qwen 3.8 Max | 57.46 | unverifiedT1 | 2026-08-02 |
| 16 | Muse Spark 1.2 | 54.87 | unverifiedT1 | 2026-08-05 |
| 17 | Claude Sonnet 5 | 53.85 | unverifiedT1 | 2026-06-30 |
| 18 | Grok 4.5 | 53.76 | unverifiedT1 | 2026-07-08 |
| 19 | Muse Spark 1.1 | 53.32 | unverifiedT1 | 2026-07-09 |
| 20 | GPT-5.4 | 51.77 | unverifiedT1 | 2026-03-05 |
| 21 | Gemini 3.6 Flash | 46.68 | unverifiedT1 | 2026-07-21 |
| 22 | GLM-5.2 | 43.78 | unverifiedT1 | 2026-06-16 |
| 23 | Gemini 3.5 Flash | 37.39 | unverifiedT1 | 2026-05-19 |
| 24 | Kimi K2.7 Code | 30.53 | unverifiedT1 | 2026-06-12 |
| 25 | Claude Sonnet 4.6 | 29.93 | unverifiedT1 | 2026-02-17 |
| 26 | Gemini 3.1 Pro | 11.73 | unverifiedT1 | 2026-02-19 |
Continue from this leaderboard into the tightest head-to-head reads.
Weighted composite. Each dimension opens to the source-backed sub-signals it is computed from.
Claims drawn from cited facts, not live model generation.
This benchmark scores 26 models, and the gap between the best and worst performing models spans 62.39 points. Because that spread is wide, the models are genuinely separated in performance, meaning the rank differences observed here are more likely to reflect real capability gaps than close, hard-to-distinguish scores.
3 cited facts
This benchmark's Benchmark Integrity Index is 84, graded B, and ranks 35 of 62 benchmarks, reflecting the benchmark's health as a discriminator of frontier models under a disclosed harness rather than any model's capability. The weakest integrity component is contamination, which narrows how much readers should trust top scores.
6 cited facts
The top score is 74.12, leaving 25.88 points of headroom to the ceiling, with four models clustered within a couple of points of the top. Since the saturation flag is false, the benchmark is not saturated, meaning there is still genuine room to separate the field at the top rather than treating small differences among the leaders as noise.
4 cited facts
For this benchmark, harness comparability is consistent, meaning scores are directly comparable under the disclosed harness; if comparability were low, scores from different harnesses should not be compared head-to-head. test set privacy is unknown, so the test set cannot be confirmed as public or held out, and contamination history is unknown, so no contamination record is available; if the test set were public, high task-performance scores would warrant more contamination scrutiny than a held-out set. The harness notes identify only reporting fields, so scores should be understood as task performance under a disclosed harness rather than as deployed capability or broad model quality.
4 cited facts
What this benchmark measures is not documented, so its target capability or behavior cannot be stated here. The construction procedure is likewise not documented, leaving unstated how items or tasks were authored, sampled, or validated. The benchmark's key assumptions are not documented, as are its limitations and any human baseline for comparison. The sharpest caveat follows directly: because what it measures, how it was built, and its assumptions and limitations are all undocumented, a reader cannot judge its validity or know what conclusions it can and cannot support.
9 cited facts
DeepSWE: DeepSWE as reported in Epoch AI's Capabilities Index CSV.
GPT-6 Astra leads DeepSWE at 74.12. The full leaderboard above lists every recorded measurement, not just the headline number.
26 models have recorded scores on DeepSWE, spanning a score spread of 62.39.
tensor.news grades DeepSWE B for integrity (score 84/100), ranking #36 of 65 benchmarks we assess across discrimination, saturation, contamination resistance, harness comparability, and freshness.
No — DeepSWE still has headroom and continues to discriminate between models rather than bunching them at the ceiling.
Its test set is unknown, so contamination risk is on the table: a high score may partly reflect training-data overlap rather than capability.
Scores are reported under a consistent harness, so comparisons on DeepSWE are reasonably apples-to-apples.
Every DeepSWE measurement is source-backed and tagged reproduced, self-reported, or unverified; the underlying record cites epoch.ai/data/eci_benchmarks.csv.