DeepSWE
Integrity rank #14 of 61 · 21 models scored · top score 73.65 · Claude Opus 5
unknown
Strongest on discrimination, weakest on saturation headroom. Caveat: the public test set does not rule out training-data contamination.
Reported scores — protocols may differ. How we compare
| # | Model | Score | Evidence | Measured |
|---|---|---|---|---|
| 1 | Claude Opus 5 | 73.65 | unverifiedT1 | 2026-07-24 |
| 2 | GPT-5.6 Sol | 72.67 | unverifiedT1 | 2026-07-09 |
| 3 | Claude Fable 5 | 69.91 | unverifiedT1 | 2026-06-09 |
| 4 | GPT-5.6 Terra | 69.62 | unverifiedT1 | 2026-07-09 |
| 5 | Kimi K3 | 68.51 | unverifiedT1 | 2026-07-16 |
| 6 | Grok 4.6 | 67.48 | unverifiedT1 | 2026-08-12 |
| 7 | GPT-5.6 Luna | 67.19 | unverifiedT1 | 2026-07-09 |
| 8 | GPT-5.5 | 67.04 | unverifiedT1 | 2026-04-23 |
| 9 | Gemini 3.7 Flash | 65.49 | unverifiedT1 | 2026-08-13 |
| 10 | Claude Opus 4.8 | 58.97 | unverifiedT1 | 2026-05-28 |
| 11 | Qwen 3.8 Max | 57.46 | unverifiedT1 | 2026-07-19 |
| 12 | Claude Sonnet 5 | 53.85 | unverifiedT1 | 2026-06-30 |
| 13 | Grok 4.5 | 53.76 | unverifiedT1 | 2026-07-08 |
| 14 | Muse Spark 1.1 | 53.32 | unverifiedT1 | 2026-07-09 |
| 15 | GPT-5.4 | 51.77 | unverifiedT1 | 2026-03-05 |
| 16 | Gemini 3.6 Flash | 48.56 | unverifiedT1 | 2026-07-21 |
| 17 | GLM-5.2 | 43.78 | unverifiedT1 | 2026-06-16 |
| 18 | Gemini 3.5 Flash | 37.39 | unverifiedT1 | 2026-05-19 |
| 19 | Kimi K2.7 Code | 30.53 | unverifiedT1 | 2026-06-12 |
| 20 | Claude Sonnet 4.6 | 29.93 | unverifiedT1 | 2026-02-17 |
| 21 | Gemini 3.1 Pro | 11.75 | unverifiedT1 | 2026-02-19 |
Continue from this leaderboard into the tightest head-to-head reads.
Weighted composite. Each dimension opens to the source-backed sub-signals it is computed from.
Claims drawn from cited facts, not live model generation.
This benchmark scores 21 models, with a 61.9-point gap between the best and worst results. Such a wide spread suggests that observed rank differences likely reflect genuine capability separation rather than measurement noise.
2 cited facts
The benchmark's Integrity Index is 98, with a grade of A, ranking 14th out of 61 benchmarks. The weakest integrity component is saturation, which means the ceiling is crowded and top scores are less distinguishable. This score should be read as the benchmark's health as a discriminator of frontier models under its disclosed harness, not as any model's capability.
6 cited facts
This benchmark is not saturated, with a top score of 73.65 leaving 26.35 points of headroom, so the ranking is not yet close to noise. Only two models cluster near the top, within a couple of points, meaning the leading field remains sparsely packed. Because saturation is false, small differences among top models still reflect genuine capability gaps rather than measurement noise.
5 cited facts
This benchmark's scores were produced under a consistent harness, so task performance is directly comparable across runs within that setup. Test-set privacy is unknown, so it cannot be determined whether high scores should receive the extra scrutiny for contamination that a public test set would warrant. Contamination history is unknown, and the available harness notes do not support a qualitative assessment of sensitivity to harness configuration.
4 cited facts
This benchmark's intended measurement is not documented, as what it measures is listed as unknown. Its construction is likewise undocumented, so the process by which it was built cannot be described. Because both key assumptions and limitations are recorded as unknown, the sharpest caveat is that no documented caveat can be stated here, and the benchmark's most likely failure modes remain unspecified.
4 cited facts
DeepSWE: DeepSWE as reported in Epoch AI's Capabilities Index CSV.
Claude Opus 5 leads DeepSWE at 73.65. The full leaderboard above lists every recorded measurement, not just the headline number.
21 models have recorded scores on DeepSWE, spanning a score spread of 61.9.
tensor.news grades DeepSWE A for integrity (score 98/100), ranking #14 of 61 benchmarks we assess across discrimination, saturation, contamination resistance, harness comparability, and freshness.
No — DeepSWE still has headroom and continues to discriminate between models rather than bunching them at the ceiling.
Its test set is unknown, so contamination risk is on the table: a high score may partly reflect training-data overlap rather than capability.
Scores are reported under a consistent harness, so comparisons on DeepSWE are reasonably apples-to-apples.
Every DeepSWE measurement is source-backed and tagged reproduced, self-reported, or unverified; the underlying record cites epoch.ai/data/eci_benchmarks.csv.