GSO: Challenging Software Optimization Tasks for Evaluating SWE-Agents
Integrity rank #24 of 65 · 28 models scored · top score 88.24 · Claude Fable 5.1
SWE-agents' ability to write high-performance code optimizations
Strongest on discrimination, weakest on contamination resistance. Caveat: the public test set does not rule out training-data contamination.
Reported scores — protocols may differ. How we compare
| # | Model | Score | Evidence | Measured |
|---|---|---|---|---|
| 1 | Claude Fable 5.1 | 88.24 | unverifiedT1 | 2026-09-01 |
| 2 | GPT-6 Astra | 79.41 | unverifiedT1 | 2026-09-03 |
| 3 | Claude Fable 5 | 78.43 | unverifiedT1 | 2026-06-09 |
| 4 | GPT-5.6 Sol | 76.47 | unverifiedT1 | 2026-07-09 |
| 5 | Claude Opus 4.8 | 47.06 | unverifiedT1 | 2026-05-28 |
| 6 | Claude Opus 4.7 | 44.12 | unverifiedT1 | 2026-04-16 |
| 7 | Claude Opus 4.6 | 41.2 | unverifiedT1 | 2026-02-05 |
| 8 | GPT-5.5 | 40.2 | unverifiedT1 | 2026-04-23 |
| 9 | Claude Sonnet 5 | 37.25 | unverifiedT1 | 2026-06-30 |
| 10 | GPT-5.4 | 31.37 | unverifiedT1 | 2026-03-05 |
| 11 | GPT-5.2 | 27.45 | unverifiedT1 | 2025-12-11 |
| 12 | Claude Opus 4.5 | 26.5 | unverifiedT1 | 2025-11-24 |
| 13 | Gemini 3.1 Pro | 22.55 | unverifiedT1 | 2026-02-19 |
| 14 | Gemini 3 Pro | 18.63 | unverifiedT1 | 2025-11-18 |
| 15 | Claude Sonnet 4.5 | 14.71 | unverifiedT1 | 2025-09-29 |
| 16 | GPT-5.1 | 13.73 | unverifiedT1 | 2025-11-13 |
| 17 | Gemini 3 Flash | 9.8 | unverifiedT1 | 2025-12-17 |
| 18 | o3 | 8.8 | unverifiedT1 | 2025-04-16 |
| 19 | Claude Opus 4 | 6.9 | unverifiedT1 | 2025-05-22 |
| 20 | GPT-5 | 6.86 | unverifiedT1 | 2025-08-07 |
| 21 | Kimi K2 (Jul 2025) | 4.9 | unverifiedT1 | 2025-07-12 |
| 22 | Claude Sonnet 4 | 4.9 | unverifiedT1 | 2025-05-22 |
| 23 | Claude 3.5 Sonnet (October 2024) | 4.6 | unverifiedT1 | 2024-10-22 |
| 24 | Gemini 2.5 Pro (Jun 2025) | 3.92 | unverifiedT1 | 2025-06-05 |
| 25 | Claude 3.7 Sonnet | 3.8 | unverifiedT1 | 2025-02-24 |
| 26 | o4-mini | 3.6 | unverifiedT1 | 2025-04-16 |
| 27 | o3-mini | 1.3 | unverifiedT1 | 2025-01-31 |
| 28 | GPT-4o (Nov 2024) | 0 | unverifiedT1 | 2024-11-20 |
Continue from this leaderboard into the tightest head-to-head reads.
Weighted composite. Each dimension opens to the source-backed sub-signals it is computed from.
Claims drawn from cited facts, not live model generation.
This benchmark tests software agents on writing high-performance code optimizations.
1 cited fact
This benchmark has scores from 28 tracked models, with a recorded score range spanning 88.24.
2 cited facts
Its Benchmark Integrity Index score is 88, with grade A and rank 24 out of 65 tracked benchmarks.
4 cited facts
It is not flagged as saturated. The top recorded score is 88.24, with 11.76 points of headroom. The computed top-score cluster contains 1 models.
4 cited facts
The harness classification is consistent harness. Test-set privacy is recorded as public. The recorded contamination-window ratio is 0.714.
3 cited facts
GSO-Bench (GSO: Challenging Software Optimization Tasks for Evaluating SWE-Agents): GSO (canonical name 'GSO'; the acronym is not expanded in the paper, titled 'Challenging Software Optimization Tasks for Evaluating SWE-Agents') tasks coding agents with modifying real-world codebases to achieve expert-level runtime speedups, verified by performance tests. Leading SWE-agents solve less than 5%.
Claude Fable 5.1 leads GSO-Bench at 88.24. The full leaderboard above lists every recorded measurement, not just the headline number.
28 models have recorded scores on GSO-Bench, spanning a score spread of 88.24.
tensor.news grades GSO-Bench A for integrity (score 88/100), ranking #24 of 65 benchmarks we assess across discrimination, saturation, contamination resistance, harness comparability, and freshness.
No — GSO-Bench still has headroom and continues to discriminate between models rather than bunching them at the ceiling.
Its test set is public, so contamination risk is on the table: a high score may partly reflect training-data overlap rather than capability.
Scores are reported under a consistent harness, so comparisons on GSO-Bench are reasonably apples-to-apples.
Every GSO-Bench measurement is source-backed and tagged reproduced, self-reported, or unverified; the underlying record cites arxiv.org/abs/2505.23671.