tensor.news
Integrity IndexModelsCompareLabsPapersMethodologyAbout
tensor.news

© 2026

Evidence

  • Integrity Index
  • Model Claim Ledger
  • Compare

Entities

  • Models
  • Benchmarks
  • Labs
  • Papers

Method

  • Methodology
  • About
  • Privacy

Machine

  • Feed (Atom)
  • llms.txt
  • llms-full.txt
  • Sitemap

tensor.news grades benchmarks and tags model claims by evidence status. Every fact carries a source record and provenance. Scores are framed as task performance under a disclosed harness, never as deployed capability.

← Integrity Index

SWE-Bench verified

SWE-bench Verified

Rank #28 of 51 · 32 models scored · top score 83.47

BINTEGRITY 83 / 100

Resolve real GitHub issues so hidden repo tests pass

Strongest on discrimination, weakest on contamination resistance. Caveat: the public test set leaves contamination on the table.

consistent harnesspublic test set

Headroom to ceiling

16.5 headroomtop 83.5ceiling 100

Score distribution

median 73.8Claude Opus 4.7 83.5285786

State of the art over time

28578683.520242026

Integrity breakdown

Weighted composite. Each signal is computed from the source-backed leaderboard below.

Discrimination30%100

Does it still separate models?

Saturation headroom30%97

How far from ceiling / clustered at the top?

Contamination resistance15%3

Public vs held-out; training-leak risk.

Harness comparability15%100

Apples-to-apples, or mixed / vendor-optimized?

Freshness10%82

How old is the benchmark?

Leaderboard

#ModelScoreEvidenceMeasured
1Claude Opus 4.783.47reproduced· optimized2026-04-16
2GPT-5.580.58reproduced· optimized2026-04-23
3Gemini 3.5 Flash79.34reproduced· optimized2026-05-19
4Claude Opus 4.678.72reproduced· optimized2026-02-05
5GLM-5.278.7reproduced· optimized2026-06-16
6DeepSeek-V4-Pro77.64reproduced· optimized2026-04-24
7Qwen3.7-Max77.27reproduced· optimized2026-05-19
8GPT-5.476.86reproduced· optimized2026-03-05
9Kimi K2.676.65reproduced· optimized2026-04-20
10Qwen 3.6 Max (Preview)76.65reproduced· optimized2026-04-20
11Claude Opus 4.576.65reproduced· optimized2025-11-24
12Gemini 3.1 Pro75.62reproduced· optimized2026-02-19
13Gemini 3 Flash75.41reproduced· optimized2025-12-17
14Claude Sonnet 4.675.21reproduced· optimized2026-02-17
15GPT-5.3 Codex74.79reproduced· optimized2026-02-05
16GLM-5.174.17reproduced· optimized2026-04-07
17Kimi K2.573.76reproduced· optimized2026-02-02
18GPT-5.273.76reproduced· optimized2025-12-11
19GPT-573.55reproduced· optimized2025-08-07
20Claude Opus 4.173.35reproduced· optimized2025-08-05
21Gemini 3 Pro72.93reproduced· optimized2025-11-18
22GLM-572.08reproduced· optimized2026-02-11
23Claude Sonnet 4.571.28reproduced· optimized2025-09-29
24Claude Opus 470.66reproduced· optimized2025-05-22
25GPT-5.167.98reproduced· optimized2025-11-13
26GPT-5 mini64.67reproduced· optimized2025-08-07
27o362.32reproduced· optimized2024-12-20
28Claude 3.7 Sonnet60.95reproduced· optimized2025-02-24
29Qwen 3.6 Plus57.85reproduced· optimized2026-04-01
30Gemini 2.5 Pro (Jun 2025)57.56reproduced· optimized2025-06-05
31GPT-4.148.54reproduced· optimized2025-04-14
32GPT-4o (Nov 2024)30.99reproduced· optimized2024-05-13

At a glance

Models scored
32
Top model
Claude Opus 4.7
Top score
83.47
Score spread
52.48
Headroom
16.53

Benchmark context

Task type
agentic-coding
Metric
% resolved (pass@1, test-verified)
Score ceiling
100
Human baseline
N/A (real merged fix = 100% by construction)
Test-set privacy
public_plus_private_holdout
Construction
500 human-validated (OpenAI-filtered) tasks from 2294 SWE-bench items across 12 repos
Key assumptions
Passing hidden tests = correctly fixing the issue
Limitations
Gameable w/ overfit patches; Python-only; score dominated by agent scaffold not model
Contamination history
Public GitHub PRs predate cutoffs; documented 'cheating' by reading future commit state
Harness notes
Fully agentic edit-run-test; same model swings 10-30+ pts across scaffolds — core integrity problem
Authors
OpenAI (Verified); Princeton/UChicago (base)
Released
2024-08
License
MIT (harness)
Source
openai.com/index/introducing-swe-bench-verified/

Provenance

32 source records · scores from epoch_benchmarks. Every fact on this page is evidence-linked.

Follow the record

Integrity neighbors

  • VPCTA · 87
  • SimpleBenchA · 86
  • Terminal BenchB · 84
  • CL-benchB · 82
  • CL-bench LifeB · 81
  • GPQA diamondB · 81

Top models here

  • Claude Opus 4.783.47
  • GPT-5.580.58
  • Gemini 3.5 Flash79.34
  • Claude Opus 4.678.72
  • GLM-5.278.7

Head-to-head

  • Claude Opus 4.7 vs GPT-5.5
  • Claude Opus 4.7 vs Gemini 3.5 Flash
  • GPT-5.5 vs Gemini 3.5 Flash