Methodology

A benchmark score is a function of the model, the harness, and the protocol, not of the model alone. Run the same model through a different scaffold and the number moves. And even a clean score on a held-out benchmark doesn't tell you how a model performs on an open-ended, real-world, agentic task. So we don't only grade models. We grade the evaluations.

The Integrity score

Every benchmark on tensor.news carries an Integrity score from 0 to 100, built from five weighted signals:

Integrity = 0.30 · discrimination + 0.30 · saturation + 0.15 · contamination + 0.15 · harness + 0.10 · age

Discrimination (30%)

Whether the benchmark still separates models. When every model clusters within a point or two of each other, the benchmark has stopped measuring anything useful. We score this from the spread of results across the full field, not just the top few entries.

Saturation (30%)

How close the leading models sit to the ceiling, and how tightly they're bunched there. A benchmark near saturation can no longer tell a genuinely better model apart from one that's merely tied for first.

Contamination (15%)

Whether the test set is public or held out, and whether a model's release date postdates the benchmark's publication. A public test set paired with a model trained after it was released is a training-leak risk, not a clean measurement.

Harness comparability (15%)

Whether scores come from one comparable evaluation setup, or a mix of vendor-optimized and independently-run scaffolds. We lean on Epoch's optimized flag, plus whether a score is independently reproduced or self-reported, to catch mixed harnesses before they get averaged into a single leaderboard.

Age (10%)

How long the benchmark has been in circulation. Older benchmarks accumulate more contamination surface and are more likely to have been optimized against directly, whether deliberately or not.

Grade bands

The composite score maps to a letter grade:

• A (85-100): still discriminating, far from saturated, low contamination risk, consistent harness.
• B (70-84): solid on most signals, with at least one showing early wear.
• C (55-69): usable, but read the scores as directional, not authoritative.
• D (40-54): multiple weak signals; the top of the leaderboard is likely noise.
• F (<40): saturated, contaminated, or run under incomparable harnesses. The score no longer measures the claim it's attached to.

The Model Claim Ledger

Every benchmark score attached to a model carries a status: independently reproduced, self-reported, contradicted, or unverified. Independently-reproduced scores come from Epoch's own evaluation runs; self-reported scores come from a vendor's technical report. We keep the two separate rather than blending them into one number.

What a score is not

Every number on this site is task performance under a disclosed harness, never a statement about deployed, real-world, or agentic capability. Where a benchmark's integrity is low, or its harness is mixed, we say so next to the score rather than leaving it to be read as a verdict.

Provenance

Every fact behind these scores links to a source record: which evaluator ran it, when it was measured, and where it was published. Nothing here is asserted without an evidence trail you can follow.