MATH-500 (500-problem MATH subset)
Integrity rank #46 of 61 · 8 models scored · top score 97.3 · DeepSeek-R1
Competition-level math problem solving with exact final answers
Strongest on contamination resistance, weakest on discrimination. Caveat: the public test set does not rule out training-data contamination.
Reported scores — protocols may differ. How we compare
| # | Model | Score | Evidence | Measured |
|---|---|---|---|---|
| 1 | DeepSeek-R1 | 97.3 | self-reported· optimizedT1 | 2025-01-22 |
| 2 | DeepSeek-R1-Distill-Llama-70B | 94.5 | self-reported· optimizedT1 | 2025-01-22 |
| 3 | DeepSeek-R1-Distill-Qwen-32B | 94.3 | self-reported· optimizedT1 | 2025-01-22 |
| 4 | DeepSeek-R1-Distill-Qwen-14B | 93.9 | self-reported· optimizedT1 | 2025-01-22 |
| 5 | DeepSeek-R1-Distill-Qwen-7B | 92.8 | self-reported· optimizedT1 | 2025-01-22 |
| 6 | DeepSeek-V3 | 90.2 | self-reported· optimizedT1 | 2025-01-22 |
| 7 | DeepSeek-R1-Distill-Llama-8B | 89.1 | self-reported· optimizedT1 | 2025-01-22 |
| 8 | DeepSeek-R1-Distill-Qwen-1.5B | 83.9 | self-reported· optimizedT1 | 2025-01-22 |
Continue from this leaderboard into the tightest head-to-head reads.
Weighted composite. Each dimension opens to the source-backed sub-signals it is computed from.
Claims drawn from cited facts, not live model generation.
This benchmark scores eight models, producing a score spread of 13.4 points across the leaderboard. Because this spread is wide, it supports real separation between the models and indicates that rank differences likely reflect genuine capability gaps rather than marginal variations.
2 cited facts
This benchmark is not currently saturated, a status confirmed by the flag. The leading performance reaches 97.3, leaving a remaining headroom of 2.7 points to the theoretical ceiling. With performance clustered within a narrow band of models near the top, there is still genuine room to separate the field, meaning today's ranking among the top performers is not yet close to noise.
4 cited facts
Task performance under this benchmark utilizes a consistent harness, meaning scores are directly comparable across evaluations, while the public test set means high scores deserve more scrutiny for possible contamination than a held-out alternative. Although no specific contamination history is documented, the open nature of the evaluation data necessitates careful verification of model training pipelines. All reported metrics should be interpreted strictly as task performance under this disclosed harness rather than indicators of deployed capability.
2 cited facts
This benchmark evaluates competition-level math problem solving with exact final answers. The test is constructed as a subset of problems from an earlier benchmark to assess model capabilities. The methodology assumes that exact string matching determines correctness, meaning semantically equivalent answers formatted differently may be penalized. A critical caveat is that the public availability of the source data since the original release creates a high risk of contamination for models trained on recent corpora.
4 cited facts
MATH-500 (MATH-500 (500-problem MATH subset)): MATH-500 — a 500-problem subset of the MATH competition dataset (Hendrycks et al.), drawn from the hardest levels; exact-match on the final boxed answer, scored as pass@1 (%). Introduced in OpenAI's o1 evaluation as the headline math set.
DeepSeek-R1 leads MATH-500 at 97.3 — self-reported by the lab. The full leaderboard above lists every recorded measurement, not just the headline number.
8 models have recorded scores on MATH-500, spanning a score spread of 13.4.
tensor.news grades MATH-500 B for integrity (score 76/100), ranking #46 of 61 benchmarks we assess across discrimination, saturation, contamination resistance, harness comparability, and freshness.
No — MATH-500 still has headroom and continues to discriminate between models rather than bunching them at the ceiling.
Its test set is public, so contamination risk is on the table: a high score may partly reflect training-data overlap rather than capability.
Scores are reported under a consistent harness, so comparisons on MATH-500 are reasonably apples-to-apples.
Every MATH-500 measurement is source-backed and tagged reproduced, self-reported, or unverified; the underlying record cites openai.com/index/learning-to-reason-with-llms.