Benchmark explainer
What is Cybench?
Cybersecurity reasoning on agentic capture-the-flag tasks modeled after real-world CTF competitions.
Cybench as reported in Epoch AI's Capabilities Index CSV.
How it's scored
- Metric
- normalized performance in Epoch CSV, rendered as accuracy (%)
- Score ceiling
- 100
- Construction
- 40 professional CTF tasks sourced from 4 competitions (HackTheBox, SekaiCTF, Glacier, HKCert, 2022-2024) across 6 cybersecurity categories.
- Human baseline
- Human expert solve-rates vary by category (paper §4); original CTF 'first-solve time' ranges 2 min – 25 hr.
How trustworthy
Discrimination
100/100Does it still separate models?
Saturation headroom
95.1/100How far from ceiling / clustered at the top?
Contamination resistance
26.3/100Public vs held-out; training-leak risk.
Harness comparability
50/100Is it apples-to-apples, or mixed / vendor-optimized?
Freshness
76.4/100How old is the benchmark?
Contamination history: Released 2024-08 with full public task text; source CTFs span 2022-2024 — any model trained on 2024 web crawl risks exposure.
Limitations: Subtask formulation biases toward step-by-step agents; full tasks exceed current frontier capability; publicly available CTF source text risks contamination.
Who leads Cybench
| Model | Score | Evidence |
|---|---|---|
| Claude Opus 4.6 | 93 | self-reported |
| Claude Opus 4.5 | 82 | self-reported |
| Claude Sonnet 4.5 | 60 | self-reported |
| Grok 4 | 43 | self-reported |
| Claude Opus 4.1 | 42 | unverified |
Compare the top Cybench scorers
Frequently asked questions
Cybench: Cybench as reported in Epoch AI's Capabilities Index CSV.
Claude Opus 4.6 leads Cybench at 93 — self-reported by the lab. The full leaderboard above lists every recorded measurement, not just the headline number.
19 models have recorded scores on Cybench, spanning a score spread of 88.
tensor.news grades Cybench B for integrity (score 78/100), ranking #44 of 65 benchmarks we assess across discrimination, saturation, contamination resistance, harness comparability, and freshness.
No — Cybench still has headroom and continues to discriminate between models rather than bunching them at the ceiling.
Its test set is unknown, so contamination risk is on the table: a high score may partly reflect training-data overlap rather than capability.
Only partly — scores come from a mix of harnesses and protocols, so a head-to-head on Cybench is not fully apples-to-apples.
Every Cybench measurement is source-backed and tagged reproduced, self-reported, or unverified; the underlying record cites arxiv.org/abs/2408.08926.