ScienceQA
Integrity rank #54 of 61 · 6 models scored · top score 84.67 · GPT-4o (May 2024)
unknown
Strongest on discrimination, weakest on contamination resistance. Caveat: scores come from an inconsistent mix of harnesses, so head-to-head comparisons are not apples-to-apples.
Reported scores — protocols may differ. How we compare
| # | Model | Score | Evidence | Measured |
|---|---|---|---|---|
| 1 | GPT-4o (May 2024) | 84.67 | self-reported· optimizedT1 | 2024-05-13 |
| 2 | Claude 3 Haiku | 62.67 | self-reported· optimizedT1 | 2024-03-04 |
| 3 | Llama 2-13B | 41.04 | unverified· optimizedT1 | 2023-07-18 |
| 4 | LLaMA-13B | 24.44 | unverified· optimizedT1 | 2023-02-27 |
| 5 | Llama 2-7B | 24.11 | unverified· optimizedT1 | 2023-07-18 |
| 6 | LLaMA-7B | 14.92 | unverified· optimizedT1 | 2023-02-24 |
Continue from this leaderboard into the tightest head-to-head reads.
Weighted composite. Each dimension opens to the source-backed sub-signals it is computed from.
Claims drawn from cited facts, not live model generation.
Six entrants spread over 69.75 points shows real separation on a tiny sample; read it as a sketch of the range, not a ranking to cite. A spread this wide indicates that score differences among models likely reflect genuine capability gaps rather than measurement noise.
3 cited facts
With an Integrity Index of 68 (grade C), this benchmark ranks 42nd among 51 peers, suffering from contamination as its weakest component, which narrows how much to trust top scores.
4 cited facts
This benchmark is not saturated, meaning today's ranking among the top models still has genuine headroom. The top score of 84.67 leaves 15.33 points to the ceiling, and a single model holds the lead, so small differences at the top are meaningful.
4 cited facts
Mixed-harness scores are the weakest kind this section flags. Differences between models here may be differences between setups, so treat the ordering itself, not just the absolute numbers, as unreliable. Test set privacy is unknown, meaning it is unclear whether the test set is public or held out; combined with an unknown contamination history and harness documentation that reports model version and score source, this cautions against assuming freedom from contamination.
4 cited facts
Undocumented intent, construction, assumptions, and limitations leave these numbers with no stated test to interpret them against; read them as provisional. Because neither the design nor the underlying rationale are recorded, no concrete caveat can be drawn from the provided information.
6 cited facts
ScienceQA: ScienceQA as reported in Epoch AI's Capabilities Index CSV.
GPT-4o (May 2024) leads ScienceQA at 84.67 — self-reported by the lab. The full leaderboard above lists every recorded measurement, not just the headline number.
6 models have recorded scores on ScienceQA, spanning a score spread of 69.75.
tensor.news grades ScienceQA C for integrity (score 67/100), ranking #54 of 61 benchmarks we assess across discrimination, saturation, contamination resistance, harness comparability, and freshness.
No — ScienceQA still has headroom and continues to discriminate between models rather than bunching them at the ceiling.
Its test set is unknown, so contamination risk is on the table: a high score may partly reflect training-data overlap rather than capability.
Only partly — scores come from a mix of harnesses and protocols, so a head-to-head on ScienceQA is not fully apples-to-apples.
Every ScienceQA measurement is source-backed and tagged reproduced, self-reported, or unverified; the underlying record cites epoch.ai/data/eci_benchmarks.csv.