Adversarial Natural Language Inference
Integrity rank #60 of 61 · 9 models scored · top score 37.15 · phi-3-small 7.4B
Robust natural-language inference on adversarially-collected examples
Strongest on harness comparability, weakest on contamination resistance. Caveat: the public test set does not rule out training-data contamination.
Reported scores — protocols may differ. How we compare
| # | Model | Score | Evidence | Measured |
|---|---|---|---|---|
| 1 | phi-3-small 7.4B | 37.15 | self-reported· optimizedT1 | 2024-04-23 |
| 2 | GPT-3.5 Turbo (Nov 2023) | 37.15 | self-reported· optimizedT1 | 2023-06-13 |
| 3 | Llama 3-8B | 35.95 | self-reported· optimizedT1 | 2024-04-18 |
| 4 | phi-3-medium 14B | 33.7 | self-reported· optimizedT1 | 2024-04-23 |
| 5 | Mixtral 8x7B | 32.8 | self-reported· optimizedT1 | 2023-12-11 |
| 6 | phi-3-mini 3.8B | 29.2 | self-reported· optimizedT1 | 2024-04-23 |
| 7 | Gemma 7B | 23.05 | self-reported· optimizedT1 | 2024-02-21 |
| 8 | Mistral 7B v0.1 | 20.65 | self-reported· optimizedT1 | 2023-10-10 |
| 9 | Phi-2 | 13.75 | self-reported· optimizedT1 | 2023-12-12 |
Continue from this leaderboard into the tightest head-to-head reads.
Weighted composite. Each dimension opens to the source-backed sub-signals it is computed from.
Claims drawn from cited facts, not live model generation.
Nine models separated by just 23.4 points is a bunched, thin table; treat neighboring ranks as roughly interchangeable rather than as a capability ordering. Given the wide spread of 23.4 points, this suggests that rank differences between models likely reflect genuine capability gaps rather than minor variations.
3 cited facts
A C grade at 59, ranked 50 of 51, leaves this benchmark next to last in the tracked set — treat results here as directional context, not evidence strong enough to settle a comparison between models. Saturation is the weakest component of integrity, causing the ceiling to be crowded and limiting the benchmark's ability to discriminate between frontier models. This benchmark's health as a discriminator under a disclosed harness is compromised by high saturation, which clusters scores near the ceiling and reduces differentiation power.
7 cited facts
Most of the scale sits above the best result: 37.15 at the top with 62.85 points of headroom. Movement at the top here is signal — a benchmark this far from its ceiling separates models on measured results, not ceiling effects. A cluster of 3 models near the top score suggests performance differences within a couple of points may not reflect substantial capability gaps, aligning with the non-saturated status. The non-saturated benchmark status implies that current top-tier rankings are not noise-dominated, allowing for meaningful differentiation among models.
5 cited facts
Gaps between models here reflect the models, not run-to-run differences in setup; the useful corollary is that the ordering can be trusted even where the absolute numbers mean little on their own. The test set is public, so high scores merit careful scrutiny for possible contamination. Contamination history shows the data has been public since 2019 and appears in common pretraining corpora, indicating sensitivity to data leakage.
3 cited facts
ANLI (Adversarial Natural Language Inference): Adversarial Natural Language Inference — a 3-way (entailment/neutral/contradiction) NLI benchmark built over three human-and-model-in-the-loop rounds (R1-R3) of escalating difficulty; scored as classification accuracy.
phi-3-small 7.4B leads ANLI at 37.15 — self-reported by the lab. The full leaderboard above lists every recorded measurement, not just the headline number.
9 models have recorded scores on ANLI, spanning a score spread of 23.4.
tensor.news grades ANLI C for integrity (score 59/100), ranking #60 of 61 benchmarks we assess across discrimination, saturation, contamination resistance, harness comparability, and freshness.
No — ANLI still has headroom and continues to discriminate between models rather than bunching them at the ceiling.
Its test set is public, so contamination risk is on the table: a high score may partly reflect training-data overlap rather than capability.
Scores are reported under a consistent harness, so comparisons on ANLI are reasonably apples-to-apples.
Every ANLI measurement is source-backed and tagged reproduced, self-reported, or unverified; the underlying record cites arxiv.org/abs/1910.14599.