OpenBookQA
Rank #49 of 51 · 33 models scored · top score 84
unknown
Strongest on discrimination, weakest on contamination resistance. Caveat: scores come from mixed harnesses, so a head-to-head is not apples-to-apples.
Weighted composite. Each signal is computed from the source-backed leaderboard below.
Does it still separate models?
How far from ceiling / clustered at the top?
Public vs held-out; training-leak risk.
Apples-to-apples, or mixed / vendor-optimized?
How old is the benchmark?
| # | Model | Score | Evidence | Measured |
|---|---|---|---|---|
| 1 | phi-3-mini 3.8B | 84 | self-reported· optimized | 2024-04-23 |
| 2 | phi-3-small 7.4B | 84 | self-reported· optimized | 2024-04-23 |
| 3 | phi-3-medium 14B | 83.2 | self-reported· optimized | 2024-04-23 |
| 4 | GPT-3.5 Turbo (Nov 2023) | 81.33 | self-reported· optimized | 2023-06-13 |
| 5 | Mixtral 8x7B | 81.07 | self-reported· optimized | 2023-12-11 |
| 6 | Llama 3-8B | 76.8 | self-reported· optimized | 2024-04-18 |
| 7 | Mistral 7B v0.1 | 73.07 | self-reported· optimized | 2023-10-10 |
| 8 | Gemma 7B | 71.47 | self-reported· optimized | 2024-02-21 |
| 9 | Phi-2 | 64.8 | self-reported· optimized | 2023-12-12 |
| 10 | Falcon-180B | 52.27 | unverified· optimized | 2023-09-06 |
| 11 | LLaMA-65B | 46.93 | unverified· optimized | 2023-02-24 |
| 12 | Llama 2-70B | 46.93 | unverified· optimized | 2023-07-18 |
| 13 | LLaMA-33B | 44.8 | unverified· optimized | 2023-02-27 |
| 14 | Llama 2-7B | 44.8 | unverified· optimized | 2023-07-18 |
| 15 | Llama 2-34B | 44.27 | unverified· optimized | 2023-07-18 |
| 16 | PaLM 2-M | 43.2 | self-reported· optimized | 2023-05-17 |
| 17 | LLaMA-7B | 42.93 | unverified· optimized | 2023-02-24 |
| 18 | Llama 2-13B | 42.67 | unverified· optimized | 2023-07-18 |
| 19 | Falcon-40B | 42.13 | unverified· optimized | 2023-03-15 |
| 20 | LLaMA-13B | 41.87 | unverified· optimized | 2023-02-27 |
| 21 | PaLM 2-S | 41.6 | self-reported· optimized | 2023-05-17 |
| 22 | PowerMoE-3b | 41 | unverified | — |
| 23 | MPT-30B | 36 | unverified· optimized | 2023-06-22 |
| 24 | Falcon-7B | 35.47 | unverified· optimized | 2023-04-24 |
| 25 | MPT-7B | 35.2 | unverified· optimized | 2023-05-05 |
| 26 | XGen-7B | 20.27 | self-reported· optimized | 2023-09-07 |
| 27 | RedPajama-INCITE-7B-Base | 20 | self-reported· optimized | 2023-05-04 |
| 28 | Dolly 2.0-12b | 18.93 | self-reported· optimized | 2023-04-12 |
| 29 | open_llama_7b | 18.67 | self-reported· optimized | 2023-06-07 |
| 30 | Phi-1.5 | 16.27 | self-reported· optimized | 2023-09-11 |
| 31 | Cerebras-GPT-13B | 14.4 | self-reported· optimized | 2023-04-06 |
| 32 | vicuna-13b-v1.1 | 10.67 | self-reported· optimized | 2023-04-12 |
| 33 | stablelm-tuned-alpha-7b | 9.87 | self-reported· optimized | 2023-04-19 |