ARC AI2
Rank #45 of 51 · 66 models scored · top score 93.73
unknown
Strongest on discrimination, weakest on contamination resistance. Caveat: scores come from mixed harnesses, so a head-to-head is not apples-to-apples.
Weighted composite. Each signal is computed from the source-backed leaderboard below.
Does it still separate models?
How far from ceiling / clustered at the top?
Public vs held-out; training-leak risk.
Apples-to-apples, or mixed / vendor-optimized?
How old is the benchmark?
| # | Model | Score | Evidence | Measured |
|---|---|---|---|---|
| 1 | Llama 3.1-405B | 93.73 | self-reported· optimized | 2024-07-23 |
| 2 | DeepSeek-V3 | 93.73 | self-reported· optimized | 2024-12-24 |
| 3 | Qwen2.5-72B | 92.67 | self-reported· optimized | 2024-09-19 |
| 4 | DeepSeek-V2 (MoE-236B, May 2024) | 89.6 | self-reported· optimized | 2024-05-07 |
| 5 | phi-3-medium 14B | 88.8 | self-reported· optimized | 2024-04-23 |
| 6 | phi-3-small 7.4B | 87.6 | self-reported· optimized | 2024-04-23 |
| 7 | GPT-3.5 Turbo (Nov 2023) | 83.2 | self-reported· optimized | 2023-06-13 |
| 8 | Mixtral 8x7B | 83.07 | unverified· optimized | 2023-12-11 |
| 9 | Claude Instant | 81.73 | unverified· optimized | 2023-08-09 |
| 10 | Stable Beluga 2 | 81.47 | self-reported· optimized | 2023-07-20 |
| 11 | phi-3-mini 3.8B | 79.87 | self-reported· optimized | 2024-04-23 |
| 12 | Qwen-14B | 79.2 | self-reported· optimized | 2023-09-24 |
| 13 | Llama 3-8B | 77.07 | self-reported· optimized | 2024-04-18 |
| 14 | internlm-20b | 75.6 | self-reported· optimized | 2023-09-18 |
| 15 | Mistral 7B v0.1 | 71.47 | self-reported· optimized | 2023-10-10 |
| 16 | Gemma 7B | 71.07 | self-reported· optimized | 2024-02-21 |
| 17 | Llama 2-70B | 71.07 | unverified· optimized | 2023-07-18 |
| 18 | Phi-2 | 67.87 | self-reported· optimized | 2023-12-12 |
| 19 | Qwen-7B | 67.07 | self-reported· optimized | 2023-09-28 |
| 20 | Qwen2.5-Coder-32B | 60.67 | self-reported· optimized | 2024-09-18 |
| 21 | LLaMA-65B | 59.33 | self-reported· optimized | 2023-02-24 |
| 22 | internlm-7b | 59.33 | self-reported· optimized | 2023-07-05 |
| 23 | PaLM 2-L | 58.93 | unverified· optimized | 2023-05-17 |
| 24 | Falcon-180B | 57.07 | unverified· optimized | 2023-09-06 |
| 25 | LLaMA-33B | 56.67 | self-reported· optimized | 2023-02-27 |
| 26 | Qwen2.5-Coder-14B | 54.67 | self-reported· optimized | 2024-09-18 |
| 27 | PaLM 2-M | 53.2 | unverified· optimized | 2023-05-17 |
| 28 | Falcon-40B | 49.15 | unverified· optimized | 2023-03-15 |
| 29 | chatglm2-6b | 48 | self-reported· optimized | 2023-06-24 |
| 30 | Qwen2.5-Coder (7B) | 47.87 | self-reported· optimized | 2024-09-18 |
| 31 | Llama 2-13B | 47.07 | unverified· optimized | 2023-07-18 |
| 32 | PaLM 2-S | 46.13 | unverified· optimized | 2023-05-17 |
| 33 | DeepSeek-Coder-V2-Lite-Base | 43.07 | self-reported· optimized | 2024-06-13 |
| 34 | Yi-9B | 40.8 | unverified· optimized | 2024-03-01 |
| 35 | Nemotron-4 15B | 40.67 | self-reported· optimized | 2024-02-27 |
| 36 | INTELLECT-1 | 39.36 | self-reported· optimized | 2024-11-29 |
| 37 | Llama 2-34B | 39.33 | unverified· optimized | 2023-07-18 |
| 38 | Qwen-1_8B | 37.6 | self-reported· optimized | 2023-11-30 |
| 39 | Qwen2.5-Coder-3B | 37.2 | self-reported· optimized | 2024-09-18 |
| 40 | LLaMA-13B | 36.93 | self-reported· optimized | 2023-02-27 |
| 41 | MPT-30B | 34.13 | unverified· optimized | 2023-06-22 |
| 42 | Yi 6B | 33.73 | unverified· optimized | 2023-11-02 |
| 43 | Falcon-7B | 30.48 | unverified· optimized | 2023-04-24 |
| 44 | LLaMA-7B | 30.13 | self-reported· optimized | 2023-02-24 |
| 45 | StarCoder 2 15B | 29.6 | self-reported· optimized | 2024-02-29 |
| 46 | Llama 2-7B | 27.87 | unverified· optimized | 2023-07-18 |
| 47 | Qwen2.5-Coder (1.5B) | 26.93 | self-reported· optimized | 2024-09-18 |
| 48 | Phi-1.5 | 25.87 | self-reported· optimized | 2023-09-11 |
| 49 | vicuna-13b-v1.1 | 24.27 | self-reported· optimized | 2023-04-12 |
| 50 | MPT-7B | 23.47 | self-reported· optimized | 2023-05-05 |
| 51 | DeepSeek Coder 33B | 22.93 | self-reported· optimized | 2024-01-25 |
| 52 | Gemma 2B | 22.8 | unverified· optimized | 2024-02-21 |
| 53 | XGen-7B | 21.6 | self-reported· optimized | 2023-09-07 |
| 54 | Dolly 2.0-12b | 19.47 | self-reported· optimized | 2023-04-12 |
| 55 | RedPajama-INCITE-7B-Base | 18.8 | self-reported· optimized | 2023-05-04 |
| 56 | StarCoder 2 7B | 18.27 | self-reported· optimized | 2024-02-29 |
| 57 | open_llama_7b | 18.27 | self-reported· optimized | 2023-06-07 |
| 58 | Baichuan2-13B | 17.33 | self-reported· optimized | 2023-09-06 |
| 59 | DeepSeek Coder 6.7B | 15.2 | self-reported· optimized | 2024-01-25 |
| 60 | CodeQwen1.5-7B | 14.27 | self-reported· optimized | 2024-04-15 |
| 61 | Qwen2.5-Coder-0.5B | 12.53 | self-reported· optimized | 2024-09-18 |
| 62 | StarCoder 2 3B | 12.27 | self-reported· optimized | 2024-02-29 |
| 63 | Baichuan 2-7B | 10 | self-reported· optimized | 2023-09-20 |
| 64 | Cerebras-GPT-13B | 9.87 | self-reported· optimized | 2023-04-06 |
| 65 | stablelm-tuned-alpha-7b | 2.67 | self-reported· optimized | 2023-04-19 |
| 66 | DeepSeek Coder 1.3B | 0.53 | self-reported· optimized | 2024-01-25 |