Technology Innovation Institute·released 2023-03-151 source
Falcon-40B benchmark scores: 10 benchmarks tracked. 0% of its scores are independently reproduced — the rest are self-reported or unverified.
Leads on 0 of 10 benchmarks·best result 80.37 on HellaSwag (self-reported)·0 of 10 independently reproduced
| Benchmark | Score | Evidence status | Measured |
|---|---|---|---|
| HellaSwag | 80.37accuracy (%) | self-reported· optimizedT1 | 2023-03-15Falcon2-11B Technical Report |
| TriviaQA | 79.9accuracy (%) | unverified· optimizedT1 | 2023-03-15Llama 2: Open Foundation and Fine-Tuned Chat Models |
| LAMBADA | 77.3accuracy (%) | unverified· optimizedT1 | 2023-03-15The Falcon Series of Open Language Models |
| PIQA | 66accuracy (%) | unverified· optimizedT1 | 2023-03-15Llama 2: Open Foundation and Fine-Tuned Chat Models |
| Winogrande | 53.8accuracy (%) | unverified· optimizedT1 | 2023-03-15Llama 2: Open Foundation and Fine-Tuned Chat Models |
| ARC AI2 | 49.15accuracy (%) | unverified· optimizedT1 | 2023-03-15The Falcon Series of Open Language Models |
| MMLU | 42.53accuracy (%) | self-reported· optimizedT1 | 2023-03-15Falcon2-11B Technical ReportFalcon2-11B Technical Report |
| OpenBookQA | 42.13accuracy (%) | unverified· optimizedT1 | 2023-03-15Llama 2: Open Foundation and Fine-Tuned Chat Models |
| GSM8K | 33.8accuracy (%) | unverified· optimizedT1 | 2023-05-25Stanford HELM |
| BBH | 16.13accuracy (%) | self-reported· optimizedT1 | 2023-03-15Qwen Technical Report |
Claims drawn from cited facts, not live model generation.
Developed by Technology Innovation Institute, this model is a frontier-scale model with 40 billion parameters and a 2023 release. That scale implies ample headroom but also significant computational expense, unlike the efficiency and lower cost of a compact model.
4 cited facts
This model is tracked across 10 benchmarks and currently holds no top score on any of them. These figures are self-reported, so they should be read as vendor claims rather than independently verified results.
3 cited facts
With an average gap of 31.7 points behind the leader, this model sits well behind the front of the pack on its benchmarks rather than at the frontier. It holds the top score on none of the tracked sota leader count benchmarks, so it shows no genuine category leadership.
2 cited facts
2 facts cross-checked across data sources: 2 single-source.
Source facts, citations, and refresh stamp for this record.
Sources: curated_param_counts, epoch_benchmarks
Falcon-40B is an AI model developed by Technology Innovation Institute, released 2023-03-15. Its benchmark record below tags every score as independently reproduced, self-reported, or unverified.
Falcon-40B has recorded scores on 10 benchmarks, each shown with its evidence status.
Falcon-40B has recorded scores on 10 benchmarks — PIQA, OpenBookQA, Winogrande, TriviaQA, MMLU, ARC AI2, and 4 more. The full table above shows each score with its evidence status.
0 of 10 recorded scores (0%) are independently reproduced rather than self-reported by the lab.
Falcon-40B has 10 tracked claims: 3 self-reported, 7 unverified.