bigscience·released 2022-10-081 source
bloomz-560m benchmark scores: 53 benchmarks tracked, leading 51. 0% of its scores are independently reproduced — the rest are self-reported or unverified.
Leads on 51 of 51 benchmarks·best result 67.15 on SuperGLUE (rte) (unverified)·0 of 53 independently reproduced
Leads on: ANLI (r1), ANLI (r2), ANLI (r3), HumanEval, StoryCloze (2016), SuperGLUE (cb), SuperGLUE (copa), SuperGLUE (rte), Winogrande XL (xl), XCOPA (et), XCOPA (ht), XCOPA (id), XCOPA (it), XCOPA (qu), XCOPA (sw), XCOPA (ta), XCOPA (th), XCOPA (tr), XCOPA (vi), XCOPA (zh), XNLI (ar), XNLI (bg), XNLI (de), XNLI (el), XNLI (en), XNLI (es), XNLI (fr), XNLI (hi), XNLI (ru), XNLI (sw), XNLI (th), XNLI (tr), XNLI (ur), XNLI (vi), XNLI (zh), XStoryCloze (ar), XStoryCloze (es), XStoryCloze (eu), XStoryCloze (hi), XStoryCloze (id), XStoryCloze (my), XStoryCloze (ru), XStoryCloze (sw), XStoryCloze (te), XStoryCloze (zh), XWinograd (en), XWinograd (fr), XWinograd (jp), XWinograd (pt), XWinograd (ru), XWinograd (zh)
| Benchmark | Score | Evidence status | Measured |
|---|---|---|---|
| SuperGLUE (rte) | 67.15Accuracy | unverifiedT2 | — |
| XCOPA (tr) | 61Accuracy | unverifiedT2 | — |
| XCOPA (vi) | 61Accuracy | unverifiedT2 | — |
| XCOPA (zh) | 61Accuracy | unverifiedT2 | — |
| StoryCloze (2016) | 60.29Accuracy | unverifiedT2 | — |
| XCOPA (ta) | 58Accuracy | unverifiedT2 | — |
| XCOPA (th) | 58Accuracy | unverifiedT2 | — |
| XStoryCloze (id) | 57.84Accuracy | unverifiedT2 | — |
| XCOPA (id) | 57Accuracy | unverifiedT2 | — |
| XStoryCloze (zh) | 56.52Accuracy | unverifiedT2 | — |
| XStoryCloze (es) | 56.45Accuracy | unverifiedT2 | — |
| XCOPA (sw) | 56Accuracy | unverifiedT2 | — |
| XStoryCloze (hi) | 55.79Accuracy | unverifiedT2 | — |
| XCOPA (qu) | 55Accuracy | unverifiedT2 | — |
| XStoryCloze (te) | 54.86Accuracy | unverifiedT2 | — |
| XWinograd (zh) | 54.76Accuracy | unverifiedT2 | — |
| XStoryCloze (ar) | 54.4Accuracy | unverifiedT2 | — |
| XWinograd (pt) | 53.99Accuracy | unverifiedT2 | — |
| XWinograd (ru) | 53.97Accuracy | unverifiedT2 | — |
| SuperGLUE (cb) | 53.57Accuracy | unverifiedT2 | — |
| XStoryCloze (ru) | 53.14Accuracy | unverifiedT2 | — |
| XCOPA (et) | 53Accuracy | unverifiedT2 | — |
| Winogrande XL (xl) | 52.41Accuracy | unverifiedT2 | — |
| XWinograd (jp) | 52.03Accuracy | unverifiedT2 | — |
| SuperGLUE (copa) | 52Accuracy | unverifiedT2 | — |
| XCOPA (it) | 52Accuracy | unverifiedT2 | — |
| XWinograd (fr) | 51.81Accuracy | unverifiedT2 | — |
| XStoryCloze (sw) | 51.36Accuracy | unverifiedT2 | — |
| XWinograd (en) | 51.01Accuracy | unverifiedT2 | — |
| XStoryCloze (eu) | 50.56Accuracy | unverifiedT2 | — |
| XCOPA (ht) | 49Accuracy | unverifiedT2 | — |
| XStoryCloze (my) | 47.05Accuracy | unverifiedT2 | — |
| XNLI (en) | 46.43Accuracy | unverifiedT2 | — |
| XNLI (fr) | 45.54Accuracy | unverifiedT2 | — |
| XNLI (es) | 44.98Accuracy | unverifiedT2 | — |
| XNLI (vi) | 44.74Accuracy | unverifiedT2 | — |
| XNLI (zh) | 44.66Accuracy | unverifiedT2 | — |
| XNLI (ar) | 44.46Accuracy | unverifiedT2 | — |
| XNLI (hi) | 41.81Accuracy | unverifiedT2 | — |
| XNLI (el) | 40.96Accuracy | unverifiedT2 | — |
| XNLI (bg) | 39.76Accuracy | unverifiedT2 | — |
| XNLI (ru) | 39.64Accuracy | unverifiedT2 | — |
| XNLI (de) | 39.36Accuracy | unverifiedT2 | — |
| XNLI (ur) | 38.96Accuracy | unverifiedT2 | — |
| XNLI (sw) | 38.35Accuracy | unverifiedT2 | — |
| XNLI (tr) | 37.31Accuracy | unverifiedT2 | — |
| XNLI (th) | 35.5Accuracy | unverifiedT2 | — |
| ANLI (r3) | 33.5Accuracy | unverifiedT2 | — |
| ANLI (r1) | 33.4Accuracy | unverifiedT2 | — |
| ANLI (r2) | 33.4Accuracy | unverifiedT2 | — |
| HumanEval | 9Pass@100 | unverifiedT2 | — |
| HumanEval | 4.11Pass@10 | unverifiedT2 | — |
| HumanEval | 2.18Pass@1 | unverifiedT2 | — |
Claims drawn from cited facts, not live model generation.
This model, developed by BigScience, is a compact 0.56-billion-parameter system released in 2022, with the efficiency and lower cost typical of smaller models rather than the headroom of frontier-scale systems.
3 cited facts
This model is tracked across 51 benchmarks and currently holds the top score on all 51 of them. Because these results are unverified by outside evaluation, they should be read as untested claims rather than confirmed facts.
3 cited facts
With an average gap of 0.0 points to the state of the art under the disclosed harness, it sits at the frontier. It also holds the top score on 51 benchmarks.
2 cited facts
Benchmarks with more than one recorded measurement for this model — every one shown, not just the headline number.
2 facts cross-checked across data sources: 2 single-source.
Source facts, citations, and refresh stamp for this record.
Sources: huggingface_models
bloomz-560m is an AI model developed by bigscience, released 2022-10-08. Its benchmark record below tags every score as independently reproduced, self-reported, or unverified.
bloomz-560m has recorded scores on 51 benchmarks, each shown with its evidence status.
bloomz-560m has recorded scores on 53 benchmarks — Winogrande XL (xl), XWinograd (en), XWinograd (fr), XWinograd (jp), XWinograd (pt), XWinograd (ru), and 47 more. The full table above shows each score with its evidence status.
0 of 53 recorded scores (0%) are independently reproduced rather than self-reported by the lab.
bloomz-560m has 51 tracked claims: 51 unverified.