2020-12-01 · article · Topic Modeling · en
highly citedfrom the source record
Transformer-based models have pushed state of the art in many areas of NLP, but our understanding of what is behind their success is still limited. This paper is the first survey of over 150 studies of the popular BERT model. We review the current state of knowledge about how BERT works, what kind of information it learns and how it is represented, common modifications to its training objectives and architecture, the overparameterization issue, and approaches to compression. We then outline directions for future research.
Claims drawn from cited facts, not live model generation.
This paper is a published article on topic modeling, as indicated by its title and designated topic. Because it is an article rather than a preprint, its claims carry the weight of a work that has cleared peer review.
4 cited facts
This work was produced by a small team of 3 authors.
1 cited fact
With 1544 citations, it is already an established reference point in its field. It references 182 works.
3 cited facts
A Primer in BERTology: What We Know About How BERT Works is an article, focusing on Topic Modeling, published in 2020.
A Primer in BERTology: What We Know About How BERT Works has 1,544 citations in our record, placing it among the highly cited works we track.
Our record lists 3 authors for A Primer in BERTology: What We Know About How BERT Works and references 182 other works. Author names aren't in our data yet — the source link on the page lists the full byline.
Source facts, citations, and refresh stamp for this record.
Sources: openalex_works