Language Models are Unsupervised Multitask Learners

Versions

No other versions of this document are recorded. The evidence applies to the source version listed above. Other versions, such as a preprint, working paper or journal publication, are not identified or substituted.

Issue and authorship records

• Author: Alec RadfordAlec Radford is a credited author of Language Models are Unsupervised Multitask Learners.Date unknown · Stated in source · Supported Inspect claim & evidence ↗Alec Radford: person page →
• Author: Dario AmodeiDario Amodei is a credited author of Language Models are Unsupervised Multitask Learners.Date unknown · Stated in source · Supported Inspect claim & evidence ↗Dario Amodei: person page →
• Author: David LuanDavid Luan is a credited author of Language Models are Unsupervised Multitask Learners.Date unknown · Stated in source · Supported Inspect claim & evidence ↗David Luan: person page →
• Author: Ilya SutskeverIlya Sutskever is a credited author of Language Models are Unsupervised Multitask Learners.Date unknown · Stated in source · Supported Inspect claim & evidence ↗Ilya Sutskever: person page →
• Author: Jeffrey WuJeffrey Wu is a credited author of Language Models are Unsupervised Multitask Learners.Date unknown · Stated in source · Supported Inspect claim & evidence ↗Jeffrey Wu: person page →
• Author: Rewon ChildRewon Child is a credited author of Language Models are Unsupervised Multitask Learners.Date unknown · Stated in source · Supported Inspect claim & evidence ↗Rewon Child: person page →

Documentary relations

Each relation is listed separately. Citing, crediting and reporting the use of a method are different acts, and none of them shows agreement or influence by itself.

Not recorded for this document: cites · explicitly credits · applies parts of · parts applied by · discusses · reports evaluation of. A missing record does not show that the relation does not exist.

What the document says it adds

The document’s own description of its contribution, quoted from the version each row names. A claim to be the first is the document’s claim; Selekoa does not assess it.

• In its own wordsThe authors state that language models begin to learn tasks such as question answering and translation without explicit supervision when trained on WebText, a new dataset of millions of webpages.“We demonstrate that language models begin to learn these tasks without any explicit supervision when trained on a new dataset of millions of webpages called WebText.” PDF p. 1, abstractDate unknown · Stated in source · Supported Inspect claim & evidence ↗
• In its own wordsThe paper describes its largest model, GPT-2, as a 1.5-billion-parameter Transformer and reports that it improves on previously reported results on 7 of 8 tested language modelling datasets in a zero-shot setting while still underfitting WebText.“Our largest model, GPT-2, is a 1.5B parameter Transformer that achieves state of the art results on 7 out of 8 tested language modeling datasets in a zero-shot setting but still underfits WebText.” PDF p. 1, abstractDate unknown · Stated in source · Supported Inspect claim & evidence ↗