Claim · 2019-07-26 · publication · Stated in source · Supported

The authors state that BERT was significantly undertrained and propose a training recipe, RoBERTa, which they report can match or exceed post-BERT methods.

One recorded claim. The claim panel gives its qualification, the source passages with their place in each source, and how it was checked.

What it connects

RoBERTa: A Robustly Optimized BERT Pretraining Approach → states its contribution → The authors state that BERT was significantly undertrained and propose a training recipe, RoBERTa, which they report can match or exceed post-BERT methods.

Where it appears

No collection page holds this claim. It is listed with its records in the explorer.

Claim and evidence

RoBERTa: A Robustly Optimized BERT Pretraining Approachstates its contribution ↓The authors state that BERT was significantly undertrained and propose a training recipe, RoBERTa, which they report can match or exceed post-BERT methods.

Machine-checked, not reviewed by an editorStated in sourceSupported

Supporting evidence: primary · 1 source

Date: 2019-07-26 · publication Records dated 2019 →

Scope and limitsThe paper's own description of its contribution; not an assessment of it.

The evidence 01

primary sourceSupports the claim

RoBERTa: A Robustly Optimized BERT Pretraining Approach

arXiv · 2019-07-26

Cite the original source

Cite the original document, not Selekoa. Check these details against it.

Source excerpt

“We find that BERT was significantly undertrained and propose an improved recipe for training BERT models, which we call RoBERTa, that can match or exceed the performance of all of the post-BERT methods.”

Location in source arXiv v1, PDF p. 1, §1 Introduction, para. 2

Machine-checked · 2026-10-31

Source details and check notes

Authors: Yinhan Liu; Myle Ott; Naman Goyal; Jingfei Du; Mandar Joshi; Danqi Chen; Omer Levy; Mike Lewis; Luke Zettlemoyer; Veselin Stoyanov
Version: arXiv:1907.11692v1, submitted 26 July 2019 (UTC). Retrieved for the candidate pack on 2026-10-28 (SHA-256 76a3872d244793563a5b000b818cbaf0ca8972ab1145d32be474c05b2a8f3070). The author list is the byline as printed on the first page.
Version date: 2019-07-26
Canonical address: Open the canonical source ↗
Accessed: 2026-10-28
Language: English (en)
Attribution: Yinhan Liu, Myle Ott, Naman Goyal, Jingfei Du, Mandar Joshi, Danqi Chen, Omer Levy, Mike Lewis, Luke Zettlemoyer, Veselin Stoyanov, “RoBERTa: A Robustly Optimized BERT Pretraining Approach”, 2019-07-26.
Reuse: No open redistribution licence asserted. Bibliographic metadata, editorial paraphrases and short attributed quotations (at most 50 words each). Recheck terms before wider redistribution.
Check notes: Candidate pack passage, key papers (research/key-papers). The quotation (33 words) matched the text retrieved for the candidate pack on 2026-10-28 (research/key-papers/verification.json). Matching ignored differences of whitespace, case, quotation marks, dashes and line-break hyphens, so the source’s typography can differ. A cross-family machine check (2026-10-30, item B034) found the quotation at its locator. No human editorial review.

Status and meaning

Editorial status. Not reviewed by an editor. An exploratory record that no editor has accepted. This status does not say whether a person has read any linked passage. Each evidence record above shows how it was checked.

Relation. A paper or other publication describes its own contribution, quoted from the expression the record names, usually its abstract or introduction. The record is the document’s own description, not an assessment: a claim of priority or novelty it contains is the document’s claim, flagged as such, and says nothing about earlier or later work.

Date. Date of the arXiv version quoted, submitted 26 July 2019 (UTC); later versions may word it differently.

Record ID kp-states-roberta-robustly-optimized-bert-pretraining-2019-2