Document · Research paper

Megatron-LM: Training Multi-Billion Parameter Language Models Using Model Parallelism

Work-level record. Its arXiv versions and any conference or journal version are listed as sources, each with its own date. A citation to the work does not identify which version the citing author read.

Issued by

No issuing record yet. A missing record is a gap in coverage.

Authors credited

No authorship claim recorded yet. The source records list arXiv, Mohammad Shoeybi, Mostofa Patwary, Raul Puri, Patrick LeGresley, Jared Casper and Bryan Catanzaro.

Dates

No date claim recorded yet. The source records give the publication date as 2019-09-17.

Exact source

Megatron-LM: Training Multi-Billion Parameter Language Models Using Model Parallelism arXiv · published 2019-09-17 · arXiv abstract page, retrieved for the candidate pack on 2026-10-28 (SHA-256 2e24100c037c6fe4475fff00d4a35bcbdc4de43371199609b972f69c769a86e1). Submission history as arXiv records it (UTC): v1 17 September 2019; v2 19 September 2019; v3 5 October 2019; v4 13 March 2020. The page serves the latest version; each version is a separate expression, and only the one a record names was read.
Megatron-LM: Training Multi-Billion Parameter Language Models Using Model Parallelism arXiv · published 2019-09-17 · arXiv:1909.08053v1, submitted 17 September 2019 (UTC). Retrieved for the candidate pack on 2026-10-28 (SHA-256 014b8a8d3cec8bee5e371b23e4f49d4e74597641febc36992faf33d8834f8fe2). The author list is the byline as printed on the first page.

Appears in

No collection holds a record about this document yet.

Versions

No other versions of this document are recorded. The evidence applies to the source version listed above. Other versions, such as a preprint, working paper or journal publication, are not identified or substituted.

Documentary relations

Each relation is listed separately. Citing, crediting and reporting the use of a method are different acts, and none of them shows agreement or influence by itself.

Cited by · 6

Recorded documents that explicitly reference this one. A citation to a work does not identify which version was read.

Not recorded for this document: cites · explicitly credits · reports using a method from · method reported as used by · applies parts of · parts applied by · discusses · reports evaluation of. A missing record does not show that the relation does not exist.

What the document says it adds

The document’s own description of its contribution, quoted from the version each row names. A claim to be the first is the document’s claim; Selekoa does not assess it.

In its own wordsThe paper presents an intra-layer model-parallel approach that the authors state enables training transformer language models with billions of parameters.“In this work, we implement a simple, efficient intra-layer model parallel approach that enables training state of the art transformer language models with billions of parameters.” arXiv v1, PDF p. 1, abstract2019-09-17 · publication · Stated in source · Supported Inspect claim & evidence ↗