ZeRO: Memory optimizations Toward Training Trillion Parameter Models
Work-level record. Its arXiv versions and any conference or journal version are listed as sources, each with its own date. A citation to the work does not identify which version the citing author read.
- Issued by
- No issuing record yet. A missing record is a gap in coverage.
- Authors credited
- No authorship claim recorded yet. The source records list arXiv, Crossref, Samyam Rajbhandari, Jeff Rasley, Olatunji Ruwase and Yuxiong He.
- Dates
- No date claim recorded yet. The source records give the publication date as 2019-10-04, 2020 and 2019-10-07.
- Exact source
- ZeRO: Memory optimizations Toward Training Trillion Parameter Models arXiv · published 2019-10-04 · arXiv abstract page, retrieved for the candidate pack on 2026-10-28 (SHA-256 0a796e197fe5f8a956a7d4e7e910d600bc507450aedec3462b8ce574cd93df6e). Submission history as arXiv records it (UTC): v1 4 October 2019; v2 7 October 2019; v3 13 May 2020. The page serves the latest version; each version is a separate expression, and only the one a record names was read.
- ZeRO: Memory optimizations Toward Training Trillion Parameter Models Crossref · published 2020 · Crossref record of the conference version: SC20: International Conference for High Performance Computing, Networking, Storage and Analysis, pp. 1-16 (DOI 10.1109/sc41405.2020.00024), retrieved for the candidate pack on 2026-10-28. The date is the publisher-deposited date, kept at year precision; it was not checked against the issue.
- ZeRO: Memory optimizations Toward Training Trillion Parameter Models arXiv · published 2019-10-07 · arXiv:1910.02054v2, submitted 7 October 2019 (UTC). Retrieved for the candidate pack on 2026-10-28 (SHA-256 fe2991f241e807b1b90366e07ee4dab8f474b1449d32d446157a93886421bf1b). The author list is the byline as printed on the first page. arXiv lists v1 (4 October 2019) as withdrawn and serves no PDF for it; the text read is v2 (7 October 2019).
Appears in
No collection holds a record about this document yet.
Versions
No other versions of this document are recorded. The evidence applies to the source version listed above. Other versions, such as a preprint, working paper or journal publication, are not identified or substituted.
Documentary relations
Each relation is listed separately. Citing, crediting and reporting the use of a method are different acts, and none of them shows agreement or influence by itself.
Cited by · 7
Recorded documents that explicitly reference this one. A citation to a work does not identify which version was read.
- ERNIE 3.0: Large-scale Knowledge Enhanced Pre-training for Language Understanding and GenerationERNIE 3.0: Large-scale Knowledge Enhanced Pre-training for Language Understanding and Generation cites ZeRO: Memory optimizations Toward Training Trillion Parameter Models.2021-07-05 · publication · Stated in source · Supported Inspect claim & evidence ↗ERNIE 3.0: Large-scale Knowledge Enhanced Pre-training for Language Understanding and Generation: document page →
- ERNIE 3.0 Titan: Exploring Larger-scale Knowledge Enhanced Pre-training for Language Understanding and GenerationERNIE 3.0 Titan: Exploring Larger-scale Knowledge Enhanced Pre-training for Language Understanding and Generation cites ZeRO: Memory optimizations Toward Training Trillion Parameter Models.2021-12-23 · publication · Stated in source · Supported Inspect claim & evidence ↗ERNIE 3.0 Titan: Exploring Larger-scale Knowledge Enhanced Pre-training for Language Understanding and Generation: document page →
- Training Compute-Optimal Large Language ModelsTraining Compute-Optimal Large Language Models cites ZeRO: Memory optimizations Toward Training Trillion Parameter Models.2022-03-29 · publication · Stated in source · Supported Inspect claim & evidence ↗Training Compute-Optimal Large Language Models: document page →
- DeepSeek-V3 Technical ReportDeepSeek-V3 Technical Report cites ZeRO: Memory optimizations Toward Training Trillion Parameter Models.2024-12-27 · publication · Stated in source · Supported Inspect claim & evidence ↗DeepSeek-V3 Technical Report: document page →
- Kimi K2: Open Agentic IntelligenceKimi K2: Open Agentic Intelligence cites ZeRO: Memory optimizations Toward Training Trillion Parameter Models.2025-07-28 · publication · Stated in source · Supported Inspect claim & evidence ↗Kimi K2: Open Agentic Intelligence: document page →
- ERNIE 5.0 Technical ReportERNIE 5.0 Technical Report cites ZeRO: Memory optimizations Toward Training Trillion Parameter Models.2026-02-04 · publication · Stated in source · Supported Inspect claim & evidence ↗ERNIE 5.0 Technical Report: document page →
- Kimi K3: Open Frontier IntelligenceKimi K3: Open Frontier Intelligence cites ZeRO: Memory optimizations Toward Training Trillion Parameter Models.2026-07-28 · publication · Stated in source · Supported Inspect claim & evidence ↗Kimi K3: Open Frontier Intelligence: document page →
Not recorded for this document: cites · explicitly credits · reports using a method from · method reported as used by · applies parts of · parts applied by · discusses · reports evaluation of. A missing record does not show that the relation does not exist.
What the document says it adds
The document’s own description of its contribution, quoted from the version each row names. A claim to be the first is the document’s claim; Selekoa does not assess it.