The paper presents an intra-layer model-parallel approach that the authors state enables training transformer language models with billions of parameters.
One recorded claim. The claim panel gives its qualification, the source passages with their place in each source, and how it was checked.
What it connects
Megatron-LM: Training Multi-Billion Parameter Language Models Using Model Parallelism → states its contribution → The paper presents an intra-layer model-parallel approach that the authors state enables training transformer language models with billions of parameters.
Where it appears
No collection page holds this claim. It is listed with its records in the explorer.