research
∙
02/01/2023
TAP: Accelerating Large-Scale DNN Training Through Tensor Automatic Parallelisation
Model parallelism has become necessary to train large neural networks. H...
share
research
∙
08/30/2022
EasyScale: Accuracy-consistent Elastic Training for Deep Learning
Distributed synchronized GPU training is commonly used for deep learning...
research
∙
10/08/2021
M6-10T: A Sharing-Delinking Paradigm for Efficient Multi-Trillion Parameter Pretraining
Recent expeditious developments in deep learning algorithms, distributed...
research
∙
05/31/2021
Exploring Sparse Expert Models and Beyond
Mixture-of-Experts (MoE) models can achieve promising results with outra...
research
∙
03/01/2021
M6: A Chinese Multimodal Pretrainer
In this work, we construct the largest dataset for multimodal pretrainin...
research
∙
11/18/2020
Whale: A Unified Distributed Training Framework
Data parallelism (DP) has been a common practice to speed up the trainin...
research
∙
07/30/2018
Highly Scalable Deep Learning Training System with Mixed-Precision: Training ImageNet in Four Minutes
Synchronized stochastic gradient descent (SGD) optimizers with data para...
research
∙
04/16/2018
BigDL: A Distributed Deep Learning Framework for Big Data
In this paper, we present BigDL, a distributed deep learning framework f...
research
∙
11/29/2016