research
∙
02/01/2023
TAP: Accelerating Large-Scale DNN Training Through Tensor Automatic Parallelisation
Model parallelism has become necessary to train large neural networks. H...
share
research
∙
10/11/2022
Revisiting and Advancing Chinese Natural Language Understanding with Accelerated Heterogeneous Knowledge Pre-training
Recently, knowledge-enhanced pre-trained language models (KEPLMs) improv...
research
∙
10/08/2021
M6-10T: A Sharing-Delinking Paradigm for Efficient Multi-Trillion Parameter Pretraining
Recent expeditious developments in deep learning algorithms, distributed...
research
∙
05/31/2021
Exploring Sparse Expert Models and Beyond
Mixture-of-Experts (MoE) models can achieve promising results with outra...
research
∙
03/01/2021
M6: A Chinese Multimodal Pretrainer
In this work, we construct the largest dataset for multimodal pretrainin...
research
∙
11/18/2020
EasyTransfer – A Simple and Scalable Deep Transfer Learning Platform for NLP Applications
The literature has witnessed the success of applying deep Transfer Learn...
research
∙
11/18/2020