research
∙
12/13/2021
GLaM: Efficient Scaling of Language Models with Mixture-of-Experts
Scaling language models with more data, compute and parameters has drive...
share
research
∙
05/10/2021
GSPMD: General and Scalable Parallelization for ML Computation Graphs
We present GSPMD, an automatic, compiler-based parallelization system fo...
research
∙
06/30/2020
GShard: Scaling Giant Models with Conditional Computation and Automatic Sharding
Neural network scaling has been critical for improving the model quality...
research
∙
07/11/2019
Massively Multilingual Neural Machine Translation in the Wild: Findings and Challenges
We introduce our efforts towards building a universal neural machine tra...
research
∙
02/21/2019