research
∙
05/09/2023
StarCoder: may the source be with you!
The BigCode community, an open-scientific collaboration working on the r...
share
research
∙
01/09/2023
SantaCoder: don't reach for the stars!
The BigCode project is an open-scientific collaboration working on the r...
research
∙
11/11/2022
Breadth-First Pipeline Parallelism
We introduce Breadth-First Pipeline Parallelism, a novel training schedu...
research
∙
06/04/2021
Layered gradient accumulation and modular pipeline parallelism: fast and efficient training of large language models
The advent of the transformer has sparked a quick growth in the size of ...
research
∙
12/15/2018