research
∙
02/06/2023
Colossal-Auto: Unified Automation of Parallelization and Activation Checkpoint for Large-scale Models
In recent years, large-scale models have demonstrated state-of-the-art p...
share
research
∙
12/10/2022
Elixir: Train a Large Language Model on a Small GPU Cluster
In recent years, the number of parameters of one deep learning (DL) mode...
research
∙
09/06/2022
EnergonAI: An Inference System for 10-100 Billion Parameter Transformer Models
Large transformer models display promising performance on a wide range o...
research
∙
08/08/2022
A Frequency-aware Software Cache for Large Recommendation System Embeddings
Deep learning recommendation models (DLRMs) have been widely applied in ...
research
∙
08/12/2021
PatrickStar: Parallel Training of Pre-trained Models via a Chunk-based Memory Management
The pre-trained model (PTM) is revolutionizing Artificial intelligence (...
research
∙
10/09/2020
TurboTransformers: An Efficient GPU Serving System For Transformer Models
The transformer is the most critical algorithm innovation of the Nature ...
research
∙
03/16/2019
swCaffe: a Parallel Framework for Accelerating Deep Learning Applications on Sunway TaihuLight
This paper reports our efforts on swCaffe, a highly efficient parallel f...
research
∙
08/13/2018