research
∙
03/13/2023
FlexGen: High-Throughput Generative Inference of Large Language Models with a Single GPU
The high computational and memory requirements of large language model (...
share
research
∙
02/09/2023
Is This Loss Informative? Speeding Up Textual Inversion with Deterministic Objective Evaluation
Text-to-image generation models represent the next step of evolution in ...
research
∙
01/27/2023
SWARM Parallelism: Training Large Models Can Be Surprisingly Communication-Efficient
Many deep learning applications benefit from using large models with bil...
research
∙
10/23/2022
RuCoLA: Russian Corpus of Linguistic Acceptability
Linguistic acceptability (LA) attracts the attention of the research com...
research
∙
09/02/2022
Petals: Collaborative Inference and Fine-tuning of Large Models
Many NLP tasks benefit from using large language models (LLMs) that ofte...
research
∙
07/11/2022
Towards Crowdsourced Training of Large Neural Networks using Decentralized Mixture-of-Experts
Many recent breakthroughs in deep learning were achieved by training inc...
research
∙
07/07/2022
Training Transformers Together
The infrastructure necessary for training state-of-the-art models is bec...
research
∙
06/22/2021
It's All in the Heads: Using Attention Heads as a Baseline for Cross-Lingual Transfer in Commonsense Reasoning
Commonsense reasoning is one of the key problems in natural language pro...
research
∙
06/21/2021
Secure Distributed Training at Scale
Some of the hardest problems in deep learning can be solved with the com...
research
∙
06/18/2021
Distributed Deep Learning in Open Collaborations
Modern deep learning applications require increasingly more compute to t...
research
∙
05/14/2021
Scaling Ensemble Distribution Distillation to Many Classes with Proxy Targets
Ensembles of machine learning models yield improved system performance a...
research
∙
03/04/2021
Moshpit SGD: Communication-Efficient Decentralized Training on Heterogeneous Unreliable Devices
Training deep neural networks on large datasets can often be accelerated...
research
∙
10/06/2020