research
∙
02/14/2023
Adding Instructions during Pretraining: Effective Way of Controlling Toxicity in Language Models
Pretrained large language models have become indispensable for solving v...
share
research
∙
10/25/2022
Evaluating Parameter Efficient Learning for Generation
Parameter efficient learning methods (PERMs) have recently gained signif...
research
∙
10/12/2022
Context Generation Improves Open Domain Question Answering
Closed-book question answering (QA) requires a model to directly answer ...
research
∙
06/09/2022
Factuality Enhanced Language Models for Open-Ended Text Generation
Pretrained language models (LMs) are susceptible to generate text with n...
research
∙
03/16/2022
Multi-Stage Prompting for Knowledgeable Dialogue Generation
Existing knowledge-grounded dialogue systems typically use finetuned ver...
research
∙
02/08/2022
Exploring the Limits of Domain-Adaptive Training for Detoxifying Large-Scale Language Models
Pre-trained language models (LMs) are shown to easily generate toxic lan...
research
∙
01/28/2022
Using DeepSpeed and Megatron to Train Megatron-Turing NLG 530B, A Large-Scale Generative Language Model
Pretrained general-purpose language models can achieve state-of-the-art ...
research
∙
04/09/2021
Efficient Large-Scale Language Model Training on GPU Clusters
Large language models have led to state-of-the-art accuracies across a r...
research
∙
01/02/2021
End-to-End Training of Neural Retrievers for Open-Domain Question Answering
Recent work on training neural retrievers for open-domain question answe...
research
∙
10/20/2020
Local Knowledge Powered Conversational Agents
State-of-the-art conversational agents have advanced significantly in co...
research
∙
10/12/2020
BioMegatron: Larger Biomedical Domain Language Model
There has been an influx of biomedical domain-specific language models, ...
research
∙
10/02/2020
MEGATRON-CNTRL: Controllable Story Generation with External Knowledge Using Large-Scale Language Models
Existing pre-trained large language models have shown unparalleled gener...
research
∙
05/13/2020
Large Scale Multi-Actor Generative Dialog Modeling
Non-goal oriented dialog agents (i.e. chatbots) aim to produce varying a...
research
∙
02/22/2020
Training Question Answering Models From Synthetic Data
Question and answer generation is a data augmentation method that aims t...
research
∙
09/25/2019
DisCo: Physics-Based Unsupervised Discovery of Coherent Structures in Spatiotemporal Systems
Extracting actionable insight from complex unlabeled scientific data is ...
research
∙
09/17/2019
Megatron-LM: Training Multi-Billion Parameter Language Models Using Model Parallelism
Recent work in unsupervised language modeling demonstrates that training...
research
∙
09/17/2019
Megatron-LM: Training Multi-Billion Parameter Language Models Using GPU Model Parallelism
Recent work in unsupervised language modeling demonstrates that training...
research
∙
02/26/2019
Coloring Big Graphs with AlphaGoZero
We show that recent innovations in deep reinforcement learning can effec...
research
∙
10/23/2018