research
∙
12/06/2022
Robust Speech Recognition via Large-Scale Weak Supervision
We study the capabilities of speech processing systems trained simply to...
share
research
∙
01/24/2022
Text and Code Embeddings by Contrastive Pre-Training
Text embeddings are useful features in many applications such as semanti...
research
∙
10/11/2021
Unsupervised Neural Machine Translation with Generative Language Models Only
We show how to derive state-of-the-art unsupervised neural machine trans...
research
∙
08/05/2021
Evaluating CLIP: Towards Characterization of Broader Capabilities and Downstream Implications
Recently, there have been breakthroughs in computer vision ("CV") models...
research
∙
07/07/2021
Evaluating Large Language Models Trained on Code
We introduce Codex, a GPT language model fine-tuned on publicly availabl...
research
∙
02/26/2021
Learning Transferable Visual Models From Natural Language Supervision
State-of-the-art computer vision systems are trained to predict a fixed ...
research
∙
02/24/2021
Zero-Shot Text-to-Image Generation
Text-to-image generation has traditionally focused on finding better mod...
research
∙
10/28/2020
Scaling Laws for Autoregressive Generative Modeling
We identify empirical scaling laws for the cross-entropy loss in four do...
research
∙
09/02/2020
Learning to summarize from human feedback
As language models become more powerful, training and evaluation are inc...
research
∙
05/28/2020
Language Models are Few-Shot Learners
Recent work has demonstrated substantial gains on many NLP tasks and ben...
research
∙
04/30/2020
Jukebox: A Generative Model for Music
We introduce Jukebox, a model that generates music with singing in the r...
research
∙
01/23/2020
Scaling Laws for Neural Language Models
We study empirical scaling laws for language model performance on the cr...
research
∙
09/18/2019
Fine-Tuning Language Models from Human Preferences
Reward learning enables the application of reinforcement learning (RL) t...
research
∙
08/24/2019
Release Strategies and the Social Impacts of Language Models
Large language models have a range of beneficial uses: they can assist i...
research
∙
04/23/2019
Generating Long Sequences with Sparse Transformers
Transformers are powerful sequence models, but require time and memory t...
research
∙
03/15/2018
Improving GANs Using Optimal Transport
We present Optimal Transport GAN (OT-GAN), a variant of generative adver...
research
∙
04/05/2017
Learning to Generate Reviews and Discovering Sentiment
We explore the properties of byte-level recurrent language models. When ...
research
∙
06/10/2016