research
∙
06/30/2023
Stay on topic with Classifier-Free Guidance
Classifier-Free Guidance (CFG) has recently emerged in text-to-image gen...
share
research
∙
06/06/2023
LEACE: Perfect linear concept erasure in closed form
Concept erasure aims to remove specified features from a representation....
research
∙
06/02/2023
GAIA Search: Hugging Face and Pyserini Interoperability for NLP Training Data Exploration
Noticing the urgent need to provide tools for fast and user-friendly qua...
research
∙
05/31/2023
Recasting Self-Attention with Holographic Reduced Representations
In recent years, self-attention has become the dominant paradigm for seq...
research
∙
05/24/2023
Can Transformers Learn to Solve Problems Recursively?
Neural networks have in recent years shown promise for helping software ...
research
∙
04/21/2023
Emergent and Predictable Memorization in Large Language Models
Memorization, or the tendency of large language models (LLMs) to output ...
research
∙
04/03/2023
Pythia: A Suite for Analyzing Large Language Models Across Training and Scaling
How do large language models (LLMs) develop and evolve over the course o...
research
∙
03/14/2023
Eliciting Latent Predictions from Transformers with the Tuned Lens
We analyze transformers from the perspective of iterative inference, see...
research
∙
03/07/2023
The BigScience ROOTS Corpus: A 1.6TB Composite Multilingual Dataset
As language models grow ever larger, the need for large-scale high-quali...
research
∙
12/19/2022
BLOOM+1: Adding Language Support to BLOOM for Zero-Shot Prompting
The BLOOM model is a large open-source multilingual language model capab...
research
∙
11/03/2022
Crosslingual Generalization through Multitask Finetuning
Multitask prompted finetuning (MTF) has been shown to help large languag...
research
∙
10/26/2022
Large language models are not zero-shot communicators
Despite widespread use of LLMs as conversational agents, evaluations of ...
research
∙
10/12/2022
EleutherAI: Going Beyond "Open Science" to "Science in the Open"
Over the past two years, EleutherAI has established itself as a radicall...
research
∙
05/04/2022
Data Governance in the Age of Large-Scale Data-Driven Language Technology
The recent emergence and adoption of Machine Learning technology, and sp...
research
∙
04/18/2022
VQGAN-CLIP: Open Domain Image Generation and Editing with Natural Language Guidance
Generating and editing images from open domain text prompts is a challen...
research
∙
04/14/2022
GPT-NeoX-20B: An Open-Source Autoregressive Language Model
We introduce GPT-NeoX-20B, a 20 billion parameter autoregressive languag...
research
∙
01/25/2022
Documenting Geographically and Contextually Diverse Data Sources: The BigScience Catalogue of Language Data and Resources
In recent years, large-scale data collection efforts have prioritized th...
research
∙
01/19/2022
Neural Language Models are Effective Plagiarists
As artificial intelligence (AI) technologies become increasingly powerfu...
research
∙
01/13/2022
Datasheet for the Pile
This datasheet describes the Pile, a 825 GiB dataset of human-authored t...
research
∙
10/15/2021
Multitask Prompted Training Enables Zero-Shot Task Generalization
Large language models have recently been shown to attain reasonable zero...
research
∙
03/23/2021
Towards a Formal Model of Narratives
In this paper, we propose the beginnings of a formal framework for model...
research
∙
03/22/2021
Quality at a Glance: An Audit of Web-Crawled Multilingual Datasets
With the success of large-scale pre-training and multilingual modeling i...
research
∙
12/31/2020
The Pile: An 800GB Dataset of Diverse Text for Language Modeling
Recent work has demonstrated that increased training dataset diversity i...
research
∙
11/04/2020
Pitfalls in Machine Learning Research: Reexamining the Development Cycle
Machine learning has the potential to fuel further advances in data scie...
research
∙
03/11/2020
Magic: the Gathering is as Hard as Arithmetic
Magic: the Gathering is a popular and famously complicated card game abo...
research
∙
03/24/2019