research
∙
08/29/2023
Characterizing Learning Curves During Language Model Pre-Training: Learning, Forgetting, and Stability
How do language models learn to make predictions during pre-training? To...
share
research
∙
05/26/2023
Characterizing and Measuring Linguistic Dataset Drift
NLP models often degrade in performance when real world data distributio...
research
∙
03/20/2023
Language Model Behavior: A Comprehensive Survey
Transformer language models have received widespread public attention, y...
research
∙
05/22/2022
The Geometry of Multilingual Language Model Representations
We assess how multilingual language models maintain a shared multilingua...
research
∙
10/05/2021
Word Acquisition in Neural Language Models
We investigate how neural language models acquire individual words durin...
research
∙
06/10/2021
Convolutions and Self-Attention: Re-interpreting Relative Positions in Pre-trained Language Models
In this paper, we detail the relationship between convolutions and self-...
research
∙
05/17/2020