research
∙
06/06/2023
A Cross-Linguistic Pressure for Uniform Information Density in Word Order
While natural languages differ widely in both canonical word order and w...
share
research
∙
03/14/2023
A Theory of Emergent In-Context Learning as Implicit Structure Induction
Scaling large language models (LLMs) leads to an emergent capacity to le...
research
∙
06/09/2022
Crosslinguistic word order variation reflects evolutionary pressures of dependency and information locality
Languages vary considerably in syntactic structure. About 40 languages h...
research
∙
04/21/2021
Sensitivity as a Complexity Measure for Sequence Classification Tasks
We introduce a theoretical framework for understanding and predicting th...
research
∙
10/15/2020
RNNs can generate bounded hierarchical languages with optimal memory
Recurrent neural networks empirically generate natural language with hig...
research
∙
06/17/2019
Tabula nearly rasa: Probing the Linguistic Knowledge of Character-Level Neural Language Models Trained on Unsegmented Text
Recurrent neural networks (RNNs) have reached striking performance in ma...
research
∙
06/16/2019
Theoretical Limitations of Self-Attention in Neural Sequence Models
Transformers are emerging as the new workhorse of NLP, showing great suc...
research
∙
02/02/2019
Character-based Surprisal as a Model of Human Reading in the Presence of Errors
Intuitively, human readers cope easily with errors in text; typos, missp...
research
∙
07/31/2018
Modeling Task Effects in Human Reading with Neural Attention
Humans read by making a sequence of fixations and saccades. They often s...
research
∙
08/19/2016