research
∙
07/07/2023
On the Efficacy of Sampling Adapters
Sampling is a common strategy for generating text from probabilistic mod...
share
research
∙
07/07/2023
Testing the Predictions of Surprisal Theory in 11 Languages
A fundamental result in psycholinguistics is that less predictable words...
research
∙
06/06/2023
A Cross-Linguistic Pressure for Uniform Information Density in Word Order
While natural languages differ widely in both canonical word order and w...
research
∙
05/26/2023
Few-shot Fine-tuning vs. In-context Learning: A Fair Comparison and Evaluation
Few-shot fine-tuning and in-context learning are two alternative strateg...
research
∙
12/20/2022
A Measure-Theoretic Characterization of Tight Language Models
Language modeling, a central task in natural language processing, involv...
research
∙
12/19/2022
A Natural Bias for Language Generation Models
After just a few hundred training updates, a standard probabilistic mode...
research
∙
11/25/2022
On the Effect of Anticipation on Reading Times
Over the past two decades, numerous studies have demonstrated how less p...
research
∙
11/11/2022
The Architectural Bottleneck Principle
In this paper, we seek to measure how much information a component in a ...
research
∙
09/14/2022
On the Intersection of Context-Free and Regular Languages
The Bar-Hillel construction is a classic result in formal language theor...
research
∙
06/15/2022
Rethinking Reinforcement Learning for Recommendation: A Prompt Perspective
Modern recommender systems aim to improve user experience. As reinforcem...
research
∙
05/31/2022
Cluster-based Evaluation of Automatically Generated Text
While probabilistic language generators have improved dramatically over ...
research
∙
05/14/2022
Naturalistic Causal Probing for Morpho-Syntax
Probing has become a go-to methodology for interpreting and analyzing de...
research
∙
05/07/2022
UniMorph 4.0: Universal Morphology
The Universal Morphology (UniMorph) project is a collaborative effort pr...
research
∙
04/19/2022
Probing for the Usage of Grammatical Number
A central quest of probing is to uncover how pre-trained models encode a...
research
∙
03/31/2022
On the probability-quality paradox in language generation
When generating natural language from neural probabilistic models, high ...
research
∙
03/31/2022
Analyzing Wrap-Up Effects through an Information-Theoretic Lens
Numerous analyses of reading time (RT) data have been implemented – all ...
research
∙
02/01/2022
Typical Decoding for Natural Language Generation
Despite achieving incredibly low perplexities on myriad natural language...
research
∙
09/30/2021
A surprisal–duration trade-off across and within the world's languages
While there exist scores of natural languages, each with its unique feat...
research
∙
09/28/2021
On Homophony and Rényi Entropy
Homophony's widespread presence in natural languages is a controversial ...
research
∙
09/23/2021
Revisiting the Uniform Information Density Hypothesis
The uniform information density (UID) hypothesis posits a preference amo...
research
∙
09/08/2021
A Bayesian Framework for Information-Theoretic Probing
Pimentel et al. (2020) recently analysed probing from an information-the...
research
∙
06/04/2021
Modeling the Unigram Distribution
The unigram distribution is the non-contextual probability of finding a ...
research
∙
05/21/2021
A Non-Linear Structural Probe
Probes are models devised to investigate the encoding of knowledge – e.g...
research
∙
04/29/2021
How (Non-)Optimal is the Lexicon?
The mapping of lexical meanings to wordforms is a major feature of natur...
research
∙
04/25/2021
What About the Precedent: An Information-Theoretic Analysis of Common Law
In common law, the outcome of a new case is determined mostly by precede...
research
∙
04/15/2021
Quantifying Gender Bias Towards Politicians in Cross-Lingual Language Models
While the prevalence of large pre-trained language models has led to sig...
research
∙
04/13/2021
Finding Concept-specific Biases in Form–Meaning Associations
This work presents an information-theoretic operationalisation of cross-...
research
∙
02/03/2021
Disambiguatory Signals are Stronger in Word-initial Positions
Psycholinguistic studies of human word processing and lexical access pro...
research
∙
10/05/2020
Pareto Probing: Trading Off Accuracy for Complexity
The question of how to probe contextual word representations in a way th...
research
∙
10/05/2020
Speakers Fill Lexical Semantic Gaps with Context
Lexical ambiguity is widespread in language, allowing for the reuse of e...
research
∙
06/20/2020
SIGMORPHON 2020 Shared Task 0: Typologically Diverse Morphological Inflection
A broad goal in natural language processing (NLP) is to develop a system...
research
∙
05/28/2020
A Corpus for Large-Scale Phonetic Typology
A major hurdle in data-driven research on typology is having sufficient ...
research
∙
05/07/2020
Phonotactic Complexity and its Trade-offs
We present methods for calculating a measure of phonotactic complexity—b...
research
∙
05/04/2020
A Tale of a Probe and a Parser
Measuring what linguistic information is encoded in neural models of lan...
research
∙
05/01/2020
Predicting Declension Class from Form and Meaning
The noun lexica of many natural languages are divided into several decle...
research
∙
04/22/2020
Assessing the Reliability of Visual Explanations of Deep Models with Adversarial Perturbations
The interest in complex deep neural networks for computer vision applica...
research
∙
04/07/2020
Information-Theoretic Probing for Linguistic Structure
The success of neural networks on a diverse set of NLP tasks has led res...
research
∙
06/13/2019
Meaning to Form: Measuring Systematicity as Information
A longstanding debate in semiotics centers on the relationship between l...
research
∙
05/23/2018
A Generalized Active Learning Approach for Unsupervised Anomaly Detection
This work formalizes the new framework for anomaly detection, called act...
research
∙
07/09/2017