research
∙
06/21/2023
Feature Interactions Reveal Linguistic Structure in Language Models
We study feature interactions in the context of feature attribution meth...
share
research
∙
01/30/2023
Quantifying Context Mixing in Transformers
Self-attention weights and their transformed variants have been the main...
research
∙
11/24/2022
Undesirable biases in NLP: Averting a crisis of measurement
As Natural Language Processing (NLP) technology rapidly develops and spr...
research
∙
07/21/2022
The Birth of Bias: A case study on the evolution of gender bias in an English language model
Detecting and mitigating harmful biases in modern language models are wi...
research
∙
09/30/2021
Syntactic Persistence in Language Models: Priming as a Window into Abstract Language Representations
We investigate the extent to which modern, neural language models are su...
research
∙
11/10/2020
DoLFIn: Distributions over Latent Features for Interpretability
Interpreting the inner workings of neural models is a key step in ensuri...
research
∙
05/31/2020
Transferring Inductive Biases through Knowledge Distillation
Having the right inductive biases can be crucial in many tasks or scenar...
research
∙
05/02/2020
Quantifying Attention Flow in Transformers
In the Transformer model, "self-attention" combines information from att...
research
∙
09/19/2019
Analysing Neural Language Models: Contextual Decomposition Reveals Default Reasoning in Number and Gender Assignment
Extensive research has recently shown that recurrent neural language mod...
research
∙
06/04/2019
Blackbox meets blackbox: Representational Similarity and Stability Analysis of Neural Language Models and Brains
In this paper, we define and apply representational stability analysis (...
research
∙
06/01/2019
Siamese recurrent networks learn first-order logic reasoning and exhibit zero-shot compositional generalization
Can neural nets learn logic? We approach this classic question with curr...
research
∙
01/16/2019
Formal models of Structure Building in Music, Language and Animal Songs
Human language, music and a variety of animal vocalisations constitute w...
research
∙
08/24/2018
Under the Hood: Using Diagnostic Classifiers to Investigate and Improve how Language Models Track Agreement Information
How do neural language models keep track of number agreement between sub...
research
∙
11/28/2017
Visualisation and 'diagnostic classifiers' reveal how recurrent and recursive neural networks process hierarchical structure
We investigate how neural networks can learn and process languages with ...
research
∙
11/25/2017
Experiential, Distributional and Dependency-based Word Embeddings have Complementary Roles in Decoding Brain Activity
We evaluate 8 different word embedding models on their usefulness for pr...
research
∙
03/01/2016
Quantifying the vanishing gradient and long distance dependency problem in recursive neural networks and recursive LSTMs
Recursive neural networks (RNN) and their recently proposed extension re...
research
∙
04/18/2015
Unsupervised Dependency Parsing: Let's Use Supervised Parsers
We present a self-training approach to unsupervised dependency parsing t...
research
∙
03/09/2015