research
∙
09/20/2023
The Languini Kitchen: Enabling Language Modelling Research at Different Scales of Compute
The Languini Kitchen serves as both a research collective and codebase d...
share
research
∙
05/09/2023
Large Language Model Programs
In recent years, large pre-trained language models (LLMs) have demonstra...
research
∙
06/29/2022
Solving Quantitative Reasoning Problems with Language Models
Language models have achieved remarkable performance on a wide range of ...
research
∙
03/11/2022
Block-Recurrent Transformers
We introduce the Block-Recurrent Transformer, which applies a transforme...
research
∙
02/11/2022
A Modern Self-Referential Weight Matrix That Learns to Modify Itself
The weight matrix (WM) of a neural network (NN) is its program. The prog...
research
∙
12/31/2021
Improving Baselines in the Wild
We share our experience with the recently released WILDS benchmark, a co...
research
∙
06/11/2021
Going Beyond Linear Transformers with Recurrent Fast Weight Programmers
Transformers with linearised attention ("linear Transformers") have demo...
research
∙
02/22/2021
Linear Transformers Are Secretly Fast Weight Memory Systems
We show the formal equivalence of linearised self-attention mechanisms a...
research
∙
11/16/2020
Learning Associative Inference Using Fast Weight Memory
Humans can quickly associate stimuli to solve problems in novel contexts...
research
∙
10/15/2019
Enhancing the Transformer with Explicit Relational Encoding for Math Problem Solving
We incorporate Tensor-Product Representations within the Transformer in ...
research
∙
11/29/2018