research
∙
06/05/2023
On "Scientific Debt" in NLP: A Case for More Rigour in Language Model Pre-Training Research
This evidence-based position paper critiques current research practices ...
share
research
∙
12/19/2022
A Natural Bias for Language Generation Models
After just a few hundred training updates, a standard probabilistic mode...
research
∙
03/01/2022
Transformer Grammars: Augmenting Transformer Language Models with Syntactic Inductive Biases at Scale
Transformer language models that are trained on vast amounts of data hav...
research
∙
04/16/2021
IndoNLG: Benchmark and Resources for Evaluating Indonesian Natural Language Generation
A benchmark provides an ecosystem to measure the advancement of models w...
research
∙
02/03/2021
Pitfalls of Static Language Modelling
Our world is open-ended, non-stationary and constantly evolving; thus wh...
research
∙
05/27/2020
Syntactic Structure Distillation Pretraining For Bidirectional Encoders
Textual representation learners trained on large amounts of data have ac...
research
∙
06/14/2019
Scalable Syntax-Aware Language Models Using Knowledge Distillation
Prior work has shown that, on small amounts of training data, syntactic ...
research
∙
04/07/2019
Unsupervised Recurrent Neural Network Grammars
Recurrent neural network grammars (RNNG) are generative models of langua...
research
∙
06/11/2018
Finding Syntax in Human Encephalography with Beam Search
Recurrent neural network grammars (RNNGs) are generative models of (tree...
research
∙
01/15/2017
DyNet: The Dynamic Neural Network Toolkit
We describe DyNet, a toolkit for implementing neural network models base...
research
∙
11/17/2016
What Do Recurrent Neural Network Grammars Learn About Syntax?
Recurrent neural network grammars (RNNG) are a recently proposed probabi...
research
∙
04/22/2016
Dependency Parsing with LSTMs: An Empirical Evaluation
We propose a transition-based dependency parser using Recurrent Neural N...
research
∙
02/25/2016