research
∙
07/11/2023
Stack More Layers Differently: High-Rank Training Through Low-Rank Updates
Despite the dominance and effectiveness of scaling, resulting in large n...
share
research
∙
06/14/2023
Recipes for Sequential Pre-training of Multilingual Encoder and Seq2Seq Models
Pre-trained encoder-only and sequence-to-sequence (seq2seq) models each ...
research
∙
05/26/2023
Honey, I Shrunk the Language: Language Model Behavior at Reduced Scale
In recent years, language models have drastically grown in size, and the...
research
∙
04/04/2023
Scalable and Accurate Self-supervised Multimodal Representation Learning without Aligned Video and Text Data
Scaling up weakly-supervised datasets has shown to be highly effective i...
research
∙
03/29/2023
Larger Probes Tell a Different Story: Extending Psycholinguistic Datasets Via In-Context Learning
Language model probing is often used to test specific capabilities of th...
research
∙
03/28/2023
Scaling Down to Scale Up: A Guide to Parameter-Efficient Fine-Tuning
This paper presents a systematic overview and comparison of parameter-ef...
research
∙
11/15/2022
Reasoning Circuits: Few-shot Multihop Question Generation with Structured Rationales
Multi-hop Question Generation is the task of generating questions which ...
research
∙
10/08/2022
On Task-Adaptive Pretraining for Dialogue Response Selection
Recent advancements in dialogue response selection (DRS) are based on th...
research
∙
08/02/2022
AlexaTM 20B: Few-Shot Learning Using a Large-Scale Multilingual Seq2Seq Model
In this work, we demonstrate that multilingual large-scale sequence-to-s...
research
∙
06/06/2022
Learning to Ask Like a Physician
Existing question answering (QA) datasets derived from electronic health...
research
∙
05/21/2022
Life after BERT: What do Other Muppets Understand about Language?
Existing pre-trained transformer analysis works usually focus only on on...
research
∙
05/20/2022
Down and Across: Introducing Crossword-Solving as a New NLP Benchmark
Solving crossword puzzles requires diverse reasoning capabilities, acces...
research
∙
05/06/2022
Federated Learning with Noisy User Feedback
Machine Learning (ML) systems are getting increasingly popular, and driv...
research
∙
07/21/2021
Multi-Stream Transformers
Transformer-based encoder-decoder models produce a fused token-wise repr...
research
∙
07/14/2021
An Efficient DP-SGD Mechanism for Large Scale NLP Models
Recent advances in deep learning have drastically improved performance o...
research
∙
05/14/2021
BERT Busters: Outlier LayerNorm Dimensions that Disrupt BERT
Multiple studies have shown that BERT is remarkably robust to pruning, y...
research
∙
10/15/2020
Continual Learning for Neural Semantic Parsing
A semantic parsing model is crucial to natural language processing appli...
research
∙
05/01/2020
When BERT Plays the Lottery, All Tickets Are Winning
Much of the recent success in NLP is due to the large Transformer-based ...
research
∙
02/27/2020
A Primer in BERTology: What we know about how BERT works
Transformer-based models are now widely used in NLP, but we still do not...
research
∙
10/16/2019
Memory-Augmented Recurrent Networks for Dialogue Coherence
Recent dialogue approaches operate by reading each word in a conversatio...
research
∙
10/16/2019
Injecting Hierarchy with U-Net Transformers
The Transformer architecture has become increasingly popular over the pa...
research
∙
08/29/2019
NarrativeTime: Dense High-Speed Temporal Annotation on a Timeline
We present NarrativeTime, a new timeline-based annotation scheme for tem...
research
∙
08/28/2019
Solving Math Word Problems with Double-Decoder Transformer
This paper proposes a Transformer-based model to generate equations for ...
research
∙
08/21/2019
Revealing the Dark Secrets of BERT
BERT-based architectures currently give state-of-the-art performance on ...
research
∙
04/10/2019
What's in a Name? Reducing Bias in Bios without Access to Protected Attributes
There is a growing body of work that proposes methods for mitigating bia...
research
∙
10/11/2018
Adversarial Text Generation Without Reinforcement Learning
Generative Adversarial Networks (GANs) have experienced a recent surge i...
research
∙
09/18/2018
Triad-based Neural Network for Coreference Resolution
We propose a triad-based neural network system that generates affinity s...
research
∙
08/27/2018
Adversarial Decomposition of Text Representation
In this paper, we present a method for adversarial decomposition of text...
research
∙
03/06/2018
CliNER 2.0: Accessible and Accurate Clinical Concept Extraction
Clinical notes often describe important aspects of a patient's stay and ...
research
∙
05/01/2017
Forced to Learn: Discovering Disentangled Representations Without Exhaustive Labels
Learning a better representation with neural networks is a challenging p...
research
∙
03/17/2017
Temporal Information Extraction for Question Answering Using Syntactic Dependencies in an LSTM-based Architecture
In this paper, we propose to use a set of simple, uniform in architectur...
research
∙
12/28/2016
Here's My Point: Joint Pointer Architecture for Argument Mining
One of the major goals in automated argumentation mining is to uncover t...
research
∙
12/09/2016
#HashtagWars: Learning a Sense of Humor
In this work, we present a new dataset for computational humor, specific...
research
∙
12/09/2016
Evaluating Creative Language Generation: The Case of Rap Lyric Ghostwriting
Language generation tasks that seek to mimic human ability to use langua...
research
∙
10/16/2015