research
∙
06/29/2023
An Efficient General-Purpose Modular Vision Model via Multi-Task Heterogeneous Training
We present a model that can perform multiple vision tasks and can be ada...
share
research
∙
06/07/2023
ModuleFormer: Learning Modular Large Language Models From Uncurated Data
Large Language Models (LLMs) have achieved remarkable results. But exist...
research
∙
05/04/2023
Principle-Driven Self-Alignment of Language Models from Scratch with Minimal Human Supervision
Recent AI-assistant agents, such as ChatGPT, predominantly rely on super...
research
∙
04/17/2023
Hyper-Decision Transformer for Efficient Online Policy Adaptation
Decision Transformers (DT) have demonstrated strong performances in offl...
research
∙
04/06/2023
Visual Dependency Transformers: Dependency Tree Emerges from Reversed Attention
Humans possess a versatile mechanism for extracting structured represent...
research
∙
03/09/2023
Planning with Large Language Models for Code Generation
Existing large language model-based code generation pipelines typically ...
research
∙
01/24/2023
Transformer-Patcher: One Mistake worth One Neuron
Large Transformer-based Pretrained Language Models (PLMs) dominate almos...
research
∙
12/15/2022
Mod-Squad: Designing Mixture of Experts As Modular Multi-Task Learners
Optimization in multi-task learning (MTL) is more challenging than singl...
research
∙
10/11/2022
Mixture of Attention Heads: Selecting Attention Heads Per Token
Mixture-of-Experts (MoE) networks have been proposed as an efficient way...
research
∙
06/27/2022
Prompting Decision Transformer for Few-Shot Policy Generalization
Humans can leverage prior experience and learn novel tasks from a handfu...
research
∙
06/08/2022
Syntactic Inductive Biases for Deep Learning Methods
In this thesis, we try to build a connection between the two schools by ...
research
∙
03/19/2021
Learning Task Decomposition with Ordered Memory Policy Network
Many complex real-world tasks are composed of several levels of sub-task...
research
∙
12/01/2020
StructFormer: Joint Unsupervised Induction of Dependency and Constituency Structure from Masked Language Modeling
There are two major classes of natural language grammars – the dependenc...
research
∙
11/08/2020
Long Range Arena: A Benchmark for Efficient Transformers
Transformers do not scale very well to long sequence lengths largely bec...
research
∙
10/09/2020
Recursive Top-Down Production for Sentence Generation with Latent Trees
We model the recursive production property of context-free grammars for ...
research
∙
05/12/2020
Exploiting Syntactic Structure for Better Language Modeling: A Syntactic Distance Approach
It is commonly believed that knowledge of syntactic structure should imp...
research
∙
10/29/2019
Ordered Memory
Stack-augmented recurrent neural networks (RNNs) have been of interest t...
research
∙
06/23/2019
Investigating Biases in Textual Entailment Datasets
The ability to understand logical relationships between sentences is an ...
research
∙
10/22/2018
Ordered Neurons: Integrating Tree Structures into Recurrent Neural Networks
Recurrent neural network (RNN) models are widely used for processing seq...
research
∙
09/25/2018
BanditSum: Extractive Summarization as a Contextual Bandit
In this work, we propose a novel method for training neural networks to ...
research
∙
06/11/2018
Straight to the Tree: Constituency Parsing with Neural Syntactic Distance
In this work, we propose a novel constituency parsing scheme. The model ...
research
∙
03/07/2018
Generating Contradictory, Neutral, and Entailing Sentences
Learning distributed sentence representations remains an interesting pro...
research
∙
11/02/2017
Neural Language Modeling by Jointly Learning Syntax and Lexicon
We propose a neural language model capable of unsupervised syntactic str...
research
∙
07/26/2017
Self-organized Hierarchical Softmax
We propose a new self-organizing hierarchical softmax formulation for ne...
research
∙
11/15/2015