research
∙
08/09/2023
Extrapolating Large Language Models to Non-English by Aligning Languages
Due to the unbalanced training data distribution, the language ability o...
share
research
∙
07/18/2023
Linearized Relative Positional Encoding
Relative positional encoding is widely used in vanilla and linear transf...
research
∙
06/11/2023
Language Versatilists vs. Specialists: An Empirical Revisiting on Multilingual Transfer Ability
Multilingual transfer ability, which reflects how well the models fine-t...
research
∙
06/10/2023
INK: Injecting kNN Knowledge in Nearest Neighbor Machine Translation
Neural machine translation has achieved promising results on many transl...
research
∙
06/07/2023
M^3IT: A Large-Scale Dataset towards Multi-Modal Multilingual Instruction Tuning
Instruction tuning has significantly advanced large language models (LLM...
research
∙
05/25/2023
Decomposing the Enigma: Subgoal-based Demonstration Learning for Formal Theorem Proving
Large language models (LLMs) present an intriguing avenue of exploration...
research
∙
05/23/2023
Can Language Models Understand Physical Concepts?
Language models (LMs) gradually become general-purpose interfaces in the...
research
∙
05/23/2023
Generating Data for Symbolic Language with Large Language Models
While large language models (LLMs) bring not only performance but also c...
research
∙
05/17/2023
Statistical Knowledge Assessment for Generative Language Models
Generative Language Models (GLMs) have demonstrated capabilities to stor...
research
∙
05/14/2023
A Cognitive Stimulation Dialogue System with Multi-source Knowledge Fusion for Elders with Cognitive Impairment
When communicating with elders with cognitive impairment, cognitive stim...
research
∙
05/08/2023
Toeplitz Neural Network for Sequence Modeling
Sequence modeling has important applications in natural language process...
research
∙
04/18/2023
TTIDA: Controllable Generative Data Augmentation via Text-to-Text and Text-to-Image Models
Data augmentation has been established as an efficacious approach to sup...
research
∙
04/10/2023
Multilingual Machine Translation with Large Language Models: Empirical Results and Analysis
Large language models (LLMs) have demonstrated remarkable potential in h...
research
∙
03/27/2023
Fine-grained Audible Video Description
We explore a new task for audio-visual-language modeling called fine-gra...
research
∙
03/07/2023
A Challenging Benchmark for Low-Resource Learning
With promising yet saturated results in high-resource settings, low-reso...
research
∙
02/24/2023
Retrieved Sequence Augmentation for Protein Representation Learning
Protein language models have excelled in a variety of tasks, ranging fro...
research
∙
02/11/2023
A Reparameterized Discrete Diffusion Model for Text Generation
This work studies discrete diffusion probabilistic models with applicati...
research
∙
02/11/2023
Compositional Exemplars for In-context Learning
Large pretrained language models (LMs) have shown impressive In-Context ...
research
∙
12/20/2022
Lego-MT: Towards Detachable Models in Massively Multilingual Machine Translation
Traditional multilingual neural machine translation (MNMT) uses a single...
research
∙
12/20/2022
Self-adaptive In-context Learning
Despite the surprising few-shot performance of in-context learning (ICL)...
research
∙
12/19/2022
Explanation Regeneration via Information Bottleneck
Explaining the black-box predictions of NLP models naturally and accurat...
research
∙
10/24/2022
An Empirical Revisiting of Linguistic Knowledge Fusion in Language Understanding Tasks
Though linguistic knowledge emerges during large-scale language model pr...
research
∙
10/22/2022
ProGen: Progressive Zero-shot Dataset Generation via In-context Feedback
Recently, dataset-generation-based zero-shot learning has shown promisin...
research
∙
10/19/2022
The Devil in Linear Transformer
Linear transformers aim to reduce the quadratic space-time complexity of...
research
∙
10/17/2022
DiffuSeq: Sequence to Sequence Text Generation with Diffusion Models
Recently, diffusion models have emerged as a new paradigm for generative...
research
∙
10/14/2022
CAB: Comprehensive Attention Benchmarking on Long Sequence Modeling
Transformer has achieved remarkable success in language, image, and spee...
research
∙
06/21/2022
Vicinity Vision Transformer
Vision transformers have shown great success on numerous computer vision...
research
∙
05/29/2022
CoNT: Contrastive Neural Text Generation
Recently, contrastive learning attracts increasing interests in neural t...
research
∙
05/25/2022
ZeroGen^+: Self-Guided High-Quality Data Generation in Efficient Zero-Shot Learning
Nowadays, owing to the superior capacity of the large pre-trained langua...
research
∙
04/20/2022
Event Transition Planning for Open-ended Text Generation
Open-ended text generation tasks, such as dialogue generation and story ...
research
∙
04/10/2022
Linear Complexity Randomized Self-attention Mechanism
Recently, random feature attentions (RFAs) are proposed to approximate t...
research
∙
02/17/2022
cosFormer: Rethinking Softmax in Attention
Transformer has shown great successes in natural language processing, co...
research
∙
02/17/2022
Revisiting Over-smoothing in BERT from the Perspective of Graph
Recently over-smoothing phenomenon of Transformer-based models is observ...
research
∙
02/16/2022
ZeroGen: Efficient Zero-shot Learning via Dataset Generation
There is a growing interest in dataset generation recently due to the su...
research
∙
02/13/2022
A Contrastive Framework for Neural Text Generation
Text generation is of great importance to many natural language processi...
research
∙
01/16/2022
SNCSE: Contrastive Learning for Unsupervised Sentence Embedding with Soft Negative Samples
Unsupervised sentence embedding aims to obtain the most appropriate embe...
research
∙
12/15/2021
Oracle Linguistic Graphs Complement a Pretrained Transformer Language Model: A Cross-formalism Comparison
We examine the extent to which, in principle, linguistic graph represent...
research
∙
10/06/2021
ABC: Attention with Bounded-memory Control
Transformer architectures have achieved state-of-the-art results on a va...
research
∙
10/06/2021
Ripple Attention for Visual Perception with Sub-quadratic Complexity
Transformer architectures are now central to modeling in natural languag...
research
∙
05/31/2021
Cascaded Head-colliding Attention
Transformers have advanced the field of natural language processing (NLP...
research
∙
03/03/2021
Random Feature Attention
Transformers are state-of-the-art models for a variety of sequence model...
research
∙
02/04/2021
Adaptive Semiparametric Language Models
We present a language model that combines a large parametric neural netw...
research
∙
05/27/2020
Syntactic Structure Distillation Pretraining For Bidirectional Encoders
Textual representation learners trained on large amounts of data have ac...
research
∙
10/18/2019
A Mutual Information Maximization Perspective of Language Representation Learning
We show state-of-the-art word representation learning methods maximize a...
research
∙
10/01/2019
Putting Machine Translation in Context with the Noisy Channel Model
We show that Bayes' rule provides a compelling mechanism for controlling...
research
∙
06/03/2019
Episodic Memory in Lifelong Language Learning
We introduce a lifelong language learning setup where a model needs to l...
research
∙
01/31/2019
Learning and Evaluating General Linguistic Intelligence
We define general linguistic intelligence as the ability to reuse previo...
research
∙
01/27/2019
Variational Smoothing in Recurrent Neural Network Language Models
We present a new theoretical perspective of data noising in recurrent ne...
research
∙
11/26/2018
Sentence Encoding with Tree-constrained Relation Networks
The meaning of a sentence is a function of the relations that hold betwe...
research
∙
11/06/2018