research
∙
09/20/2023
Kosmos-2.5: A Multimodal Literate Model
We present Kosmos-2.5, a multimodal literate model for machine reading o...
share
research
∙
07/17/2023
Retentive Network: A Successor to Transformer for Large Language Models
In this work, we propose Retentive Network (RetNet) as a foundation arch...
research
∙
07/05/2023
LongNet: Scaling Transformers to 1,000,000,000 Tokens
Scaling sequence length has become a critical demand in the era of large...
research
∙
06/26/2023
Kosmos-2: Grounding Multimodal Large Language Models to the World
We introduce Kosmos-2, a Multimodal Large Language Model (MLLM), enablin...
research
∙
05/18/2023
Discourse Centric Evaluation of Machine Translation with a Densely Annotated Parallel Corpus
Several recent papers claim human parity at sentence-level Machine Trans...
research
∙
05/18/2023
On the Off-Target Problem of Zero-Shot Multilingual Neural Machine Translation
While multilingual neural machine translation has achieved great success...
research
∙
04/06/2023
On the Pareto Front of Multilingual Neural Machine Translation
In this work, we study how the generalization performance of a given dir...
research
∙
03/01/2023
Are More Layers Beneficial to Graph Transformers?
Despite that going deep has proven successful in many neural architectur...
research
∙
02/27/2023
Language Is Not All You Need: Aligning Perception with Language Models
A big convergence of language, multimodal perception, action, and world ...
research
∙
01/17/2023
HanoiT: Enhancing Context-aware Translation via Selective Context
Context-aware neural machine translation aims to use the document-level ...
research
∙
12/20/2022
A Length-Extrapolatable Transformer
Position modeling plays a critical role in Transformers. In this paper, ...
research
∙
12/20/2022
GanLM: Encoder-Decoder Pre-training with an Auxiliary Discriminator
Pre-trained models have achieved remarkable success in natural language ...
research
∙
12/15/2022
TRIP: Triangular Document-level Pre-training for Multilingual Language Models
Despite the current success of multilingual pre-training, most prior wor...
research
∙
11/23/2022
TorchScale: Transformers at Scale
Large Transformers have achieved state-of-the-art performance across man...
research
∙
10/26/2022
A Bilingual Parallel Corpus with Discourse Annotations
Machine translation (MT) has almost achieved human parity at sentence-le...
research
∙
10/13/2022
CROP: Zero-shot Cross-lingual Named Entity Recognition with Multilingual Labeled Sequence Translation
Named entity recognition (NER) suffers from the scarcity of annotated tr...
research
∙
10/12/2022
Foundation Transformers
A big convergence of model architectures across language, vision, speech...
research
∙
09/28/2022
Multilingual Transitivity and Bidirectional Multilingual Agreement for Multilingual Document-level Machine Translation
Multilingual machine translation has been proven an effective strategy t...
research
∙
07/29/2022
GTrans: Grouping and Fusing Transformer Layers for Neural Machine Translation
Transformer structure, stacked by a sequence of encoder and decoder netw...
research
∙
07/11/2022
HLT-MT: High-resource Language-specific Training for Multilingual Neural Machine Translation
Multilingual neural machine translation (MNMT) trained in multiple langu...
research
∙
07/11/2022
UM4: Unified Multilingual Multiple Teacher-Student Model for Zero-Resource Neural Machine Translation
Most translation tasks among languages belong to the zero-resource trans...
research
∙
06/13/2022
Language Models are General-Purpose Interfaces
Foundation models have received much attention due to their effectivenes...
research
∙
04/20/2022
On the Representation Collapse of Sparse Mixture of Experts
Sparse mixture of experts provides larger model capacity while requiring...
research
∙
04/18/2022
StableMoE: Stable Routing Strategy for Mixture of Experts
The Mixture-of-Experts (MoE) technique can scale up the model size of Tr...
research
∙
03/01/2022
DeepNet: Scaling Transformers to 1,000 Layers
In this paper, we propose a simple yet effective method to stabilize ext...
research
∙
02/23/2022
Zero-shot Cross-lingual Transfer of Prompt-based Tuning with a Unified Multilingual Prompt
Prompt-based tuning has been proven effective for pretrained language mo...
research
∙
01/26/2022
A Unified Strategy for Multilingual Grammatical Error Correction with Pre-trained Cross-Lingual Language Model
Synthetic data construction of Grammatical Error Correction (GEC) for no...
research
∙
01/06/2022
Phrase-level Adversarial Example Generation for Neural Machine Translation
While end-to-end neural machine translation (NMT) has achieved impressiv...
research
∙
01/05/2022
SMDT: Selective Memory-Augmented Neural Document Translation
Existing document-level neural machine translation (NMT) models have suf...
research
∙
11/03/2021
Multilingual Machine Translation Systems from Microsoft for WMT21 Shared Task
This report describes Microsoft's machine translation systems for the WM...
research
∙
10/16/2021
Towards Making the Most of Multilingual Pretraining for Zero-Shot Neural Machine Translation
This paper demonstrates that multilingual pretraining, a proper fine-tun...
research
∙
06/30/2021
XLM-E: Cross-lingual Language Model Pre-training via ELECTRA
In this paper, we introduce ELECTRA-style tasks to cross-lingual languag...
research
∙
06/25/2021
DeltaLM: Encoder-Decoder Pre-training for Language Generation and Translation by Augmenting Pretrained Multilingual Encoders
While pretrained encoders have achieved success in various natural langu...
research
∙
05/27/2021
How Does Distilled Data Complexity Impact the Quality and Confidence of Non-Autoregressive Machine Translation?
While non-autoregressive (NAR) models are showing great promise for mach...
research
∙
04/18/2021
Zero-shot Cross-lingual Transfer of Neural Machine Translation with Multilingual Pretrained Encoders
Previous works mainly focus on improving cross-lingual transfer for NLU ...
research
∙
04/18/2021
mT6: Multilingual Pretrained Text-to-Text Transformer with Translation Pairs
Multilingual T5 (mT5) pretrains a sequence-to-sequence model on massive ...
research
∙
12/31/2020
XLM-T: Scaling up Multilingual Machine Translation with Pretrained Cross-lingual Transformer Encoders
Multilingual machine translation enables a single model to translate bet...
research
∙
02/07/2020
Multimodal Matching Transformer for Live Commenting
Automatic live commenting aims to provide real-time comments on videos f...
research
∙
10/11/2019
Group, Extract and Aggregate: Summarizing a Large Amount of Finance News for Forex Movement Prediction
Incorporating related text information has proven successful in stock ma...
research
∙
09/18/2019
Recursive Graphical Neural Networks for Text Classification
The complicated syntax structure of natural language is hard to be expli...
research
∙
08/08/2019
Key Fact as Pivot: A Two-Stage Model for Low Resource Table-to-Text Generation
Table-to-text generation aims to translate the structured data into the ...
research
∙
09/13/2018
Unsupervised Machine Commenting with Neural Variational Topic Model
Article comments can provide supplementary opinions and facts for reader...
research
∙
09/13/2018
LiveBot: Generating Live Video Comments Based on Visual and Textual Contexts
We introduce the task of automatic live commenting. Live commenting, whi...
research
∙
09/10/2018
A Deep Reinforced Sequence-to-Set Model for Multi-Label Text Classification
Multi-label text classification (MLTC) aims to assign multiple labels to...
research
∙
08/26/2018
Semantic-Unit-Based Dilated Convolution for Multi-Label Text Classification
We propose a novel model for multi-label text classification, which is b...
research
∙
08/22/2018
Identifying High-Quality Chinese News Comments Based on Multi-Target Text Matching Model
With the development of information technology, there is an explosive gr...
research
∙
06/13/2018
SGM: Sequence Generation Model for Multi-label Classification
Multi-label classification is an important yet challenging task in natur...
research
∙
06/10/2018
Deconvolution-Based Global Decoding for Neural Machine Translation
A great proportion of sequence-to-sequence (Seq2Seq) models for Neural M...
research
∙
05/13/2018
Bag-of-Words as Target for Neural Machine Translation
A sentence can be translated into more than one correct sentences. Howev...
research
∙
05/13/2018