research
∙
08/22/2023
SeamlessM4T-Massively Multilingual Multimodal Machine Translation
What does it take to create the Babel Fish, a tool that can help individ...
share
research
∙
04/10/2023
Enhancing Speech-to-Speech Translation with Multiple TTS Targets
It has been known that direct speech-to-speech translation (S2ST) models...
research
∙
03/01/2023
MuAViC: A Multilingual Audio-Visual Corpus for Robust Speech Recognition and Robust Speech-to-Text Translation
We introduce MuAViC, a multilingual audio-visual corpus for robust speec...
research
∙
01/27/2023
Pre-training for Speech Translation: CTC Meets Optimal Transport
The gap between speech and text modalities is a major challenge in speec...
research
∙
01/25/2023
A Holistic Cascade System, benchmark, and Human Evaluation Protocol for Expressive Speech-to-Speech Translation
Expressive speech-to-speech translation (S2ST) aims to transfer prosodic...
research
∙
12/15/2022
UnitY: Two-pass Direct Speech-to-speech Translation with Discrete Units
Direct speech-to-speech translation (S2ST), in which all components can ...
research
∙
11/15/2022
Introducing Semantics into Speech Encoders
Recent studies find existing self-supervised speech encoders contain pri...
research
∙
11/11/2022
Speech-to-Speech Translation For A Real-world Unwritten Language
We study speech-to-speech translation (S2ST) that translates speech from...
research
∙
11/08/2022
SpeechMatrix: A Large-Scale Mined Corpus of Multilingual Speech-to-Speech Translations
We present SpeechMatrix, a large-scale multilingual corpus of speech-to-...
research
∙
10/26/2022
Improving Speech-to-Speech Translation Through Unlabeled Text
Direct speech-to-speech translation (S2ST) is among the most challenging...
research
∙
10/18/2022
Simple and Effective Unsupervised Speech Translation
The amount of labeled data to train models for speech tasks is limited f...
research
∙
04/11/2022
Unified Speech-Text Pre-training for Speech Translation and Recognition
We describe a method to jointly pre-train speech and text in an encoder-...
research
∙
04/06/2022
Enhanced Direct Speech-to-Speech Translation Using Self-supervised Pre-training and Data Augmentation
Direct speech-to-speech translation (S2ST) models suffer from data scarc...
research
∙
12/15/2021
Textless Speech-to-Speech Translation on Real Data
We present a textless speech-to-speech translation (S2ST) system that ca...
research
∙
11/17/2021
XLS-R: Self-supervised Cross-lingual Speech Representation Learning at Scale
This paper presents XLS-R, a large-scale model for cross-lingual speech ...
research
∙
10/15/2021
Incremental Speech Synthesis For Speech-To-Speech Translation
In a speech-to-speech translation (S2ST) pipeline, the text-to-speech (T...
research
∙
09/14/2021
fairseq S^2: A Scalable and Integrable Speech Synthesis Toolkit
This paper presents fairseq S^2, a fairseq extension for speech synthesi...
research
∙
07/14/2021
FST: the FAIR Speech Translation System for the IWSLT21 Multilingual Shared Task
In this paper, we describe our end-to-end multilingual speech translatio...
research
∙
07/12/2021
Improving Speech Translation by Understanding and Learning from the Auxiliary Text Translation Task
Pretraining and multitask learning are widely used to improve the speech...
research
∙
07/12/2021
Direct speech-to-speech translation with discrete units
We present a direct speech-to-speech translation (S2ST) model that trans...
research
∙
06/02/2021
Lightweight Adapter Tuning for Multilingual Speech Translation
Adapter modules were recently introduced as an efficient alternative to ...
research
∙
04/14/2021
Large-Scale Self- and Semi-Supervised Learning for Speech Translation
In this paper, we improve speech translation (ST) through effectively le...
research
∙
01/02/2021
VoxPopuli: A Large-Scale Multilingual Speech Corpus for Representation Learning, Semi-Supervised Learning and Interpretation
We introduce VoxPopuli, a large-scale multilingual corpus providing 100K...
research
∙
11/16/2020
Facebook AI's WMT20 News Translation Task Submission
This paper describes Facebook AI's submission to WMT20 shared news trans...
research
∙
11/02/2020
Dual-decoder Transformer for Joint Automatic Speech Recognition and Multilingual Speech Translation
We introduce dual-decoder Transformer, a new model architecture that joi...
research
∙
10/24/2020
Cross-Modal Transfer Learning for Multilingual Speech-to-Text Translation
We propose an effective approach to utilize pretrained speech and text m...
research
∙
10/21/2020
A General Multi-Task Learning Framework to Leverage Text Data for Speech to Text Tasks
Attention-based sequence-to-sequence modeling provides a powerful and el...
research
∙
10/11/2020
fairseq S2T: Fast Speech-to-Text Modeling with fairseq
We introduce fairseq S2T, a fairseq extension for speech-to-text (S2T) m...
research
∙
07/31/2020
SimulEval: An Evaluation Toolkit for Simultaneous Translation
Simultaneous translation on both text and speech focuses on a real-time ...
research
∙
07/20/2020
CoVoST 2 and Massively Multilingual Speech-to-Text Translation
Speech translation has recently become an increasingly popular topic of ...
research
∙
06/22/2020
Self-Supervised Representations Improve End-to-End Speech Translation
End-to-end speech-to-text translation can provide a simpler and smaller ...
research
∙
06/09/2020
Improving Cross-Lingual Transfer Learning for End-to-End Speech Recognition with Speech Translation
Transfer learning from high-resource languages is known to be an efficie...
research
∙
02/04/2020
CoVoST: A Diverse Multilingual Speech-To-Text Translation Corpus
Spoken language translation has recently witnessed a resurgence in popul...
research
∙
09/12/2019
VizSeq: A Visual Analysis Toolkit for Text Generation Tasks
Automatic evaluation of text generation tasks (e.g. machine translation,...
research
∙
09/07/2019
Neural Machine Translation with Byte-Level Subwords
Almost all existing machine translation models are built on top of chara...
research
∙
06/06/2019
Does Object Recognition Work for Everyone?
The paper analyzes the accuracy of publicly available object-recognition...
research
∙
05/27/2019
Levenshtein Transformer
Modern neural sequence generation models are built to either generate to...
research
∙
04/21/2018