research
∙
08/22/2023
SeamlessM4T-Massively Multilingual Multimodal Machine Translation
What does it take to create the Babel Fish, a tool that can help individ...
share
research
∙
07/17/2023
Multilingual Speech-to-Speech Translation into Multiple Target Languages
Speech-to-speech translation (S2ST) enables spoken communication between...
research
∙
06/01/2023
Exploration on HuBERT with Multiple Resolutions
Hidden-unit BERT (HuBERT) is a widely-used self-supervised learning (SSL...
research
∙
05/04/2023
Hybrid Transducer and Attention based Encoder-Decoder Modeling for Speech-to-Text Tasks
Transducer and Attention based Encoder-Decoder (AED) are two widely used...
research
∙
04/10/2023
Enhancing Speech-to-Speech Translation with Multiple TTS Targets
It has been known that direct speech-to-speech translation (S2ST) models...
research
∙
04/10/2023
ESPnet-ST-v2: Multipurpose Spoken Language Translation Toolkit
ESPnet-ST-v2 is a revamp of the open-source ESPnet-ST toolkit necessitat...
research
∙
03/01/2023
MuAViC: A Multilingual Audio-Visual Corpus for Robust Speech Recognition and Robust Speech-to-Text Translation
We introduce MuAViC, a multilingual audio-visual corpus for robust speec...
research
∙
01/27/2023
Pre-training for Speech Translation: CTC Meets Optimal Transport
The gap between speech and text modalities is a major challenge in speec...
research
∙
12/15/2022
UnitY: Two-pass Direct Speech-to-speech Translation with Discrete Units
Direct speech-to-speech translation (S2ST), in which all components can ...
research
∙
11/11/2022
Speech-to-Speech Translation For A Real-world Unwritten Language
We study speech-to-speech translation (S2ST) that translates speech from...
research
∙
11/08/2022
SpeechMatrix: A Large-Scale Mined Corpus of Multilingual Speech-to-Speech Translations
We present SpeechMatrix, a large-scale multilingual corpus of speech-to-...
research
∙
10/18/2022
Simple and Effective Unsupervised Speech Translation
The amount of labeled data to train models for speech tasks is limited f...
research
∙
04/11/2022
Unified Speech-Text Pre-training for Speech Translation and Recognition
We describe a method to jointly pre-train speech and text in an encoder-...
research
∙
04/06/2022
Enhanced Direct Speech-to-Speech Translation Using Self-supervised Pre-training and Data Augmentation
Direct speech-to-speech translation (S2ST) models suffer from data scarc...
research
∙
12/15/2021
Textless Speech-to-Speech Translation on Real Data
We present a textless speech-to-speech translation (S2ST) system that ca...
research
∙
11/17/2021
XLS-R: Self-supervised Cross-lingual Speech Representation Learning at Scale
This paper presents XLS-R, a large-scale model for cross-lingual speech ...
research
∙
10/15/2021
Direct simultaneous speech to speech translation
We present the first direct simultaneous speech-to-speech translation (S...
research
∙
10/15/2021
Incremental Speech Synthesis For Speech-To-Speech Translation
In a speech-to-speech translation (S2ST) pipeline, the text-to-speech (T...
research
∙
09/14/2021
fairseq S^2: A Scalable and Integrable Speech Synthesis Toolkit
This paper presents fairseq S^2, a fairseq extension for speech synthesi...
research
∙
07/14/2021
FST: the FAIR Speech Translation System for the IWSLT21 Multilingual Shared Task
In this paper, we describe our end-to-end multilingual speech translatio...
research
∙
07/12/2021
Improving Speech Translation by Understanding and Learning from the Auxiliary Text Translation Task
Pretraining and multitask learning are widely used to improve the speech...
research
∙
07/12/2021
Direct speech-to-speech translation with discrete units
We present a direct speech-to-speech translation (S2ST) model that trans...
research
∙
06/21/2021
Pay Better Attention to Attention: Head Selection in Multilingual and Multi-Domain Sequence Modeling
Multi-head attention has each of the attention heads collect salient inf...
research
∙
06/02/2021
Lightweight Adapter Tuning for Multilingual Speech Translation
Adapter modules were recently introduced as an efficient alternative to ...
research
∙
04/14/2021
Large-Scale Self- and Semi-Supervised Learning for Speech Translation
In this paper, we improve speech translation (ST) through effectively le...
research
∙
01/02/2021
VoxPopuli: A Large-Scale Multilingual Speech Corpus for Representation Learning, Semi-Supervised Learning and Interpretation
We introduce VoxPopuli, a large-scale multilingual corpus providing 100K...
research
∙
11/03/2020
SimulMT to SimulST: Adapting Simultaneous Text Translation to End-to-End Simultaneous Speech Translation
Simultaneous text translation and end-to-end speech translation have rec...
research
∙
11/02/2020
Dual-decoder Transformer for Joint Automatic Speech Recognition and Multilingual Speech Translation
We introduce dual-decoder Transformer, a new model architecture that joi...
research
∙
10/30/2020
Streaming Simultaneous Speech Translation with Augmented Memory Transformer
Transformer-based models have achieved state-of-the-art performance on s...
research
∙
10/24/2020
Cross-Modal Transfer Learning for Multilingual Speech-to-Text Translation
We propose an effective approach to utilize pretrained speech and text m...
research
∙
10/21/2020
A General Multi-Task Learning Framework to Leverage Text Data for Speech to Text Tasks
Attention-based sequence-to-sequence modeling provides a powerful and el...
research
∙
10/11/2020
fairseq S2T: Fast Speech-to-Text Modeling with fairseq
We introduce fairseq S2T, a fairseq extension for speech-to-text (S2T) m...
research
∙
07/31/2020
SimulEval: An Evaluation Toolkit for Simultaneous Translation
Simultaneous translation on both text and speech focuses on a real-time ...
research
∙
07/20/2020
CoVoST 2 and Massively Multilingual Speech-to-Text Translation
Speech translation has recently become an increasingly popular topic of ...
research
∙
06/22/2020
Self-Supervised Representations Improve End-to-End Speech Translation
End-to-end speech-to-text translation can provide a simpler and smaller ...
research
∙
06/09/2020
Improving Cross-Lingual Transfer Learning for End-to-End Speech Recognition with Speech Translation
Transfer learning from high-resource languages is known to be an efficie...
research
∙
06/03/2020
Self-Training for End-to-End Speech Translation
One of the main challenges for end-to-end speech translation is data sca...
research
∙
02/27/2020
SkinAugment: Auto-Encoding Speaker Conversions for Automatic Speech Translation
We propose autoencoding speaker conversion for training data augmentatio...
research
∙
02/04/2020
CoVoST: A Diverse Multilingual Speech-To-Text Translation Corpus
Spoken language translation has recently witnessed a resurgence in popul...
research
∙
09/26/2019
Monotonic Multihead Attention
Simultaneous machine translation models start generating a target sequen...
research
∙
09/14/2019
Harnessing Indirect Training Data for End-to-End Automatic Speech Translation: Tricks of the Trade
For automatic speech translation (AST), end-to-end approaches are outper...
research
∙
09/14/2019
Leveraging Out-of-Task Data for End-to-End Automatic Speech Translation
For automatic speech translation (AST), end-to-end approaches are outper...
research
∙
06/27/2019
Findings of the First Shared Task on Machine Translation Robustness
We share the findings of the first shared task on improving robustness o...
research
∙
02/04/2019