research
∙
05/22/2023
Scaling Speech Technology to 1,000+ Languages
Expanding the language coverage of speech technology has the potential t...
share
research
∙
05/22/2023
Textually Pretrained Speech Language Models
Speech language models (SpeechLMs) process and generate acoustic data on...
research
∙
01/10/2023
Scaling Laws for Generative Mixed-Modal Language Models
Generative language models define distributions over sequences of tokens...
research
∙
05/25/2022
FLEURS: Few-shot Learning Evaluation of Universal Representations of Speech
We introduce FLEURS, the Few-shot Learning Evaluation of Universal Repre...
research
∙
03/24/2022
Leveraging unsupervised and weakly-supervised data to improve direct speech-to-speech translation
End-to-end speech-to-speech translation (S2ST) without relying on interm...
research
∙
03/21/2022
XTREME-S: Evaluating Cross-lingual Speech Representations
We introduce XTREME-S, a new benchmark to evaluate universal cross-lingu...
research
∙
02/03/2022
mSLAM: Massively multilingual joint pre-training for speech and text
We present mSLAM, a multilingual Speech and LAnguage Model that learns c...
research
∙
11/17/2021
XLS-R: Self-supervised Cross-lingual Speech Representation Learning at Scale
This paper presents XLS-R, a large-scale model for cross-lingual speech ...
research
∙
10/20/2021
SLAM: A Unified Encoder for Speech and Language Modeling via Speech-Text Joint Pre-Training
Unsupervised pre-training is now the predominant approach for both text ...
research
∙
07/08/2021
Improved Language Identification Through Cross-Lingual Self-Supervised Learning
Language identification greatly impacts the success of downstream tasks ...
research
∙
05/24/2021
Unsupervised Speech Recognition
Despite rapid progress in the recent past, current speech recognition sy...
research
∙
05/02/2021
Larger-Scale Transformers for Multilingual Masked Language Modeling
Recent work has demonstrated the effectiveness of cross-lingual language...
research
∙
04/14/2021
Large-Scale Self- and Semi-Supervised Learning for Speech Translation
In this paper, we improve speech translation (ST) through effectively le...
research
∙
11/03/2020
Supervised Contrastive Learning for Pre-trained Language Model Fine-tuning
State-of-the-art natural language understanding classification models fo...
research
∙
10/22/2020
Self-training and Pre-training are Complementary for Speech Recognition
Self-training and unsupervised pre-training have emerged as effective ap...
research
∙
10/05/2020
Self-training Improves Pre-training for Natural Language Understanding
Unsupervised pre-training has led to much recent progress in natural lan...
research
∙
06/24/2020
Unsupervised Cross-lingual Representation Learning for Speech Recognition
This paper presents XLSR which learns cross-lingual speech representatio...
research
∙
11/05/2019
Unsupervised Cross-lingual Representation Learning at Scale
This paper shows that pretraining multilingual language models at scale ...
research
∙
11/04/2019
Emerging Cross-lingual Structure in Pretrained Language Models
We study the problem of multilingual masked language modeling, i.e. the ...
research
∙
11/01/2019
CCNet: Extracting High Quality Monolingual Datasets from Web Crawl Data
Pre-training text representations have led to significant improvements i...
research
∙
01/22/2019
Cross-lingual Language Model Pretraining
Recent studies have demonstrated the efficiency of generative pretrainin...
research
∙
09/13/2018
XNLI: Evaluating Cross-lingual Sentence Representations
State-of-the-art natural language processing systems rely on supervision...
research
∙
05/03/2018
What you can cram into a single vector: Probing sentence embeddings for linguistic properties
Although much effort has recently been devoted to training high-quality ...
research
∙
04/20/2018
Phrase-Based & Neural Unsupervised Machine Translation
Machine translation systems achieve near human-level performance on some...
research
∙
03/14/2018
SentEval: An Evaluation Toolkit for Universal Sentence Representations
We introduce SentEval, a toolkit for evaluating the quality of universal...
research
∙
10/11/2017
Word Translation Without Parallel Data
State-of-the-art methods for learning cross-lingual word embeddings have...
research
∙
07/19/2017
Learning Visually Grounded Sentence Representations
We introduce a variety of models, trained on a supervised image captioni...
research
∙
05/05/2017
Supervised Learning of Universal Sentence Representations from Natural Language Inference Data
Many modern NLP systems rely on word embeddings, previously trained in a...
research
∙
07/25/2016
Meta-Prod2Vec - Product Embeddings Using Side-Information for Recommendation
We propose Meta-Prod2vec, a novel method to compute item similarities fo...
research
∙
06/06/2016