Wav2Vec-Aug: Improved self-supervised training with limited data

06/27/2022
by   Anuroop Sriram, et al.
0

Self-supervised learning (SSL) of speech representations has received much attention over the last few years but most work has focused on languages and domains with an abundance of unlabeled data. However, for many languages there is a shortage even in the unlabeled data which limits the effectiveness of SSL. In this work, we focus on the problem of applying SSL to domains with limited available data by leveraging data augmentation for Wav2Vec 2.0 pretraining. Further, we propose improvements to each component of the model which result in a combined relative word error rate (WER) improvement of up to 13 Wav2Vec 2.0 on Librispeech test-clean / other.

READ FULL TEXT

Please sign up or login with your details

Forgot password? Click here to reset