Repeat after me: Self-supervised learning of acoustic-to-articulatory mapping by vocal imitation

04/05/2022
by   Marc-Antoine Georges, et al.
0

We propose a computational model of speech production combining a pre-trained neural articulatory synthesizer able to reproduce complex speech stimuli from a limited set of interpretable articulatory parameters, a DNN-based internal forward model predicting the sensory consequences of articulatory commands, and an internal inverse model based on a recurrent neural network recovering articulatory commands from the acoustic speech input. Both forward and inverse models are jointly trained in a self-supervised way from raw acoustic-only speech data from different speakers. The imitation simulations are evaluated objectively and subjectively and display quite encouraging performances.

READ FULL TEXT

Please sign up or login with your details

Forgot password? Click here to reset