Leveraging Text Data Using Hybrid Transformer-LSTM Based End-to-End ASR in Transfer Learning
share
In this work, we study leveraging extra text data to improve low-resource end-to-end ASR under cross-lingual transfer learning setting. To this end, we extend our prior work [1], and propose a hybrid Transformer-LSTM based architecture. This architecture not only takes advantage of the highly effective encoding capacity of the Transformer network but also benefits from extra text data due to the LSTM-based independent language model network. We conduct experiments on our in-house Malay corpus which contains limited labeled data and a large amount of extra text. Results show that the proposed architecture outperforms the previous LSTM-based architecture [1] by 24.2 relative word error rate (WER) when both are trained using limited labeled data. Starting from this, we obtain further 25.4 transfer learning from another resource-rich language. Moreover, we obtain additional 13.6 transferred model with the extra text data. Overall, our best model outperforms the vanilla Transformer ASR by 11.9 proposed hybrid architecture offers much faster inference compared to both LSTM and Transformer architectures.
READ FULL TEXT