Incremental Skip-gram Model with Negative Sampling

04/13/2017
by   Nobuhiro Kaji, et al.
0

This paper explores an incremental training strategy for the skip-gram model with negative sampling (SGNS) from both empirical and theoretical perspectives. Existing methods of neural word embeddings, including SGNS, are multi-pass algorithms and thus cannot perform incremental model update. To address this problem, we present a simple incremental extension of SGNS and provide a thorough theoretical analysis to demonstrate its validity. Empirical experiments demonstrated the correctness of the theoretical analysis as well as the practical usefulness of the incremental algorithm.

READ FULL TEXT

Please sign up or login with your details

Forgot password? Click here to reset