Improving Cross-Domain Chinese Word Segmentation with Word Embeddings
share
Cross-domain Chinese Word Segmentation (CWS) remains a challenge despite recent progress in neural-based CWS. The limited amount of annotated data in the target domain has been the key obstacle to a satisfactory performance. In this paper, we propose a semi-supervised word-based approach to improving cross-domain CWS given a baseline segmenter. Particularly, our model only deploys word embeddings trained on raw text in the target domain, discarding complex hand-crafted features and domain-specific dictionaries. Innovative subsampling and negative sampling methods are proposed to derive word embeddings optimized for CWS. We conduct experiments on five datasets in special domains, covering domains in novels, medicine, and patent. Results show that our model can significantly improve cross-domain CWS, especially in the segmentation of domain-specific noun entities. The word F-measure increases by over 3.0 unsupervised cross-domain CWS approaches with a large margin.
READ FULL TEXT