Corrigendum to Improve Language Modelling for Code Completion through Learning General Token Repetition of Source Code
share
This paper is written because I receive several inquiry emails saying it is hard to achieve good results when applying token repetition learning techniques. If REP (proposed by me) or Pointer-Mixture (proposed by Jian Li) is directly applied to source code to decide all token repetitions, the performance will decrease sharply. Actually, as presented in Pointer-Mixture, there are many kinds of tokens that do not need to learn repetition patterns. For example, the tokens represent the grammar in Abstract Syntax Tree (AST) shows no obvious regularity of repetition. As I have also mentioned in the abstract section and experiment section, the REP model is only good at predicting unseen variables or unseen types in templates. In implementation, we concentrate on predicting unseen variables using REP. The variable-tokens and non-variable-tokens (grammar tokens or string literals) are treated differently. REP ignores tokens which are not variables. Because we predict token based on AST in pre-order, we can easily know the place currently being code-completed should be a Variable or a MethodInvocation or a StringLiteral. This important implementation trick is not clearly presented in the paper which may confuse readers when they reproduce the experiments. When computing accuracy, some kind of tokens such as grammar tokens are also ignored. Thus, in this paper, we correct some mistakes, clarify some confusing content, supplement the important implementation optimization details and provide a standard method for computing accuracy on Java benchmark for the already published papers.
READ FULL TEXT