Classifiers are Better Experts for Controllable Text Generation

05/15/2022
by   Askhat Sitdikov, et al.
3

This paper proposes a simple method for controllable text generation based on weighting logits produced, namely CAIF sampling. Using an arbitrary third-party text classifier, we adjust a small part of a language model's logits and guide text generation towards or away from classifier prediction. We show that the proposed method significantly outperforms recent PPLM, GeDi, and DExperts on PPL and sentiment accuracy based on the external classifier of generated texts. A the same time, it is also easier to implement and tune, and has significantly fewer restrictions and requirements.

READ FULL TEXT

Please sign up or login with your details

Forgot password? Click here to reset