Text as Neural Operator: Image Manipulation by Text Instruction

08/11/2020
by   Tianhao Zhang, et al.
8

In this paper, we study a new task that allows users to edit an input image using language instructions. In this image generation task, the inputs are a reference image and a text instruction that describes desired modifications to the input image. We propose a GAN-based method to tackle this problem. The key idea is to treat language as neural operators to locally modify the image feature. To this end, our model decomposes the generation process into finding where (spatial region) and how (text operators) to apply modifications. We show that the proposed model performs favorably against recent baselines on three datasets.

READ FULL TEXT

Please sign up or login with your details

Forgot password? Click here to reset