Neural Puppet: Generative Layered Cartoon Characters

by   Omid Poursaeed, et al.

We propose a learning based method for generating new animations of a cartoon character given a few example images. Our method is designed to learn from a traditionally animated sequence, where each frame is drawn by an artist, and thus the input images lack any common structure, correspondences, or labels. We express pose changes as a deformation of a layered 2.5D template mesh, and devise a novel architecture that learns to predict mesh deformations matching the template to a target image. This enables us to extract a common low-dimensional structure from a diverse set of character poses. We combine recent advances in differentiable rendering as well as mesh-aware models to successfully align common template even if only a few character images are available during training. In addition to coarse poses, character appearance also varies due to shading, out-of-plane motions, and artistic effects. We capture these subtle changes by applying an image translation network to refine the mesh rendering, providing an end-to-end model to generate new animations of a character with high visual quality. We demonstrate that our generative model can be used to synthesize in-between frames and to create data-driven deformation. Our template fitting procedure outperforms state-of-the-art generic techniques for detecting image correspondences.


page 5

page 7

page 8

page 13

page 14

page 15

page 16

page 17


CTSN: Predicting Cloth Deformation for Skeleton-based Characters with a Two-stream Skinning Network

We present a novel learning method to predict the cloth deformation for ...

VINECS: Video-based Neural Character Skinning

Rigging and skinning clothed human avatars is a challenging task and tra...

Learning 3D Deformation of Animals from 2D Images

Understanding how an animal can deform and articulate is essential for a...

Collaborative Neural Rendering using Anime Character Sheets

Drawing images of characters at desired poses is an essential but labori...

Interactive 3D Character Modeling from 2D Orthogonal Drawings with Annotations

We propose an interactive 3D character modeling approach from orthograph...

Zero-calibration cVEP BCI using word prediction: a proof of concept

Brain Computer Interfaces (BCIs) based on visual evoked potentials (VEP)...

CharacterGAN: Few-Shot Keypoint Character Animation and Reposing

We introduce CharacterGAN, a generative model that can be trained on onl...

Please sign up or login with your details

Forgot password? Click here to reset