Structure-Preserving 3D Garment Modeling with Neural Sewing Machines

by   Xipeng Chen, et al.

3D Garment modeling is a critical and challenging topic in the area of computer vision and graphics, with increasing attention focused on garment representation learning, garment reconstruction, and controllable garment manipulation, whereas existing methods were constrained to model garments under specific categories or with relatively simple topologies. In this paper, we propose a novel Neural Sewing Machine (NSM), a learning-based framework for structure-preserving 3D garment modeling, which is capable of learning representations for garments with diverse shapes and topologies and is successfully applied to 3D garment reconstruction and controllable manipulation. To model generic garments, we first obtain sewing pattern embedding via a unified sewing pattern encoding module, as the sewing pattern can accurately describe the intrinsic structure and the topology of the 3D garment. Then we use a 3D garment decoder to decode the sewing pattern embedding into a 3D garment using the UV-position maps with masks. To preserve the intrinsic structure of the predicted 3D garment, we introduce an inner-panel structure-preserving loss, an inter-panel structure-preserving loss, and a surface-normal loss in the learning process of our framework. We evaluate NSM on the public 3D garment dataset with sewing patterns with diverse garment shapes and categories. Extensive experiments demonstrate that the proposed NSM is capable of representing 3D garments under diverse garment shapes and topologies, realistically reconstructing 3D garments from 2D images with the preserved structure, and accurately manipulating the 3D garment categories, shapes, and topologies, outperforming the state-of-the-art methods by a clear margin.


page 2

page 4

page 6

page 9

page 10

page 14

page 15


DeepCloth: Neural Garment Representation for Shape and Style Editing

Garment representation, animation and editing is a challenging topic in ...

Learning Anchored Unsigned Distance Functions with Gradient Direction Alignment for Single-view Garment Reconstruction

While single-view 3D reconstruction has made significant progress benefi...

Topology-Preserving Shape Reconstruction and Registration via Neural Diffeomorphic Flow

Deep Implicit Functions (DIFs) represent 3D geometry with continuous sig...

NeuralTailor: Reconstructing Sewing Pattern Structures from 3D Point Clouds of Garments

The fields of SocialVR, performance capture, and virtual try-on are ofte...

NeuRIS: Neural Reconstruction of Indoor Scenes Using Normal Priors

Reconstructing 3D indoor scenes from 2D images is an important task in m...

Discovering Pattern Structure Using Differentiable Compositing

Patterns, which are collections of elements arranged in regular or near-...

RESTORE: Graph Embedding Assessment Through Reconstruction

Following the success of Word2Vec embeddings, graph embeddings (GEs) hav...

Please sign up or login with your details

Forgot password? Click here to reset