Perceiving Systems, Computer Vision

Learning to Dress 3D People in Generative Clothing

2020

Conference Paper

ps


Three-dimensional human body models are widely used in the analysis of human pose and motion. Existing models, however, are learned from minimally-clothed 3D scans and thus do not generalize to the complexity of dressed people in common images and videos. Additionally, current models lack the expressive power needed to represent the complex non-linear geometry of pose-dependent clothing shape. To address this, we learn a generative 3D mesh model of clothed people from 3D scans with varying pose and clothing. Specifically, we train a conditional Mesh-VAE-GAN to learn the clothing deformation from the SMPL body model, making clothing an additional term on SMPL. Our model is conditioned on both pose and clothing type, giving the ability to draw samples of clothing to dress different body shapes in a variety of styles and poses. To preserve wrinkle detail, our Mesh-VAE-GAN extends patchwise discriminators to 3D meshes. Our model, named CAPE, represents global shape and fine local structure, effectively extending the SMPL body model to clothing. To our knowledge, this is the first generative model that directly dresses 3D human body meshes and generalizes to different poses.

Author(s): Qianli Ma and Jinlong Yang and Anurag Ranjan and Sergi Pujades and Gerard Pons-Moll and Siyu Tang and Michael J. Black
Book Title: 2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR 2020)
Pages: 6468--6477
Year: 2020
Month: June
Publisher: IEEE

Department(s): Perceiving Systems
Research Project(s): Clothing Capture and Modeling
Bibtex Type: Conference Paper (inproceedings)
Paper Type: Conference

DOI: 10.1109/CVPR42600.2020.00650
Event Name: IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR 2020)
Event Place: Seattle, WA, USA

Address: Piscataway, NJ
ISBN: 978-1-7281-7168-5
State: Published
URL: https://cape.is.tue.mpg.de

Links: Project page
Code
Long video
Short Video
arXiv
Video:
Video:

BibTex

@inproceedings{CAPE:CVPR:20,
  title = {Learning to Dress {3D} People in Generative Clothing},
  author = {Ma, Qianli and Yang, Jinlong and Ranjan, Anurag and Pujades, Sergi and Pons-Moll, Gerard and Tang, Siyu and Black, Michael J.},
  booktitle = {2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR 2020)},
  pages = {6468--6477},
  publisher = {IEEE},
  address = {Piscataway, NJ},
  month = jun,
  year = {2020},
  doi = {10.1109/CVPR42600.2020.00650},
  url = {https://cape.is.tue.mpg.de},
  month_numeric = {6}
}