Perceiving Systems, Computer Vision

Generating 3D Faces using Convolutional Mesh Autoencoders

2018

Conference Paper

ps


Learned 3D representations of human faces are useful for computer vision problems such as 3D face tracking and reconstruction from images, as well as graphics applications such as character generation and animation. Traditional models learn a latent representation of a face using linear subspaces or higher-order tensor generalizations. Due to this linearity, they can not capture extreme deformations and non-linear expressions. To address this, we introduce a versatile model that learns a non-linear representation of a face using spectral convolutions on a mesh surface. We introduce mesh sampling operations that enable a hierarchical mesh representation that captures non-linear variations in shape and expression at multiple scales within the model. In a variational setting, our model samples diverse realistic 3D faces from a multivariate Gaussian distribution. Our training data consists of 20,466 meshes of extreme expressions captured over 12 different subjects. Despite limited training data, our trained model outperforms state-of-the-art face models with 50% lower reconstruction error, while using 75% fewer parameters. We also show that, replacing the expression space of an existing state-of-the-art face model with our autoencoder, achieves a lower reconstruction error. Our data, model and code are available at http://coma.is.tue.mpg.de/.

Author(s): Anurag Ranjan and Timo Bolkart and Soubhik Sanyal and Michael J. Black
Book Title: European Conference on Computer Vision (ECCV)
Volume: Lecture Notes in Computer Science, vol 11207
Pages: 725--741
Year: 2018
Month: September
Publisher: Springer, Cham

Department(s): Perceiving Systems
Research Project(s): Learning Deep Representations of 3D
Bibtex Type: Conference Paper (inproceedings)
Paper Type: Conference

DOI: 10.1007/978-3-030-01219-9_43
Event Place: Munich, Germany

Links: Code (tensorflow)
Code (pytorch)
Project Page
Attachments: paper
supplementary

BibTex

@inproceedings{COMA:ECCV2018,
  title = {Generating {3D} Faces using Convolutional Mesh Autoencoders},
  author = {Ranjan, Anurag and Bolkart, Timo and Sanyal, Soubhik and Black, Michael J.},
  booktitle = {European Conference on Computer Vision (ECCV)},
  volume = {Lecture Notes in Computer Science, vol 11207},
  pages = {725--741},
  publisher = {Springer, Cham},
  month = sep,
  year = {2018},
  doi = {10.1007/978-3-030-01219-9_43},
  month_numeric = {9}
}