Perceiving Systems, Computer Vision

GIF: Generative Interpretable Faces

2020

Conference Paper

ps


Photo-realistic visualization and animation of expressive human faces have been a long standing challenge. 3D face modeling methods provide parametric control but generates unrealistic images, on the other hand, generative 2D models like GANs (Generative Adversarial Networks) output photo-realistic face images, but lack explicit control. Recent methods gain partial control, either by attempting to disentangle different factors in an unsupervised manner, or by adding control post hoc to a pre-trained model. Unconditional GANs, however, may entangle factors that are hard to undo later. We condition our generative model on pre-defined control parameters to encourage disentanglement in the generation process. Specifically, we condition StyleGAN2 on FLAME, a generative 3D face model. While conditioning on FLAME parameters yields unsatisfactory results, we find that conditioning on rendered FLAME geometry and photometric details works well. This gives us a generative 2D face model named GIF (Generative Interpretable Faces) that offers FLAME's parametric control. Here, interpretable refers to the semantic meaning of different parameters. Given FLAME parameters for shape, pose, expressions, parameters for appearance, lighting, and an additional style vector, GIF outputs photo-realistic face images. We perform an AMT based perceptual study to quantitatively and qualitatively evaluate how well GIF follows its conditioning. The code, data, and trained model are publicly available for research purposes at http://gif.is.tue.mpg.de

Author(s): Partha Ghosh and Pravir Singh Gupta and Roy Uziel and Anurag Ranjan and Michael J. Black and Timo Bolkart
Book Title: 2020 International Conference on 3D Vision (3DV 2020)
Volume: 1
Pages: 868--878
Year: 2020
Month: November
Publisher: IEEE

Department(s): Perceiving Systems
Research Project(s): Faces and Expressions
Neural Rendering
Bibtex Type: Conference Paper (inproceedings)
Paper Type: Conference

DOI: 10.1109/3DV50981.2020.00097
Event Name: International Conference on 3D Vision (3DV 2020)
Event Place: Fukuoka

Address: Piscataway, NJ
ISBN: 978-1-7281-8129-5
State: Published

Links: pdf
project
code
video
Video:

BibTex

@inproceedings{GIF:3DV:2020,
  title = {{GIF}: Generative Interpretable Faces},
  author = {Ghosh, Partha and Gupta, Pravir Singh and Uziel, Roy and Ranjan, Anurag and Black, Michael J. and Bolkart, Timo},
  booktitle = {2020 International Conference on 3D Vision (3DV 2020)},
  volume = {1},
  pages = {868--878},
  publisher = {IEEE},
  address = {Piscataway, NJ},
  month = nov,
  year = {2020},
  doi = {10.1109/3DV50981.2020.00097},
  month_numeric = {11}
}