Perceiving Systems, Computer Vision

Detailed Full-Body Reconstructions of Moving People from Monocular RGB-D Sequences

2015

Conference Paper

ps


We accurately estimate the 3D geometry and appearance of the human body from a monocular RGB-D sequence of a user moving freely in front of the sensor. Range data in each frame is first brought into alignment with a multi-resolution 3D body model in a coarse-to-fine process. The method then uses geometry and image texture over time to obtain accurate shape, pose, and appearance information despite unconstrained motion, partial views, varying resolution, occlusion, and soft tissue deformation. Our novel body model has variable shape detail, allowing it to capture faces with a high-resolution deformable head model and body shape with lower-resolution. Finally we combine range data from an entire sequence to estimate a high-resolution displacement map that captures fine shape details. We compare our recovered models with high-resolution scans from a professional system and with avatars created by a commercial product. We extract accurate 3D avatars from challenging motion sequences and even capture soft tissue dynamics.

Author(s): Bogo, Federica and Black, Michael J. and Loper, Matthew and Romero, Javier
Book Title: International Conference on Computer Vision (ICCV)
Pages: 2300--2308
Year: 2015
Month: December

Department(s): Perceiving Systems
Research Project(s): Beyond Motion Capture
Bodies from RGB-D
Bibtex Type: Conference Paper (inproceedings)
Paper Type: Conference

Links: Video
Video:
Attachments: pdf

BibTex

@inproceedings{Bogo:ICCV:2015,
  title = {Detailed Full-Body Reconstructions of Moving People from Monocular {RGB-D} Sequences},
  author = {Bogo, Federica and Black, Michael J. and Loper, Matthew and Romero, Javier},
  booktitle = {International Conference on Computer Vision (ICCV)},
  pages = {2300--2308},
  month = dec,
  year = {2015},
  doi = {},
  month_numeric = {12}
}