Perceiving Systems, Computer Vision

Semantic Multi-view Stereo: Jointly Estimating Objects and Voxels

2017

Conference Paper

avg

ps


Dense 3D reconstruction from RGB images is a highly ill-posed problem due to occlusions, textureless or reflective surfaces, as well as other challenges. We propose object-level shape priors to address these ambiguities. Towards this goal, we formulate a probabilistic model that integrates multi-view image evidence with 3D shape information from multiple objects. Inference in this model yields a dense 3D reconstruction of the scene as well as the existence and precise 3D pose of the objects in it. Our approach is able to recover fine details not captured in the input shapes while defaulting to the input models in occluded regions where image evidence is weak. Due to its probabilistic nature, the approach is able to cope with the approximate geometry of the 3D models as well as input shapes that are not present in the scene. We evaluate the approach quantitatively on several challenging indoor and outdoor datasets.

Author(s): Ali Osman Ulusoy and Michael J. Black and Andreas Geiger
Book Title: Proceedings IEEE Conference on Computer Vision and Pattern Recognition (CVPR) 2017
Pages: 4531-4540
Year: 2017
Month: July
Day: 21-26
Publisher: IEEE

Department(s): Autonomous Vision, Perceiving Systems
Research Project(s): Multi-view Stereo
Deep, Probabilistic and Semantic 3D Reconstruction
Bibtex Type: Conference Paper (inproceedings)

Event Name: IEEE Conference on Computer Vision and Pattern Recognition (CVPR)
Event Place: Honolulu, HI, USA

Address: Piscataway, NJ, USA
ISBN: 978-1-5386-0457-1
ISSN: 1063-6919

Links: YouTube
pdf
suppmat
Video:

BibTex

@inproceedings{Ulusoy2017CVPR,
  title = {Semantic Multi-view Stereo: Jointly Estimating Objects and Voxels},
  author = {Ulusoy, Ali Osman and Black, Michael J. and Geiger, Andreas},
  booktitle = {Proceedings IEEE Conference on Computer Vision and Pattern Recognition (CVPR) 2017},
  pages = {4531-4540},
  publisher = {IEEE},
  address = {Piscataway, NJ, USA},
  month = jul,
  year = {2017},
  doi = {},
  month_numeric = {7}
}