Thomas Nestmeyer

Alumni

Perceiving Systems

Education

2014 - now: Ph.D. Student
Max Planck Institute for Intelligent Systems, Perceiving Systems Department, Tübingen, Germany

2004 – 2013: Diploma Student
Computer Science and Mathematics - Eberhard-Karls-Universität Tübingen, Germany

2013 Diploma in Computer Science from the Eberhard-Karls-Universität Tübingen, Germany
Diploma Thesis "Decentralized Multi-target Exploration and Connectivity Maintenance with a Multi-robot System" at the Max Planck Institute for Biological Cybernetics, Tübingen, Germany
2010 Visiting Student
Computer Science - University of Otago, Dunedin, New Zealand
2007 Pre-Diploma in Mathematics from the Eberhard-Karls-Universität Tübingen, Germany

2003 - 2004: Military Service
German Air Force in Roth and Landsberg, Germany

1994 – 2003: High School Student
Gymnasium Neckartenzlingen in Neckartenzlingen, Germany

Experience

2018: Research Intern at Facebook, Pittsburgh, USA
Continuing to work on intrinsic images in the Oculus Research Team.

2017-2018: Teaching Assistant at the Julius-Maximilians Universität Würzburg, Germany
Summer term: Probabilistic Inference; Winter term: Algorithms and Data Structures

2012 - 2014: Research Assistant and later Research Associate at the Max Planck Institute for Biological Cybernetics, Tübingen, Germany
Working in the field of multiple robot systems on "Decentralized Simultaneous Multi-target Exploration with Continual Connectivity Maintenance"

2005 - 2012: Teaching Assistant at the Eberhard-Kars-Universität Tübingen, Germany
Computer Engineering (2 semesters), Mathematics for Computer Scientists (4 semesters), Theoretical Computer Science (2 semesters), Algebraical and Combinatorial Applications in Computer Science (1 semester), Pre-Course Mathematics (2 weeks per course, 6 courses)

2006 - 2011: Research Assistant at the Eberhard-Kars-Universität Tübingen, Germany
Chairs of Computer Engineering (1 year), Discrete Mathematics (1 year), Formal Languages (0.5 years)

Research Interest

I work on inferring physical properties from images, specifically decomposing them into their intrinsic layers, meaning mostly its reflectance and shading components. Knowing this information is beneficial in many computer vision areas. Especially when working on Augmented Reality (AR), this is an essential intermediate step.

Intrinsic Properties of Scenes

Intrinsic images correspond to physical properties of the scene. It is a long-standing hypothesis that these fundamental scene properties provide a foundation for scene interpretation.

To decouple albedo and shading given a single image, we introduce a novel prior on albedo, that models albedo values as being drawn fr...

Naejin Kong Martin Kiefel Peter Vincent Gehler Michael Black Thomas Nestmeyer

Reflectance Filtering

This is the project page for the paper "Reflectance Adaptive Filtering Improves Intrinsic Image Estimation" \cite{nestmeyer2017reflectanceFiltering}, which appears at CVPR 2017 and can be found at https://arxiv.org/abs/1612.05062.

...

Thomas Nestmeyer Peter Vincent Gehler

3 results (View BibTeX file of all listed publications)

2020

Learning Physics-guided Face Relighting under Directional Light

Nestmeyer, T., Lalonde, J., Matthews, I., Lehrmann, A. M.

In Conference on Computer Vision and Pattern Recognition, pages: 5123-5132, IEEE/CVF, June 2020 (inproceedings) Accepted

Abstract

Relighting is an essential step in realistically transferring objects from a captured image into another environment. For example, authentic telepresence in Augmented Reality requires faces to be displayed and relit consistent with the observer's scene lighting. We investigate end-to-end deep learning architectures that both de-light and relight an image of a human face. Our model decomposes the input image into intrinsic components according to a diffuse physics-based image formation model. We enable non-diffuse effects including cast shadows and specular highlights by predicting a residual correction to the diffuse render. To train and evaluate our model, we collected a portrait database of 21 subjects with various expressions and poses. Each sample is captured in a controlled light stage setup with 32 individual light sources. Our method creates precise and believable relighting results and generalizes to complex illumination conditions and challenging poses, including when the subject is not looking straight at the camera.

Paper [BibTex]

2020

Nestmeyer, T., Lalonde, J., Matthews, I., Lehrmann, A. M. Learning Physics-guided Face Relighting under Directional Light In Conference on Computer Vision and Pattern Recognition, pages: 5123-5132, IEEE/CVF, June 2020 (inproceedings) Accepted

Paper [BibTex]

2017

Reflectance Adaptive Filtering Improves Intrinsic Image Estimation

Nestmeyer, T., Gehler, P. V.

In 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR), pages: 1771-1780, IEEE, Piscataway, NJ, USA, July 2017 (inproceedings)

pre-print DOI Project Page Project Page [BibTex]

2017

Nestmeyer, T., Gehler, P. V. Reflectance Adaptive Filtering Improves Intrinsic Image Estimation In 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR), pages: 1771-1780, IEEE, Piscataway, NJ, USA, July 2017 (inproceedings)

pre-print DOI Project Page Project Page [BibTex]