The human body is special. Most images and videos are of humans and understanding them is important for many problems including human-computer interaction, video retrieval, activity recognition, special effects, sports medicine, etc. We take an approach that leverages strong models to interpret ambiguous sensor data. Such models express the statistics of the world and allow the robust integration of measurements. A model-based approach is particularly important for the analysis of complex, articulated, and non-rigid objects such as the body.
To that end we have built the most detailed and accurate statistical models of 3D human body shape to date. These models are learned from over 4000 3D body scans of different people and approximately 1800 scans capturing a wide range of poses by people of many body shapes.
Additionally we learn models of soft tissue dynamics using 40,000 scans captured by a unique full-body 4D body scanner that gives detailed 3D meshes at 60 fps.
Our latest SMPL body model is available for research purposes, makes it easy to create any human body shape and allows the body to be animated in standard game engines and graphics software. The model is appropriate for use in animation and computer vision.
Some of our current work addresses
- learning models of clothing in motion
- hands, faces, and bodies moving together
- compositional models of shape
- animal shape and motion
- estimating 3D motion and shape from monocular cues