Marker-based optical motion capture (mocap) systems are intrusive and restrict motions to controlled laboratory spaces. Therefore, simple daily activities like biking, or having coffee with friends cannot be recorded with such systems. An alternative method would be motion capture from images [ ], however such methods are still not accurate enough. To address these issues and to be able to record human motion in everyday natural situations we apply systems based on Inertial Measurement Units (IMUs), that can track the human pose without cameras which makes them more suitable for outdoor recordings.
Existing IMU sytems require considerable number of sensors, worn on the body or attached to a suit. In Sparse Inertial Poser (SIP) [ ], we present a method to recover the full 3D human pose from only 6 IMUs, measuring orientation and acceleration are attached to the wrists, lower legs, waist and head, resulting in a minimally intrusive solution to capture human activities.
SIP [ ] gives an offline non-intrusive mocap system that can be used in unconstrained settings of daily life activities. In Deep Inertial Poser (DIP) [ ], we go beyound accuracy of SIP and further make it real time. To this end, we synthesize a large IMU dataset and leverage that to learn a deep recurrent regressor to get SMPL pose parameters in realtime from 6 IMU sensor recordings.
To fuse advances in IMU mocap systems, with monocular methods we introduce VIP [ ] that combines IMUs and a moving camera, to robustly recover human pose and shape in challenging scenes. Using VIP, we collected the 3DPW dataset, that includes videos of humans in challenging scenes with accurate 3D parameters that will provide the means to quantitatively evaluate monocular methods in difficult scenes and stimulate new research in this area.