Full-body avatars from single images and textual guidance (Talk)

The reconstruction of full body appearance of clothed humans from single-view RGB images is a crucial yet challenging task, primarily due to depth ambiguities and the absence of observations from unseen regions. While existing methods have shown impressive results, they still suffer from limitations such as over-smooth surfaces and blurry textures, particularly lacking details at the backside of the avatar. In this talk, I will delve into how we have addressed these limitations by leveraging text guidance and pretrained text-image models, introducing two novel methods. Firstly, I will present ELICIT, a data-efficient approach that utilizes a SMPL-based human body prior and CLIP-based semantic prior to create an animatable human nerf from a single image. This method tackles the challenges of creating detailed back-side appearance by a CLIP embedding loss. Secondly, I will introduce TeCH, our latest project for reconstructing high-fidelity 3D clothed humans with consistent texture maps and detailed geometry. This approach employs a hybrid mesh representation and pretrained 2D text-to-image diffusion models to achieve remarkable results. Through these advancements, we aim to push the boundaries of creating digital human, bridging the gap between single-image inputs and the creation of fully textured and realistic 3D avatars.