HeteroAvatar: Generation of Gaussian Head Avatars With Correct Geometry Using Hetero Rendering

Abstract

Head avatar reconstruction is a crucial research topic with wide-ranging applications. For this purpose, utilizing the promising technique of 3D Gaussian splatting significantly improves rendering quality. However, this method encounters challenges, such as imprecise geometry and noisy surfaces. To address these issues, we propose HeteroAvatar, a novel method for generating a Gaussian head avatar with correct geometry from a monocular video of a dynamic head. To achieve this, we introduce the hetero rendering method, which concurrently employs both 3D Gaussian rendering and mesh rendering techniques during training. This approach enables HeteroAvatar to accurately capture the head geometry with smooth surfaces while maintaining photorealistic rendering quality. Additionally, our approach is capable of generating a mesh-based head avatar, which ensures compatibility with a wide range of traditional graphics applications. Experimental results demonstrate that HeteroAvatar outperforms state-of-the-art methods in terms of reconstructed geometry quality and photorealistic rendering with fewer 3D Gaussians. Project page: https://felizang.github.io/HeteroAvatar.