Introducing INFAMOUS-NeRF: Enhancing Implicit Face Modeling and Rendering
In a breakthrough development, researchers have introduced INFAMOUS-NeRF, a cutting-edge morphable face model that combines hypernetworks with NeRF technology. This innovative approach significantly improves the representation power of implicit face modeling, particularly in scenarios involving multiple training subjects. Unlike previous methods, INFAMOUS-NeRF achieves a perfect balance between representation power and editability by learning semantically-aligned latent spaces without the need for a large pretrained model. Furthermore, INFAMOUS-NeRF incorporates a unique constraint that enhances NeRF rendering along the facial boundary, resulting in more realistic surface color prediction and improved rendering near the surface. To further enhance NeRF training, this groundbreaking model introduces a loss-guided adaptive sampling method, effectively reducing sampling redundancy. Through rigorous quantitative and qualitative analysis, it has been demonstrated that INFAMOUS-NeRF outperforms existing face modeling techniques in both controlled and in-the-wild settings. The researchers have announced that the code and models for INFAMOUS-NeRF will be made publicly available upon publication, allowing further exploration and applications of this groundbreaking technology.
Abstract:We propose INFAMOUS-NeRF, an implicit morphable face model that introduces hypernetworks to NeRF to improve the representation power in the presence of many training subjects. At the same time, INFAMOUS-NeRF resolves the classic hypernetwork tradeoff of representation power and editability by learning semantically-aligned latent spaces despite the subject-specific models, all without requiring a large pretrained model. INFAMOUS-NeRF further introduces a novel constraint to improve NeRF rendering along the face boundary. Our constraint can leverage photometric surface rendering and multi-view supervision to guide surface color prediction and improve rendering near the surface. Finally, we introduce a novel, loss-guided adaptive sampling method for more effective NeRF training by reducing the sampling redundancy. We show quantitatively and qualitatively that our method achieves higher representation power than prior face modeling methods in both controlled and in-the-wild settings. Code and models will be released upon publication.