DFIE3D: 3D-Aware Disentangled Face Inversion and Editing Via Facial-contrastive Learning
Authors: Zhu, X., Zhou, J., You, L., Yang, X., Chang, J., Zhang, J.J. and Zeng, D.
Journal: IEEE Transactions on Circuits and Systems for Video Technology
eISSN: 1558-2205
ISSN: 1051-8215
DOI: 10.1109/TCSVT.2024.3377121
Abstract:Recent advances in NeRF-based 3D-aware GANs have achieved outstanding performance, especially in the realm of human facial representations, making projection of facial images back into their latent space superior and preferable compared to 2D GAN inversion. However, the direct application of 2DGAN inversion techniques to 3DGAN raises challenges due to potential appearance distortions and geometric inconsistences. To tackle these issues, this work presents a novel integrated framework that combines a composite inversion pipeline in both the SS and W+ spaces and integrates a contrastive-based training strategy, ensuring proficient disentanglement within the module. Moreover, we design a facial semantic manipulation technique based on dimensional analysis of the latent code, which is fully compatible with the proposed 3DGAN inversion pipeline. Comprehensive experimental validations substantiate the effectiveness of the proposed approach in executing 3d-aware face inversion and semantic editing tasks, presenting a robust technological solution for a diverse array of digital human modeling applications in the downstream.
Source: Scopus