HandDGCL: Two-hand 3D reconstruction based disturbing graph contrastive learning
Authors: Han, B., Yao, C., Wang, X., Chang, J. and Ban, X.
Journal: Computer Animation and Virtual Worlds
Volume: 34
Issue: 3-4
eISSN: 1546-427X
ISSN: 1546-4261
DOI: 10.1002/cav.2186
Abstract:Virtual reality (VR) and augmented reality (AR) applications are becoming increasingly prevalent. However, constructing realistic 3D hands, especially when two hands are interacting, from a single RGB image remains a major challenge due to severe mutual occlusion and the enormous diversity of hand poses. In this article, we propose a disturbing graph contrastive learning strategy for two-hand 3D reconstruction. This involves a graph disturbance network designed to generate graph feature pairs to enhance the consistency of the two-hand pose features. A contrastive learning module leverages high-quality generative features for a strong feature expression. We further propose a similarity distinguish method to divide positive and negative features for accelerating the model convergence. Additionally, a multi-term loss is designed to balance the relation among the hand pose, the visual scale and the viewpoint position. Our model has achieved state-of-the-art results in the InterHand2.6M benchmark. Ablation studies show the model's great ability to correct unreasonable hand movements. In subjective assessments, our graph disturbance learning method significantly improves the construction of realistic 3D hands, especially when two hands are interacting.
https://eprints.bournemouth.ac.uk/38863/
Source: Scopus
HandDGCL: Two-hand 3D reconstruction based disturbing graph contrastive learning
Authors: Han, B., Yao, C., Wang, X., Chang, J. and Ban, X.
Journal: COMPUTER ANIMATION AND VIRTUAL WORLDS
Volume: 34
Issue: 3-4
eISSN: 1546-427X
ISSN: 1546-4261
DOI: 10.1002/cav.2186
https://eprints.bournemouth.ac.uk/38863/
Source: Web of Science (Lite)
HandDGCL: Two-hand 3D reconstruction based disturbing graph contrastive learning
Authors: Han, B., Yao, C., Wang, X., Chang, J. and Ban, X.
Journal: Computer Animation and Virtual Worlds
Volume: 34
Issue: 3-4
ISSN: 1546-4261
Abstract:Virtual reality (VR) and augmented reality (AR) applications are becoming increasingly prevalent. However, constructing realistic 3D hands, especially when two hands are interacting, from a single RGB image remains a major challenge due to severe mutual occlusion and the enormous diversity of hand poses. In this article, we propose a disturbing graph contrastive learning strategy for two-hand 3D reconstruction. This involves a graph disturbance network designed to generate graph feature pairs to enhance the consistency of the two-hand pose features. A contrastive learning module leverages high-quality generative features for a strong feature expression. We further propose a similarity distinguish method to divide positive and negative features for accelerating the model convergence. Additionally, a multi-term loss is designed to balance the relation among the hand pose, the visual scale and the viewpoint position. Our model has achieved state-of-the-art results in the InterHand2.6M benchmark. Ablation studies show the model's great ability to correct unreasonable hand movements. In subjective assessments, our graph disturbance learning method significantly improves the construction of realistic 3D hands, especially when two hands are interacting.
https://eprints.bournemouth.ac.uk/38863/
Source: BURO EPrints