DG-PIC: Domain Generalized Point-In-Context Learning for Point Cloud Understanding
Authors: Jiang, J., Zhou, Q., Li, Y., Lu, X., Wang, M., Ma, L., Chang, J. and Zhang, J.J.
Conference: The 18th European Conference on Computer Vision
Dates: 29 September-4 October 2024
Abstract:Recent point cloud understanding research suffers from performance drops on unseen data, due to the distribution shifts across different domains. While recent studies use Domain Generalization (DG) techniques to mitigate this by learning domain-invariant features, most are designed for a single task and neglect the potential of testing data. Despite In-Context Learning (ICL) showcasing multi-task learning capability, it usually relies on high-quality context-rich data and considers a single dataset, and has rarely been studied in point cloud understanding. In this paper, we introduce a novel, practical, multi-domain multi-task setting, handling multiple domains and multiple tasks within one unified model for domain generalized point cloud understanding. To this end, we propose Domain Generalized Point-In-Context Learning (DG-PIC) that boosts the generalization ability across various tasks and domains at testing time. In particular, we develop dual-level source prototype estimation that considers both global-level shape contextual and local-level geometrical structures for representing source domains and a dual-level test-time feature shifting mechanism that leverages both macro-level domain semantic information and micro-level patch positional relationships to pull the target data closer to the source ones during testing. Our DG-PIC does not require any model updates during testing and can handle unseen domains and multiple tasks, i.e., point cloud reconstruction, denoising, and registration, within one unified model. We also introduce a benchmark for this new setting. Comprehensive experiments demonstrate that DG-PIC outperforms state-of-the-art techniques significantly. Our code and benchmark are available at: https://github.com/Jinec98/DG-PIC.
Source: Manual