ReGen3D: Generalizable Unified Representation Learning for 3D Understanding
Authors: Zhang, H., Jiang, J., Li, Y., Zhang, J., Wang, M., Chang, J.
Journal: Lecture Notes in Computer Science
Publication Date: 01/01/2026
Volume: 17032 LNCS
Pages: 471-490
eISSN: 1611-3349
ISSN: 0302-9743
DOI: 10.1007/978-3-032-37432-5_27
Abstract:Existing 3D understanding models are typically built on a fixed geometric representation and often fail when the representation format changes. Unlike domain generalization, which studies distribution shift under a fixed input structure, we focus on representation shift, where identical 3D semantics are expressed through heterogeneous geometric encodings such as points, voxels, meshes, and 3D Gaussian primitives. As 3D representations continue to evolve beyond a predefined set, this challenge is becoming increasingly important. We formalize this problem as Representation Generalization (RG), a new setting for unified 3D learning. RG requires a model to share knowledge across representations during training, remain effective when only a single representation is available at test time, and extend efficiently to unseen or newly emerging representations. To this end, we propose ReGen3D, a unified framework with three modules: Unified Representation Tokenization (URT), which maps heterogeneous representations into a shared token space; Shared Memory Interaction (SMI), which enables cross-representation learning during training while supporting inference from any single available representation; and Efficient Representation Adapter (ERA), which supports lightweight adaptation to unseen representations without modifying the shared backbone. We further establish RepShift-3D for systematic evaluation under the RG setting. Extensive experiments on classification, segmentation, and reconstruction show that ReGen3D achieves strong and robust performance across diverse representation scenarios.
Source: Scopus