Point'n Move: Interactive scene object manipulation on Gaussian splatting radiance fields

Authors: Huang, J., Yu, H., Zhang, J. and Nait-Charif, H.

Journal: IET Image Processing

Volume: 18

Issue: 12

Pages: 3507-3517

eISSN: 1751-9667

ISSN: 1751-9659

DOI: 10.1049/ipr2.13190

Abstract:

The authors propose Point'n Move, a method that achieves interactive scene object manipulation with exposed region inpainting. Interactivity here further comes from intuitive object selection and real-time editing. To achieve this, Gaussian Splatting Radiance Field is adopted as the scene representation and its explicit nature and speed advantage are fully leveraged. Its explicit representation formulation allows to devise a 2D prompt points to 3D masks dual-stage self-prompting segmentation algorithm, perform mask refinement and merging, minimize changes, and provide good initialization for scene inpainting and perform editing in real-time without per-editing training; all lead to superior quality and performance. The method was tested by editing both forward-facing and 360 scenes. The method is also compared against existing methods, showing superior quality despite being more capable and having a speed advantage.

Source: Scopus