arXiv preprint ·
Geometry-Guided Semantic Reconstruction for 3D Instance Segmentation
- Published
- Source
- arXiv preprint, posted before peer review
- Original
- Read it at arXiv
- Topics
Abstract
3D instance segmentation requires representations that are both geometrically structured and semantically discriminative. Pure 3D encoders model spatial structure well but often lack strong appearance semantics, while directly lifting 2D priors into 3D by concatenation injects rich semantics without ensuring that they align well with the underlying 3D structure. In this paper, we address this problem by formulating 2D-to-3D semantic transfer as a \textbf{masked semantic reconstruction} task. Specifically, we mask a subset of background lifted 2D priors and train the 3D encoder to recover the missing semantic priors, providing explicit supervision for distilling rich 2D semantics into the 3D backbone. We further propose \textbf{geometry-modulated semantic injection}, which uses raw 3D geometry to modulate lifted 2D priors before sparse encoding, so that geometry explicitly controls semantic injection and alleviates the distribution gap between 2D semantic features and 3D geometric features. Experiments on two baseline frameworks demonstrate that the proposed method achieves state-of-the-art results in both full-data and few-shot settings on ScanNetV2 and ScanNet200. Code is publicly available at https://github.com/yharoldsu0627/Geometry_Guided_Semantic_Reconstruction
The authors’ abstract, as arXiv lists it. arXiv’s listings, abstracts included, are free for anyone to reuse.
Read it on ReadSearch
Each opens after you create an account, and brings you back to this paper.
More in Computer Vision
Meaningful Moments: A Corpus of VLM-Derived Temporal Importance Labels for Video Action Recognition
CÆNL: Collapse-Based Active Neural Learning with Objective-Aligned Representation Control
Adversarial Robustness of Camera-Radar Sensor Fusion for Autonomous Driving
Gaussian Splatting for Real-Time Scene Reconstruction: Methods, Systems, and Trade-offs
Removing the Compute Bottleneck for Event-based Visual Place Recognition with Spiking Neural Networks
Get the few papers worth your time in Computer Vision every morning.
ReadSearch reads the new papers in your field each day and brings you the few worth reading, each with its key finding in a sentence.