GNFormer: Gaussian Neural Field Transformer for Efficient Camera-based Occupancy Prediction

Abstract

Camera-based occupancy prediction provides geometric and semantic scene understanding for robot perception and has been adopted in autonomous vehicle and UAV perception systems. Gaussian primitives offer a promising alternative by potentially representing only occupied regions, thereby concentrating computation and representational capacity where needed and reducing wasted computation and memory in empty space. However, existing Gaussian-based methods suffer from poor primitive utilization, as ineffective training objectives leave some Gaussians in empty space, while the limited expressiveness of Gaussian functions restricts their ability to model complex shapes. To address these limitations, we propose a novel 3D representation, Gaussian Neural Field (GNF), which can accurately map voxel positions to semantic occupancies. In GNF, Gaussians and a neural network jointly parameterize the field, where the Gaussians define its spatial support and provide features for neural decoding, while the neural network enables more expressive occupancy modeling than Gaussian functions alone. We further develop the Gaussian Neural Field Transformer (GNFormer), a streamlined and flexible method that predicts GNFs for camera-based occupancy prediction. Across Occ-ScanNet, SSCBench-KITTI-360, and nuScenes-SurroundOcc benchmarks, GNFormer achieves the highest IoU and mIoU and the fastest inference among the compared single-frame methods. In the experiments, GNFormer can represent indoor and outdoor scenes with only 256 and 1024 Gaussians, respectively, substantially fewer primitives than prior Gaussian-based methods. The code will be released.

The authors’ abstract, as arXiv lists it. arXiv’s listings, abstracts included, are free for anyone to reuse.

More in Robotics

• arXiv preprint · 11 October 2026

  Reinforcement Learning Based Heterogeneous Task Allocation Under Spatio-Temporal Environmental Uncertainty for Multi-Robot Teams

  Xi Lin, Yaxuan Li, Paul Szenher and 1 other
• arXiv preprint · 12 October 2026

  PathLift: Depth-Conditioned Metric Lifting for Visual-Navigation Priors

  Gwanjun Shin, Hyeongseok Suh, Hongmo Jung and 1 other
• arXiv preprint · 11 October 2026

  LeHami: Symplectic Latent Dynamics for World Models

  Romina Yalovetzky, Randall Balestriero and Yann LeCun
• arXiv preprint · 12 October 2026

  Constrained Differentiable Policy Learning for Safe Quadrotor Navigation

  Yan Rui Tan, Praveen Krishnapur, Zeyu Feng and 3 others
• arXiv preprint · 11 October 2026

  Embodiment-Aware Inference-Time Search for Human Reaction Selection in Humanoid Retargeting

  Yongpeng Cao, Masahiro Hirano and Yuji Yamakawa

All new papers in Robotics