arXiv preprint ·
SE-Mem: Semantic-Episodic Memory for Scene-Consistent Generative View Synthesis
- Published
- Source
- arXiv preprint, posted before peer review
- Original
- Read it at arXiv
- Topics
Abstract
Generative view synthesis aims to synthesize novel-view videos from reference observations along user-defined camera trajectories. Maintaining global scene consistency throughout this process remains challenging for camera-conditioned interactive video generators. Existing approaches typically select reference frames based solely on camera poses, overlooking whether the content remains semantically relevant or even visible. To address these limitations, we draw inspiration from semantic and episodic memory in human cognition, in which semantic memory captures generalized knowledge across experiences, whereas episodic memory preserves information tied to specific past events and contexts. Motivated by the complementary roles of semantic and episodic memory, we introduce SE-Mem, a Semantic-Episodic Memory framework that organizes historical observations in a structured 3D representation and augments its elements with semantic importance scores. A two-stage memory retrieval strategy is designed to identify the most informative frames to provide contextual guidance. Taking into account both semantic significance and geometric visibility, SE-Mem preserves semantic coherence and fine-grained visual details during novel view synthesis. Extensive evaluations show that our approach improves visual quality and achieves competitive geometric consistency across multiple benchmarks.
The authors’ abstract, as arXiv lists it. arXiv’s listings, abstracts included, are free for anyone to reuse.
Read it on ReadSearch
Each opens after you create an account, and brings you back to this paper.
More in Computer Vision
Spatial Uncertainty Analytics for Budget-Limited Inspection of Hyperspectral Scenes
IMPACT: Image-Conditioned Motion with Physics-Aware Collision and Trajectory Guidance for Video Generation
A Straight Line Is Not a Camera Pose: Auditing Short-Clip Pose Benchmarks
Vocabulary Aligned Explanations for Computer Vision Models
Selecting or Solving? What Best-of-N Verification Delivers in Video Reasoning
Get the few papers worth your time in Computer Vision every morning.
ReadSearch reads the new papers in your field each day and brings you the few worth reading, each with its key finding in a sentence.