arXiv preprint ·
FreeCHROMA: Generative Chroma Keying from Unaligned Background References
- Published
- Source
- arXiv preprint, posted before peer review
- Original
- Read it at arXiv
- Topics
Abstract
In film production, chroma keying composites an approved foreground performance into a new environment, but conventional workflows require coordinated background plates and dedicated harmonization. Recent generative methods often require pre-aligned backgrounds, focus on background-conditioned foreground relighting, or represent the environment only through an edited first frame. These restrictions limit background selection, scene composition, and camera motion while adding preparation or editing effort. We present FreeCHROMA, which adapts a pretrained video diffusion model in a parameter-efficient manner for generative video compositing from one to three unaligned background references. The model integrates the foreground into the referenced environment, preserving foreground identity, motion, and fine visual details while generating a spatiotemporally coherent background. Foreground-background harmonization occurs within the same generation process, without a separate relighting stage. Optional text instructions help disambiguate complex spatial relationships and camera behavior. To train the model, we construct pseudo-paired data from readily available real monocular videos through foreground-background separation and augmentation with existing foundation models. The original videos serve as training targets, while automated processing and quality verification enable scalable data construction. Across reference fidelity, foreground preservation, scene integration, harmonization, and overall video quality, FreeCHROMA consistently outperforms recent compositing and conditional video-generation baselines, with particularly clear gains under camera motion. FreeCHROMA relaxes the constraints of existing tools to enable end-to-end generative chroma keying from unaligned inputs, with exploratory experiments demonstrating its potential in robotics.
The authors’ abstract, as arXiv lists it. arXiv’s listings, abstracts included, are free for anyone to reuse.
Read it on ReadSearch
Each opens after you create an account, and brings you back to this paper.
More in Computer Vision
Design Choices for Multimodal Transformer Learning with Missing and Longitudinal Healthcare Data
Automated carotid artery atherosclerotic inflammation segmentation in PET using MSR-Net
SceneForesee: Learning Scene Evolution for Spatiotemporal Reasoning
GeoBridge: Evolving Transferable Decoder Topologies for Geometric Vision
Protein Crystallization Segmentation and Classification Using Subordinate Color Channel in Fluorescence Microscopy Images
Get the few papers worth your time in Computer Vision every morning.
ReadSearch reads the new papers in your field each day and brings you the few worth reading, each with its key finding in a sentence.