arXiv preprint ·
VideoAWR: Reinforcing Video Flow Models with Decoupled-Timestep Advantage-Weighted Rectification
- Published
- Source
- arXiv preprint, posted before peer review
- Original
- Read it at arXiv
- Topics
Abstract
Regression-based RL for flow models mandates sampling \emph{without} classifier-free guidance (CFG) to align the rectification target with the reference marginal velocity. Extending Advantage-Weighted Rectification (AWR), which dynamically scales this regression target based on reward, to video generation encounters a cold-start pathology: unguided rollouts collapse into low-fidelity regions where intra-group reward gaps remain wide but no longer track perceptual quality, rendering the optimization target arbitrary. Conversely, utilizing guided rollouts to restore fidelity, while keeping the regression target tied to the guided sampling distribution, provably triggers a geometric guidance blow-up under exponential-moving-average (EMA) updates. We propose \textbf{VideoAWR}, which resolves this tension by untying sampling from the optimization anchor. An EMA reference policy generates \emph{guided} rollouts to restore reward separability, while the trainable policy is rectified against the \emph{unguided} conditional anchor. We prove this decoupled objective is mathematically exact: the centered advantage cancels the anchor in expectation, filtering out the reward-neutral guidance offset and retaining only the reward-correlated covariance increment. Consequently, inference runs single-branch and CFG-free, yet inference-time CFG preserves its original semantics. Furthermore, we introduce \emph{Decoupled Timestep Rectification}, supervising each rollout at an independently sampled timestep without bias to improve sample efficiency. Experiments on Wan2.1-1.3B and 14B demonstrate stable optimization and superior visual and motion quality in \emph{both} CFG-free and CFG-enabled regimes. Code is available at \pdfstartlink attr{/Border [0 0 0]} user{/Subtype /Link /A >}% \textcolor{pearDark}{VideoAWR}\pdfendlink.
The authors’ abstract, as arXiv lists it. arXiv’s listings, abstracts included, are free for anyone to reuse.
Read it on ReadSearch
Each opens after you create an account, and brings you back to this paper.
More in Computer Vision
Structure-Aware Matching for Robust UAV Thermal-to-Satellite Geo-Localization
Design Choices for Multimodal Transformer Learning with Missing and Longitudinal Healthcare Data
APEX: Asynchronous Prefix Denoising for Extreme Compression
SceneForesee: Learning Scene Evolution for Spatiotemporal Reasoning
CaptchaAgent: Verifiable Visual Reasoning for CAPTCHA Solving
Get the few papers worth your time in Computer Vision every morning.
ReadSearch reads the new papers in your field each day and brings you the few worth reading, each with its key finding in a sentence.