ZeroGR: Filtering-First Constrained Decoding for Generative Recommendation

Abstract

Generative recommendation is gaining traction in production, retrieving items by generating their identifiers one token at a time. The generated sequences must identify relevant catalog items, so constrained decoding couples relevance scoring with validity filtering. Because most possible sequences are invalid, scoring before filtering wastes computation on candidates that will be discarded. Our evaluation shows that filtering first can avoid this work with negligible impact on recommendation quality. We present ZeroGR, a filtering-first constrained decoding system for generative recommendation. ZeroGR uses a step-sliced filtering index to reduce GPU memory use by tailoring data representations to each decoding step. Its execution engine adopts shared-slot execution to accelerate data exchange across stages and hybrid projection to adapt computation to workload patterns, thereby reducing execution overhead. During index refreshes, ZeroGR applies layout-guided streaming construction to speed up updates and limit serving interference. Extensive evaluation shows that ZeroGR accelerates the filtering and scoring pipeline by up to $9.9\times$ over representative baselines while reducing GPU index memory use and accelerating index reconstruction.

The authors’ abstract, as arXiv lists it. arXiv’s listings, abstracts included, are free for anyone to reuse.

More in Databases

• arXiv preprint · 3 November 2026

  Navigating Runtime Dynamism in Semantic Query Serving

  Hengrui Zhang and Xupeng Miao
• arXiv preprint · 4 November 2026

  BobFlow: Towards Human-Agent Collaborative Data Science Using Dataflows

  Jiadong Bai, Yicong Huang and Chen Li
• arXiv preprint · 3 November 2026

  Text-to-SQL Models Make Choices Without Saying So

  Yichen Xu, Shipei Lin and Chenhao Ma
• arXiv preprint · 4 November 2026

  The Invocation Bottleneck in LLM-Based Lossless Compression: What Can Be Removed and What Must Be Traded

  Zhe Yan, Yihao Liu, Antonio Boffa and 1 other
• arXiv preprint · 4 November 2026

  When Is One Query per Template Enough? Adapting Learned Cost Models to New Hardware

  Krish Patel

All new papers in Databases