Navigating Runtime Dynamism in Semantic Query Serving

Abstract

LLM-powered semantic queries increasingly combine relational processing with multiple LLM-based operators over unstructured data. Efficiently serving such queries is challenging due to runtime dynamism at two coupled levels. Across queries, dynamic arrivals and heterogeneous resource demands continuously change scheduling priorities. Within a query, heterogeneous characteristics across fine-grained input batches cause operator costs and selectivities to vary throughout execution, making a statically optimized plan unreliable. These two forms of dynamism create a circular dependency: inter-query scheduling requires estimates of remaining LLM work, while the remaining work depends on runtime plan choices whose costs become observable only during execution. We present SeqoFlow, a runtime framework that jointly adapts query plans and schedules concurrent semantic queries. SeqoFlow enumerates equivalence-preserving plan alternatives under operator dependencies and decomposes queries into fine-grained units, enabling safe plan adaptation and query preemption at batch boundaries. To support both decisions without exhaustively profiling candidate plans on the target LLM, SeqoFlow uses lightweight speculative estimation to capture runtime costs. These estimates jointly guide plan adaptation and inter-query scheduling, closing the loop between query optimization and LLM inference. Experiments across diverse query structures, data distributions, and arrival rates show that SeqoFlow substantially reduces end-to-end latency compared to existing semantic query systems, adaptive query processing techniques, and conventional scheduling policies.

The authors’ abstract, as arXiv lists it. arXiv’s listings, abstracts included, are free for anyone to reuse.

More in Databases

• arXiv preprint · 4 November 2026

  BobFlow: Towards Human-Agent Collaborative Data Science Using Dataflows

  Jiadong Bai, Yicong Huang and Chen Li
• arXiv preprint · 3 November 2026

  Text-to-SQL Models Make Choices Without Saying So

  Yichen Xu, Shipei Lin and Chenhao Ma
• arXiv preprint · 4 November 2026

  The Invocation Bottleneck in LLM-Based Lossless Compression: What Can Be Removed and What Must Be Traded

  Zhe Yan, Yihao Liu, Antonio Boffa and 1 other
• arXiv preprint · 3 November 2026

  ZeroGR: Filtering-First Constrained Decoding for Generative Recommendation

  Xiangrui Yang, Jiawei Yang, Liang Zeng and 9 others
• arXiv preprint · 4 November 2026

  When Is One Query per Template Enough? Adapting Learned Cost Models to New Hardware

  Krish Patel

All new papers in Databases