OpenAlex ·
Drift-Aware LLM Routing with Sparse Contexts and Shared Budgets
- Published
- Source
- OpenAlex
- DOI
- 10.48550/arxiv.2609.00662
- Original
- Read it at the publisher
- Topics
Abstract
A multi-model language service must route each request while preserving workload-level budgets for compute, latency, memory, or monetary cost. Two features make this problem materially harder than static model selection. Prompt representations are high dimensional, so only a small subset of embedding directions may predict the incremental value of a model, and both the request mix and the model frontier drift after launches, fine-tunes, quantization changes, and system updates. We formulate nonstationary sparse contextual routing with multiple knapsack constraints and an optional shadow-audit stream that evaluates a small fraction of prompts on several models. We propose Drift-Aware Sparse Routing (DRS). The policy estimates reward and resource use from a rolling audit window, routes using pessimistic reward and optimistic cost estimates, updates resource shadow prices online, and applies a hard meter before commitment. The analysis separates control from statistics. On any event with uniform prediction radii $\{β_t\}$, regret against a paced dynamic fluid benchmark is bounded by the sum of the radii, a capacity-buffer term, and an $O(\sqrt{T})$ pacing term. Under a sparse linear model and bounded drift $V_T$, rolling estimation gives \[ \widetilde O\left( T\sqrt{\frac{s}{ρW}}+WV_T+\sqrt{T} \right), \] where $s$ is sparsity, $ρ$ is the audit rate, and $W$ is the window length. Optimizing $W$ yields the usual stationary $O(\sqrt{sT/ρ})$ rate when $V_T=0$ and a $O(T^{2/3}(s/ρ)^{1/3}V_T^{1/3})$ adaptation term under drift.
The authors’ abstract, as arXiv lists it. arXiv’s listings, abstracts included, are free for anyone to reuse.
Read it on ReadSearch
Each opens after you create an account, and brings you back to this paper.
More in Databases
Recommender System as Slow and Fast Thinkers
LEAP: Likelihood Elicitation and Aggregation for LLM-based Probabilistic Forecasting
ClaimReceipt: Verifying Evidence Sufficiency and Coverage in Agent Evaluations
Don't You Know, Pump it Up! Investigating Cryptocurrency Manipulation in Telegram-Driven Activity
FOXDEN: FAIR Services for AI-Ready Scientific Datasets
Get the few papers worth your time in Databases every morning.
ReadSearch reads the new papers in your field each day and brings you the few worth reading, each with its key finding in a sentence.