arXiv preprint ·
To Defer or Not to Defer? Online Learning with LLMs and Congested Experts
- Published
- Source
- arXiv preprint, posted before peer review
- Original
- Read it at arXiv
- Topics
Abstract
Modern AI systems increasingly combine large language models (LLMs) with human experts, using the LLM to handle routine queries while deferring uncertain cases for expert review. When expert capacity is limited, however, deferral decisions must account not only for the reliability of the LLM but also for congestion in the expert queue. We formulate this as an online sequential decision-making problem in which, for each query, the system observes the confidence signal of the LLM's answer and must decide whether to output the answer or defer the query to a capacity-constrained expert. The reliability of the confidence signal is initially unknown and must be learned from expert feedback, which is both selectively observed (only deferred queries are labeled) and delayed by queuing. We propose \texttt{KL-PBD}, a learning algorithm that uses Kullback--Leibler lower confidence bounds on the unknown reliability parameters to guide deferral decisions. For $K$ confidence levels and horizon $T$, \texttt{KL-PBD} achieves $O(K\log T)$ gap-dependent regret when the accuracy at every confidence level is bounded away from the deferral boundary, and $\widetilde{O}(\sqrt{KT})$ worst-case regret. We establish matching lower bounds, showing that these rates are optimal up to logarithmic factors. Finally, we validate our approach on synthetic instances and real-world question-answering datasets, namely GSM8K, NuminaMath, and MedMCQA, using Qwen LLMs.
The authors’ abstract, as arXiv lists it. arXiv’s listings, abstracts included, are free for anyone to reuse.
Read it on ReadSearch
Each opens after you create an account, and brings you back to this paper.
More in Machine Learning
Coarse-Graining Recursive Expert Computation: Continuation-Relative Interfaces and Failure Modes
MoMaTune: Model, Match, Then Tune Language Models for Numerical Records
Frustratingly Easy Agentic RAG via Portable Skills: A Case Study in Customer Support
Framework for an AI-Ready ARPES Community Database
SceneForesee: Learning Scene Evolution for Spatiotemporal Reasoning
Get the few papers worth your time in Machine Learning every morning.
ReadSearch reads the new papers in your field each day and brings you the few worth reading, each with its key finding in a sentence.