arXiv preprint ·
Predicting Partial Answer Quality and Utility in Agentic Retrieval-Augmented Generation
- Published
- Source
- arXiv preprint, posted before peer review
- Original
- Read it at arXiv
- Topics
Abstract
Agentic Retrieval-Augmented Generation (RAG) has become a promising paradigm for multi-hop question answering, where a reasoning model iteratively issues queries to a retriever and incorporates newly retrieved context into subsequent reasoning steps. While this iterative process can improve final answer quality, current evaluations of agentic RAG largely focus on end-to-end outcomes and provide limited visibility into how a model's answer state changes during generation. In this work, we introduce an in-trajectory probing framework to study intermediate answer states in agentic RAG. Specifically, after each retrieval-reasoning iteration, we force an agentic model to stop reasoning and generate an intermediate answer based on its current state. This allows us to define two iteration-level measures: partial answer quality at each iteration, and partial utility as the change in partial answer quality across iterations. Our analysis across multi-hop QA benchmarks reveals that partial answer quality often plateaus before natural termination, with many later iterations contributing only small measurable improvements. Accordingly, we formulate two prediction tasks, partial answer quality prediction and partial utility prediction, and study trajectory-derived signals from intra-iteration, inter-iteration, and query-iteration perspectives. Experiments show that partial answer quality is more predictable than partial utility, with supervised models achieving Pearson's r above 0.43 for quality prediction. Finally, using predicted answer quality and utility for early stopping reduces average iteration count by about 11% while preserving about 98% of the final answer quality achieved by natural stopping.
The authors’ abstract, as arXiv lists it. arXiv’s listings, abstracts included, are free for anyone to reuse.
Read it on ReadSearch
Each opens after you create an account, and brings you back to this paper.
More in Information Retrieval
PCap: Personalized Retrieval-Stage Diversity Capping in Facebook Marketplace
DUPAR: Dual-Path Conversational Retrieval via Speech Retriever with Cross-Turn Evidence Caching
LIGE-GR: A Smooth Leap from Ranking to Generative Recommendation in the LLM Era
ReliGRec: Reliability-Oriented LLM-Based Generative Recommendation via User-Risk-Aware Prompt Routing
PageRecall: Measuring Page Selection in Literature-Grounded Question Answering
Get the few papers worth your time in Information Retrieval every morning.
ReadSearch reads the new papers in your field each day and brings you the few worth reading, each with its key finding in a sentence.