Specification-Aware Predictive Uncertainty Decomposition

Abstract

Classical decompositions of predictive uncertainty, such as the information-theoretic one, are defined for a fixed representation $x$ of the input object. They do not consider that $x$ may be incomplete, nor do they measure how much the missing information can affect the prediction. In Large Language Models, users can clarify an underspecified input without any retraining. The clarified input can describe the user's unobserved intent $z^*$ more accurately and change the predictive uncertainty about the outcome. In this paper, we formalize this setting probabilistically and derive a three-term decomposition of prediction risk into specification, residual aleatoric, and epistemic uncertainties. We show that at the original information level, when only $x$ is available, the aleatoric uncertainty, represented by a Bayes risk, has two components. The first, which we call specification uncertainty, is the expected reduction in this Bayes risk from revealing the intent. The second is the residual aleatoric uncertainty, i.e., the Bayes risk that remains once the intent is known, averaged over its possible values $z$ given $x$. The distinction is important, as specification uncertainty informs the decision to request clarification before making a prediction. We build approximations of these components and connect them to existing uncertainty measures. Our experiments across different domains and modalities show distinct roles for the three components. Specification uncertainty is the most informative for detecting underspecified inputs, while residual aleatoric uncertainty is the most informative for detecting in-distribution prediction errors.

The authors’ abstract, as arXiv lists it. arXiv’s listings, abstracts included, are free for anyone to reuse.

More in Machine Learning (Statistics)

• arXiv preprint · 2 November 2026

  On Stochastic Continuous-Time Linear System Identification for Adaptive Stabilization

  Reza Sadeghi Hafshejani and Mohamad Kazem Shirani Faradonbeh
• arXiv preprint · 3 November 2026

  Nonparametric GARCH: An Iterated Neural Estimator

  Ruizhi Deng, Guohao Shen and Ngai Hang Chan
• arXiv preprint · 2 November 2026

  Efficient Proper Agnostic Learning of Halfspaces

  Anindya De, Elchanan Mossel, Shivam Nadimpalli and 1 other
• arXiv preprint · 3 November 2026

  Descending Everest, Part II: Oracle Complexity of Minimizing a Sum of Convex $(L_0,L_1)$-Smooth Functions

  Mikhail Rudomanov, Ivan Fomin, Alexander V. Gasnikov and 1 other
• arXiv preprint · 2 November 2026

  Where Is a Prediction Model Miscalibrated, and for Whom? Local Audits in Outcome-Learned Pockets

  Bernardo Modenesi, Taoyi Chen, Sima Najafzadehkhoei and 3 others

All new papers in Machine Learning (Statistics)