arXiv preprint ·
Cost-Aware Post-Hoc Deferral Under Calibration and Shift: An Environmental AI Case Study
- Published
- Source
- arXiv preprint, posted before peer review
- Original
- Read it at arXiv
- Topics
Abstract
Choosing a deferral policy for a frozen classifier requires more than ranking uncertain cases: confidence may be miscalibrated, errors have unequal costs, reviewers can err, and deployment data can leave calibration support. We study these interactions through EcoTrust, a post-hoc framework that compares automatic action with review using a six-group error-risk estimator, class-asymmetric costs, reviewer accuracy, and an optional support gate. On a Columbia River thermal-stress testbed, the learned estimator improves error-ranking area under the receiver operating characteristic curve from 0.869 to 0.889, but Chow's confidence rule has lower in-distribution cost (0.416 versus 0.567 per day). Across 12 off-the-shelf backends, learned risk and a calibration-matched, class-aware confidence estimator each beat raw Chow on six; a paired year-block bootstrap does not resolve their mean cost difference. In transfer to ten river stations, the gate flags every case and becomes an always-review fallback, attaining the lowest cost on eight stations only when review is perfect and unconstrained. These results characterize decision boundaries on one controlled task: richer risk signals do not reliably improve on calibrated confidence, and detected extrapolation does not imply transferable case-level ranking.
The authors’ abstract, as arXiv lists it. arXiv’s listings, abstracts included, are free for anyone to reuse.
Read it on ReadSearch
Each opens after you create an account, and brings you back to this paper.
More in Software Engineering
LatentMD: Benchmarking Markdown Boundary Failures in LLM-Generated Text
Vectorizer: Vectorizing NumPy Programs with Shape-Guided Rewrite
Recompilation Is Not Enough: Test-Guided Decompiled-C Repair
SWE-Bench Pro Verified: A Reliable Benchmark for Software Engineering Agents
Quality Metrics for LLM-Generated Asset Administration Shells: A Perturbation-Based Evaluation Approach
Get the few papers worth your time in Software Engineering every morning.
ReadSearch reads the new papers in your field each day and brings you the few worth reading, each with its key finding in a sentence.