Pro-Active Inference

Abstract

Estimating a population mean is fundamental across scientific applications, but observing every outcome can be prohibitively expensive. Often, covariates and machine learning (ML) predictions are available for the full population, even when only a small subset of outcomes can be observed. How can we leverage this information to proactively select observations to label, update ML predictions as labels arrive, and construct statistically valid confidence intervals? We develop a general, without-replacement formulation of Pro-Active Inference (PAI), extending our earlier with-replacement approach and building on Active Statistical Inference. Rather than making fixed-order label-skip decisions, PAI proactively selects which observation to label next from the remaining covariate pool while accommodating online ML updates. We establish PAI's finite-sample unbiasedness and asymptotic validity, develop an improved finite-sample variance estimator, and extend PAI to superpopulation inference without Bonferroni corrections or $α$-splitting. We characterize PAI's potential efficiency gains over Batch Active Inference in a quasi-oracle setting, highlighting the role of prediction-residual imbalance. Across eight diverse datasets, PAI achieves greater sampling efficiency than Sequential Active Inference in the vast majority of tested settings with near-nominal statistical coverage, using the same ML predictor and uncertainty score. However, comparisons with simpler baselines such as Prediction-Powered Inference and uniform sampling, as well as the closely related Prediction-Powered Active Testing, reveal when additional adaptivity helps or hurts statistical efficiency. Through interpretable, theory-motivated diagnostics and frozen-ML ablations, we investigate when and why PAI gains or loses efficiency, highlighting the initial residual-estimation problem and subsequent online learning dynamics.

The authors’ abstract, as arXiv lists it. arXiv’s listings, abstracts included, are free for anyone to reuse.

More in Methodology

• arXiv preprint · 27 October 2026

  Balanced State Subspaces for Mamba-2 State Compression

  Shang Tingyu and Cao Wenfei
• arXiv preprint · 26 October 2026

  Designing Customer Feedback for Subscription Boxes

  Renjun Hu, Hyun-Soo Ahn and Stefanus Jasin
• arXiv preprint · 27 October 2026

  LLM knowledge for improved Out-of-Distribution Tabular Prediction - A Cross-Validated Regularisation Approach

  Sølve Langseth, Kristoffer Herland Hellton and Hugo L. Hammer
• arXiv preprint · 26 October 2026

  Conformal Novelty Detection via Data-enhanced Localization

  Xiaoyang Wu, Jiajun Liu, Haojie Ren and 1 other
• arXiv preprint · 27 October 2026

  Joint Distributional Copula Regression for Insurance Claim Frequency and Severity via Boosting

  Banghee So and Na Zhang

All new papers in Methodology