Conformal Policy Learning with Distribution-Free Safety Guarantees

Abstract

Policy learning aims to determine who should be treated based on individual characteristics. In high-stakes settings such as medicine and public policy where safety is a central concern, improving the average outcomes alone may not be sufficient: decision makers may also seek to protect individuals from harm, in line with the Hippocratic principle of ``do no harm.'' In this paper, we propose \textit{conformal policy learning} (CPL), a policy learning procedure with a new distribution-free safety guarantee that controls the probability of assigning treatment to an individual who would be harmed relative to control. CPL views each treatment decision as testing a hypothesis of counterfactual harm and assigns treatment by thresholding conformal p-values. These p-values use observable proxies and selective calibration to address the challenge that the potential outcomes under comparison are never simultaneously observed. For randomized experiments, under standard exchangeability conditions, CPL provides finite-sample safety guarantee at a user-specified level, without imposing any outcome modeling assumptions. Moreover, when the outcome model is consistently estimated, CPL achieves asymptotically optimal welfare subject to the safety constraint. In observational studies, CPL with learn-then-balance weights achieves doubly robust safety guarantees. We evaluate CPL through extensive simulations and apply it to an empirical study of AI-powered interventions designed to reduce conspiracy beliefs.

The authors’ abstract, as arXiv lists it. arXiv’s listings, abstracts included, are free for anyone to reuse.

More in Methodology

• arXiv preprint · 16 September 2026

  Stable Policy Learning

  Harvey Barnhard, Giacomo Opocher and Rahul Singh
• arXiv preprint · 14 September 2026

  Eigenvalue-Decomposition Cost Denoising as an Alternative to Predict-then-Optimize for Shortest-Path Problems

  Henry Aldridge-Krawciw and Irene Aldridge
• arXiv preprint · 19 September 2026

  Stealing profits: Spread-based temporal hierarchy forecasting for day-ahead electricity markets

  Arkadiusz Lipiecki, Nikolaos Kourentzes and Rafal Weron
• arXiv preprint · 11 September 2026

  Beyond Point Forecasts: A Survey on Probabilistic Forecasting for Time Series and Spatiotemporal Data

  Donia Besher, Rajdeep Pathak, Madhurima Panja and 1 other
• arXiv preprint · 22 September 2026

  Conditional Tensor Diffusion: Distributional Counterfactual Learning and Inference

  Xinbing Kong, Zeyu Li, Junfan Mao and 1 other

All new papers in Methodology