Accuracy Is Not Risk: Why Medical AI Should Not Report Only Its Most Likely Label

Abstract

Many medical AI models are trained to reproduce a single reference label and hand the most likely label to the next step in a clinical workflow. A classifier can select that label correctly while providing an inadequate summary for estimating downstream risk. Methods. We use Jeffrey's rule of conditioning to specify when probabilities over intermediate states, such as diagnoses or imaging categories, can be carried forward to an outcome risk. We separate two errors of reporting only the most likely label, discarding the probability of the other states and applying the risk of patients who have the leading diagnosis to a patient who only probably has it. We also give conditions under which the label and the full distribution lead to the same decision, and we propose guardrails for device developers, regulatory reviewers, and health insurers. Results. In a hypothetical chest-pain differential, reporting only the benign leading diagnosis greatly understates the patient's risk of death or cardiac arrest, so a patient who warrants evaluation could be discharged or denied imaging. In a hypothetical BI-RADS example, the most likely category is "probably benign," yet the propagated malignancy risk exceeds the ceiling that defines that category. Conclusions. A predicted label is not an observed state, and a calibrated classifier is not automatically a valid outcome model. Clinical and insurance coverage decisions should rest on the patient's outcome risk, judged against a threshold justified by the consequences of acting and not acting. Developers should state what each model output means and check that every step between the model and the decision preserves the patient's risk.

The authors’ abstract, as arXiv lists it. arXiv’s listings, abstracts included, are free for anyone to reuse.

More in Methodology

• arXiv preprint · 1 November 2026

  RATSIFT: Resolution-Adaptive Transformer for Solar Irradiance Forecasting and Temporal Downscaling

  Jesto Peter, Timothy M. Hansen and Hossein Moradi Rekabdarkolaee
• arXiv preprint · 31 October 2026

  Sentiment sensing: comparative analysis of multilingual sentiment dynamics

  Mara Sabina Bernardi, Elisa Di Nuovo, Francesca Torti and 6 others
• arXiv preprint · 2 November 2026

  Hybrid Federated Learning for Multimodal Fusion under Missing Modalities and Site Heterogeneity

  Xiaofei Wu, Rongmei Liang and Jian Qing Shi
• arXiv preprint · 30 October 2026

  From fleet trajectories to single-cell promises: hierarchically weighted conformal prediction for battery SOH monitoring

  Xiangyu Liu
• arXiv preprint · 2 November 2026

  Reducing SNAP Payment Errors at Lower Cost With AI: A Simulation Study

  Alex Chohlas-Wood

All new papers in Methodology