arXiv preprint ·
Counterfactual Marginalisation: Framework for Evaluating Robustness to Nuisance Variables
- Published
- Source
- arXiv preprint, posted before peer review
- Original
- Read it at arXiv
- Topics
Abstract
Machine learning models can achieve strong test performance while relying on demographic or acquisition-related shortcuts. We propose counterfactual (CF) marginalisation as a test-time evaluation procedure for assessing robustness of classification models to such variables. Given a CF image generator, we intervene on nuisance parent variables such as age or sex, generate CF versions of each test image, and average predictions over a target intervention distribution. This produces intervention-aware predictions that marginalise demographic effects while preserving patient-specific latent information. We use these predictions to define metrics for CF risk, calibration, stability and worst-case sensitivity. We demonstrate this framework's utility for quantitative robustness evaluation.
The authors’ abstract, as arXiv lists it. arXiv’s listings, abstracts included, are free for anyone to reuse.
Read it on ReadSearch
Each opens after you create an account, and brings you back to this paper.
More in Machine Learning
A Statistical Approach to Estimating Sample Size of Machine Learning Models
AUC Maximization from Biased Positive-unlabeled Data with Confidence
Arbitrary Cipher Attacks Against Large Language Models Do Not Require Fine-Tuning
Evaluating Scaffolding-Oriented Multi-Agent Large Language Model System for Clinical Interview Training
High-probability guarantees for linear accessibility in feature superposition
Get the few papers worth your time in Machine Learning every morning.
ReadSearch reads the new papers in your field each day and brings you the few worth reading, each with its key finding in a sentence.