Certification-Based Differentially Private Learning

Abstract

Differential privacy (DP) in machine learning is typically achieved by adding noise to model parameters (private learning) or to model outputs (private prediction). Recent work uses formal methods, namely abstract interpretation, to provide tighter privacy guarantees, but only for private prediction in classification settings. In this work, we investigate the use of formal methods as a general tool for tighter privacy analysis. First, we generalize the abstract gradient training (AGT) framework to private prediction in continuous, unbounded regression. Second, by reducing learning in parameterized models to a regression problem over the parameter space, we introduce Abstract Gradient Sampling (AGS), an algorithm that enables reachability-based analysis to provide guarantees for private learning. In both private prediction and private learning, we provide tightened privacy accounting for the AGT framework and a theoretical analysis demonstrating when our smooth sensitivity upper-bounds yield favourable privacy-utility trade-off. In practice, we validate that our regression bounds are tighter than global-sensitivity baselines on regression benchmarks, and, notably, yield the first finite privacy guarantees in settings where global prediction sensitivity is a priori unbounded. We also find that under matched conditions, our private learning algorithm can outperform standard private learners.

The authors’ abstract, as arXiv lists it. arXiv’s listings, abstracts included, are free for anyone to reuse.

More in Machine Learning

• arXiv preprint · 30 September 2026

  ReGain: Restoring Subject Fidelity in Personalization on Synthetic Images

  Shubhang Bhatnagar, Ishan Bhatnagar, Viraj Shah and 1 other
• arXiv preprint · 1 October 2026

  SmoothOperator: Enhancing Representations for Fine-grained Open-set Recognition via Modulated Label Smoothing

  Thiru Thillai Nadarasar Bahavan, Yu Xia, Sachith Seneviratne and 1 other
• arXiv preprint · 30 September 2026

  Code to Control: Synthesizing Parameterized Reactive Controllers

  Zergham Ahmed, Joshua B. Tenenbaum, Chris Bates and 1 other
• arXiv preprint · 1 October 2026

  Fixing a Model That Learned Worse Cancer Means Lower Risk: Monotonic Constraints in Bladder Cancer Recurrence Prediction

  Saram Abbas, David Thomas, Naeem Soomro and 3 others
• arXiv preprint · 30 September 2026

  Molecular Property Prediction under Structural Shift with Tabular Foundation Models

  Jinmo Lee, Dooho Lee, Minho Jeong and 1 other

All new papers in Machine Learning