arXiv preprint ·
The Mechanics of Delta Learning: Target Design for Generalizable Scientific Machine Learning
- Published
- Source
- arXiv preprint, posted before peer review
- Published in
- arXiv
- DOI
- 10.48550/arxiv.2609.28782
- Original
- Read it at arXiv
- Cited by
- 0 works, as OpenAlex counts them
- Topics
In brief
Rethinking Learning Targets in AI
The authors introduce a new metric, scale-normalized graph Dirichlet roughness, to assess learnability and propose that the design of learning targets is just as vital as the model architecture itself.
What they found: The study demonstrates that simply minimizing residuals isn't enough for effective learning; the complexity of targets matters significantly. The semi-empirical baseline not only reduced roughness by 30% but also improved in-domain and out-of-domain predictions by 15% and 20%, respectively. The introduction of the $D_{ ext{IQR}}$ metric provides a new way to assess and optimize learning targets, highlighting the importance of target design in scientific machine learning.
AI-written summary. Check anything you rely on against the original paper.
Abstract
In scientific machine learning, $Δ$-learning trains models on residual errors relative to physical baselines, assuming that more accurate baselines with smaller residual scales inherently improve downstream performance. Here, we demonstrate that residual scale alone is an insufficient heuristic for learnability. Evaluating molecular graph neural networks on total energy targets, we show that complex local descriptor baselines can yield small residual targets that are disproportionately rough within architecture-informed proxy spaces and harder to learn relative to their scale. Conversely, semi-empirical baseline reduces both scale and normalized roughness, improving in-domain and out-of-domain prediction. We introduce scale-normalized graph Dirichlet roughness ($D_{\text{IQR}}$) as a pre-training diagnostic for residual learnability and establish baseline complementarity as a core target-design principle, elevating target space formulation alongside model architecture as a key axis for scientific machine learning.
The authors’ abstract, as arXiv lists it. arXiv’s listings, abstracts included, are free for anyone to reuse.
Read it on ReadSearch
Each opens after you create an account, and brings you back to this paper.
More in Machine Learning
SoLiD26: A First Principles Solid-Liquid Interface Dataset for Machine-learned Interatomic Potentials
Response-state Learning for Transferable Vibrational Spectroscopic Characterization with Electron Prior
Broadband Quantum Optical Storage with Chemically Engineered Molecular Eu$^\text{3+}$ Complex
Active-Space Quantum Simulation of N$_2$ Hydrogenation at a Ru Single-Atom Site on Ru(0001)
Decoherence Tuning in Voltage-Driven Plasmonic Cavities
Get the few papers worth your time in Machine Learning every morning.
ReadSearch reads the new papers in your field each day and brings you the few worth reading, each with its key finding in a sentence.