arXiv preprint ·
Approximation Families and the Statistical Efficiency of Denoising Score Matching
- Published
- Source
- arXiv preprint, posted before peer review
- Original
- Read it at arXiv
- Topics
Abstract
We study score matching over finite-dimensional approximation spaces generated by gradients of eigenfunctions of the infinitesimal generator of a forward diffusion. We compare polynomial approximation under one-dimensional Ornstein-Uhlenbeck flow with trigonometric-polynomial approximation under heat flow on the torus. In the Ornstein-Uhlenbeck setting, we show that for every non-Gaussian initial distribution uniformly comparable to the standard Gaussian, the asymptotic score-estimation variance grows exponentially along a subsequence of polynomial degrees. Averaging the score-matching loss under the diffusion semigroup, which we call operator-informed score matching (OISM), reduces the variance at each fixed degree but does not eliminate this exponential instability at sufficiently small times. A local minimax lower bound shows that the same high-degree dependence arises when estimating the polynomial projection of the score, rather than being an artifact of OISM. In contrast, under heat flow on the torus, OISM over trigonometric polynomials admits finite-sample control of the score error along the forward diffusion and yields the classical minimax rate for estimating sufficiently smooth densities under expected Kullback-Leibler loss.
The authors’ abstract, as arXiv lists it. arXiv’s listings, abstracts included, are free for anyone to reuse.
Read it on ReadSearch
Each opens after you create an account, and brings you back to this paper.
More in Machine Learning (Statistics)
On the asymptotic behaviour of $α$-divergence generalized posteriors
A Simple Presentation of Robust Membership Inference Attacks
Minimax Rates for Conditional Generative Learning under Wasserstein Loss
Second-order Sinkhorn Geometry and $\sqrt{n}$-Inference
Provably Efficient Q-learning for Robust Reinforcement Learning
Get the few papers worth your time in Machine Learning (Statistics) every morning.
ReadSearch reads the new papers in your field each day and brings you the few worth reading, each with its key finding in a sentence.