Gaussian Approximation of Besov Functions for Logistic Classification on Manifolds

Abstract

We study Gaussian kernel approximation and regularized logistic classification on a connected compact smooth embedded manifold $X\subset\mathbb R^d$ without boundary and of intrinsic dimension $\varrho$. Two main challenges are to derive a fast oracle inequality without assuming a bounded Bayes function and to construct Gaussian approximants that exploit both the intrinsic dimension and the target's regularity without saturation. We address the first challenge through a Bernstein-type inequality whose constant depends linearly on the supremum norm of the predictor. Combined with localized complexity estimates, this yields an oracle inequality over separable normed function spaces. To address the second challenge, we establish a uniform Sobolev operator expansion and cancel its lower-order bias terms using a signed combination of Gaussian kernel integral operators at different bandwidths. Real interpolation then gives an $L_2(X)$ approximation error of order $γ^α$ for functions in $B^α_{2,\infty}(X)$, where $γ$ is the bandwidth. By constructing the approximants directly on the manifold without ambient extension, we obtain a squared Gaussian RKHS norm bound of order $γ^{-\varrho}$. This construction thus overcomes second-order saturation while retaining norm control governed by the intrinsic dimension. When the marginal distribution has a density $g\in L_p(X)$, these results yield the logistic excess-risk rate $\bigl((\log n)^{\varrho+1}/n\bigr)^{4α(1-1/p)/(4α(1-1/p)+3\varrho)}$. We further show that clipping predictions yields the faster clipped logistic excess-risk rate $\bigl((\log n)^{\varrho+2}/n\bigr)^{2α(1-1/p)/(2α(1-1/p)+\varrho)}$. For bounded densities, its polynomial exponent matches that of the minimax optimal rate for squared-loss regression, and its corresponding excess misclassification risk also matches the minimax optimal rate.

The authors’ abstract, as arXiv lists it. arXiv’s listings, abstracts included, are free for anyone to reuse.

More in Machine Learning (Statistics)

• arXiv preprint · 19 October 2026

  Local data augmentation for imbalanced binary classification

  Harold Guéneau, Alain Celisse and Pascal Delange
• arXiv preprint · 20 October 2026

  On the Accuracy of Approximate Leave-One-Out with Blockwise Nonsmooth Regularization in High-dimensions

  Honghao Lin, Songbin Liu, Arian Maleki and 5 others
• arXiv preprint · 20 October 2026

  Anytime-Valid Runtime Monitoring of Learned Barrier Certificates

  Zhipeng Cao, Taoran Wu, Weiru Cui and 2 others
• arXiv preprint · 19 October 2026

  A Short Note on Algorithms For The Noisy $k$-XOR Problem

  Kiril Bangachev
• arXiv preprint · 20 October 2026

  Optimal aggregation in multiclass classification

  Felix Abramovich and Or Davidovitch-Servadio

All new papers in Machine Learning (Statistics)