arXiv preprint ·

Dimension-free spectral laws and the risk of spectral filters

Chen Cheng

Published
Source
arXiv preprint, posted before peer review
Original
Read it at arXiv
Topics
Statistics TheoryProbabilityMachine Learning (Statistics)

Abstract

Spectral properties of the sample covariance are key to modern machine learning with high-dimensional data. Random matrix theory (RMT) supplies powerful tools; however, most classical results assume proportional asymptotics, where $d/n$ tends to a finite limit. In functional data analysis and kernel methods, a fixed infinite-dimensional Hilbert space is often the more natural setting for the observations, beyond the scope of proportional asymptotics. This calls for a dimension-free spectral theory for i.i.d. observations $\boldsymbol{x}_1, \dots, \boldsymbol{x}_n$ in a Hilbert space $\mathcal{H}$ with trace-class population covariance $Σ$. Building on the dimension-free ridge regression analysis of Cheng and Montanari (2024), we establish two deterministic equivalents with explicit nonasymptotic error bounds. The first approximates the empirical spectral law $\widehatμ_n$ of the sample Gram matrix by a deterministic law $μ_n$. The second characterizes the conditional prediction risk of analytic spectral filters in linear regression. Previous sharp approximations to prediction risk in this setting have focused on ridge regression and closely-related problems. Our analysis covers a broader class of filters at the bulk scale, with an explicit user-friendly spectral stability condition to check. We apply our theory to power and critical spectral laws, and we recover the proportional regime. For prediction, we study (negative) ridge estimators, iterated ridge, and gradient flow. Mathematically, building on the idea of stochastic interpolation, we construct a deterministic backward path to interpolate the spectral parameter. The resulting bounds on resolvent and mixed-resolvent traces give mesoscopic approximations to the spectral law within the bulk and, by Cauchy's formula, deterministic equivalents for the prediction variance and squared bias of analytic spectral filters.

The authors’ abstract, as arXiv lists it. arXiv’s listings, abstracts included, are free for anyone to reuse.

Read it on ReadSearch

Each opens after you create an account, and brings you back to this paper.

Already a reader? Sign in

All new papers in Statistics Theory

Get the few papers worth your time in Statistics Theory every morning.

ReadSearch reads the new papers in your field each day and brings you the few worth reading, each with its key finding in a sentence.

Papers belong to their authors and publishers; every page here links to the original. All research topics