Optimal spectral shrinkage via Krylov approximation

Abstract

Spectral shrinkage encompasses a broad family of estimators for high-dimensional linear regression that regularize estimation and prediction by applying spectral filters to the sample covariance matrix. Despite its widespread use, achieving oracle-optimal prediction risk under general covariance structures remains challenging, as the oracle filter requires unknown population quantities and has a closed form only for specific covariance models such as isotropic and spiked models. We propose \textit{risk-targeted Krylov fitting} (RTK), a data-driven spectral estimator that adapts to the unknown covariance structure and attains the oracle risk asymptotically. Like partial least squares, RTK operates in an $m$-dimensional Krylov space, but it targets the oracle risk directly by minimizing the leave-one-out risk, which we compute exactly without refitting the model $n$ times. Under general distributions and covariance structures, RTK attains the optimal prediction risk in the Krylov space up to $O(1/n)$. Furthermore, its risk approaches that of the optimal analytic filter with $m$ of order $\log n$. In simulations over a wide range of covariance structures, RTK and a truncated variant were robust to the choice of $m$ and consistently among the spectral estimators closest to the oracle. In applications to single-cell sequencing data and large-scale neural recordings, the proposed methods achieved higher prediction accuracy than competing estimators on most tasks.

The authors’ abstract, as arXiv lists it. arXiv’s listings, abstracts included, are free for anyone to reuse.

More in Methodology

• arXiv preprint · 1 November 2026

  Transport for Conditional Sampling: Foundations and Applications

  Ricardo Baptista, Benjamin Bloem-Reddy and Bamdad Hosseini
• arXiv preprint · 2 November 2026

  Sketching in Regression: Sketchy in Practice?

  Somarddha Das, Soupayan Das Gupta and Koustav Goswami
• arXiv preprint · 1 November 2026

  Instance-Specific Sharp Convergence Rate of the Add-$λ$ Estimator via Feature Space Decomposition

  Zong Shang
• arXiv preprint · 2 November 2026

  Calibrated Multiaccuracy: Deterministic and Randomized Sample Complexity

  Jiuyao Lu, Georgy Noarov and Aaron Roth
• arXiv preprint · 1 November 2026

  Federated Estimation of Loading Spaces in Tensor Factor Models under Differential Privacy

  Zetai Cen, Fan Wang and Haeran Cho

All new papers in Methodology