Empirical-Bayes spectral partial pooling across related tasks

Abstract

Spectral methods are central to high-dimensional statistics and machine learning, underlying procedures for covariance estimation, matrix denoising, representation learning, clustering, and latent variable modeling. In this work, we focus primarily on spectral estimators for high-dimensional factor models, where leading singular vectors of the data matrix are used to estimate latent structure and covariance parameters. In many modern applications, however, data are collected across related but heterogeneous tasks, studies, domains, or populations. Applying such estimators separately to each task can lead to unstable estimates when sample sizes are limited relative to dimension, while completely pooling the data can obscure meaningful task-specific structure. We introduce Hierarchical Spectral Shrinkage (\texttt{HSS}), a scalable empirical-Bayes framework for partially pooling task-specific spectral estimators. The method regularizes the leading spectral directions of each task toward a data-adaptivelylearned common basis, while allowing the amount of shrinkage to vary across tasks and spectral components. The proposed estimator arises as the posterior mean in a surrogate Bayesian regression formulation of the empirical singular vectors. For factor models, \texttt{HSS} yields partially pooled estimators of the group-specific loading spaces and covariance matrices and can be combined with different spectral estimators by replacing their empirical singular vectors with hierarchically regularized counterparts. More broadly, the same construction provides a mechanism for extending spectral estimators to collections of related but heterogeneous datasets. We demonstrate substantial improvements in estimation accuracy and out-of-sample performance over separate, fully pooled, and shared-subspace approaches in synthetic experiments and multi-study gene-expression data.

The authors’ abstract, as arXiv lists it. arXiv’s listings, abstracts included, are free for anyone to reuse.

More in Methodology

• arXiv preprint · 6 October 2026

  Scalable Regularized Vector Multiplicative Error Models for Positive-valued Financial Time Series

  Rohan Hemant Chhatre, Chiranjit Dutta, Nalini Ravishanker and 1 other
• arXiv preprint · 4 October 2026

  A Data-Driven Framework for Unsupervised Monitoring of Transmission Systems Using End-of-Line Testing Data: A Case Study at Ford Motor Company

  Mohammad N. Bisheh, Mehrdad Moradi, Parinaz Farajiparvar and 6 others
• arXiv preprint · 8 October 2026

  Regularized Small Area Estimation with Graph Laplacian Benchmarking priors

  Jairo Fúquene-Patiño and Brenda Betancourt
• arXiv preprint · 3 October 2026

  Gradient-Free Sampling from Generative Models via Stochastic Bounded Extremum Seeking

  Alexander Scheinker
• arXiv preprint · 8 October 2026

  Efficient quadratic entropy with distance sketches

  Steve Huntsman

All new papers in Methodology