Prior-Informed Generalizable Low-Rank Representation Learning via Regularized Adversarial Reweighting

Abstract

Modern learning problems increasingly leverage data from heterogeneous sources to improve robustness and generalizability under distribution shift, often with high-dimensional parameters admitting low-rank structure. Existing group distributionally robust optimization (group-DRO) approaches optimize against adverse source mixtures, but extending this principle to low-rank matrix estimation with a broad class of losses leads to a challenging nonconvex-concave problem for which methods and theory remain limited. We propose Prior-Informed generalizable Low-rank representation Learning via Adversarial Reweighting (PILLAR), which smooths the group-DRO objective by regularizing adversarial source weights toward a user-specified reference distribution while learning a shared low-rank representation. By tempering extreme adversarial reweighting, PILLAR improves estimation stability and reduces the potential conservativeness of group-DRO. Using Burer-Monteiro factorization, we optimize the nonconvex-concave objective by bifactored gradient descent with mirror ascent. We establish linear convergence to a statistical neighborhood of the population saddle point and derive an error bound that separates low-rank estimation from adversarial reweighting. We further show that PILLAR improves on vanilla group DRO when the latter's excess conservativeness dominates source heterogeneity and prior misspecification. Simulations and an electronic health record study of suicide-attempt risk demonstrate improved worst-source and transported predictive accuracy.

The authors’ abstract, as arXiv lists it. arXiv’s listings, abstracts included, are free for anyone to reuse.

More in Methodology

• arXiv preprint · 11 October 2026

  Explicit Bounds on Hoffman Constants for Sparse Polyhedral Regularizers

  Faris Chaudhry, Anthea Monod and Keisuke Yano
• arXiv preprint · 10 October 2026

  Copula Active Subspaces II: Error Decomposition, A Posteriori Estimation, and Sharpness of the Bounds

  Joshua Chen and Peter Jan van Leeuwen
• arXiv preprint · 11 October 2026

  When do explicit penalties improve flow matching?

  Lea Kunkel, Alberto González-Sanz and Sophie Langer
• arXiv preprint · 10 October 2026

  DisCo Algebra: Why Disintegration and Combination Are Not Inverses

  Xuexing Lu
• arXiv preprint · 11 October 2026

  Adaptive Robust Confidence Sets in High Dimensions

  Ilias Diakonikolas, Chao Gao, Daniel M. Kane and 1 other

All new papers in Methodology