arXiv preprint ·
Spectral Indistinguishability and Learning-Curve Identifiability in Quadratic Networks
- Published
- Source
- arXiv preprint, posted before peer review
- Original
- Read it at arXiv
- Topics
Abstract
Motivated by information-organization sculpting, we study spectral summaries and scalar learning curves in centered quadratic networks. We construct analytic positive input densities with Gaussian one- and two-coordinate marginals. Their complete fourth-moment operators are orthogonally intertwined by a map fixing the entire diagonal teacher algebra, giving equal anchored spectral measures. For selected common teachers with a nonzero cycle invariant, their factorized loss curves differ because the map does not preserve matrix multiplication. Cycle length $r$, which can be arbitrarily large, gives $O(\varepsilon^r)$ contrast uniformly in training time for each fixed construction. For a supplied pair, noisy scalar loss queries require order $\varepsilon^{-2r}$ observations, while bounded initialization interventions with scalar gradient readouts require order $\varepsilon^{-2}$. A shared-input score, equal to a paired-gradient difference, attains the optimal input-sample exponent without knowing the teacher. An explicit family of diverging auxiliary teacher scales instead admits a nonzero finite limiting contrast at coupling $1/\log L$. Conversely, for Gaussian inputs and a rationally independent teacher spectrum, one exact scalar loss curve identifies the initial predictor up to sign conjugation. Within this rationally independent class, with nonzero diagonal inverse-state error in every coordinate and calibrated time and loss scales, it also identifies the unknown teacher spectrum and active dimension, modulo simultaneous rotations. For fixed dimension and distinct unordered teacher sums, dimension-minimal exact observations give generic joint local recovery. Finite exact global bounds hold for a known teacher and, on the active class, under a blind dimension cap. Exact certificates and full-weight replay verify the construction; numerical inverse checks reveal severe ill-conditioning.
The authors’ abstract, as arXiv lists it. arXiv’s listings, abstracts included, are free for anyone to reuse.
Read it on ReadSearch
Each opens after you create an account, and brings you back to this paper.
More in Machine Learning
Subdata Selection for Measurement-Constrained Linear Regression under Model Misspecification
Efficient Agnostic Learning of Univariate Densities Under Local Differential Privacy
Large-Step Last-Iterate Convergence of Random-Reshuffling Stochastic Gradient Descent
Score Mismatch and Set-Constrained Sampling in Zero-Shot Conditional Discrete Diffusion
State-Matched Sequence Conformal Prediction
Get the few papers worth your time in Machine Learning every morning.
ReadSearch reads the new papers in your field each day and brings you the few worth reading, each with its key finding in a sentence.