arXiv preprint ·
General Frequency Principle for Linear Approximation Classes
- Published
- Source
- arXiv preprint, posted before peer review
- Original
- Read it at arXiv
- Topics
Abstract
In this paper, we study a general frequency principle under gradient descent training for the finite linear approximation classes. Under suitable assumptions, we prove that the learner amplitude relative to the target amplitude vanishes at high frequencies, uniformly throughout training. We also establish sufficient conditions for low-frequency priority, covering single-frequency targets trained separately and two frequency components trained in the same run. This general frequency principle does not imply that lower frequencies always converge faster. Instead, for two frequencies that satisfy certain comparison conditions, one component will take longer to learn than the other; and in common cases, the longer component has higher frequency. Numerical experiments with Legendre and trigonometric bases validate the theoretical results.
The authors’ abstract, as arXiv lists it. arXiv’s listings, abstracts included, are free for anyone to reuse.
Read it on ReadSearch
Each opens after you create an account, and brings you back to this paper.
More in Machine Learning
Representation-Aware Closed-Surface Parameterization: Exact Sensitivity Analysis and Pole-Aware Planar Preconditioning
Feature-preserving interface reconstruction using piecewise circles and cusps
Unbiased Low-Rank Approximation for Repeated Matrix-Vector Multiplication: Stability and Optimal Sampling
Extending Sample-based Quantum Diagonalization to Systems of Linear Equations
Closed-Form Gaussian Mixture Representations of d-Dimensional Triangulated Domains
Get the few papers worth your time in Machine Learning every morning.
ReadSearch reads the new papers in your field each day and brings you the few worth reading, each with its key finding in a sentence.