arXiv preprint ·
Invertible Graph Aggregation Can Increase ReLU Readout Width
- Published
- Source
- arXiv preprint, posted before peer review
- Original
- Read it at arXiv
- Topics
Abstract
An invertible graph aggregation preserves the complete feature table, but can increase the size of a classifier that reads one node at a time. For every even $N\ge4$, we construct a labeled tree with $2N$ vertices and maximum degree three. A shared one-hidden-layer rectified linear unit (ReLU) classifier needs one hidden unit on the raw scalar features and exactly $N$ units after one lazy neighborhood mean. All aggregate values are distinct, and the mixing matrix has condition number at most 8. We determine the complete error curve for heads with a free affine term. The width separation persists under scalar perturbations. Under a specified bounded uniform-noise model with one nuisance coordinate, every one-unit aggregate head has expected error at least $1/12$, while a fixed $N$-unit head has zero error. Controlled fits compare trained heads with these explicit feasible scores. On Roman-empire and Tolokers, aggregation improves fitted prediction, and retaining raw features helps at the largest tested parameter budget. Training-only diagnostics include conditional output-layer optimality bounds for eight fixed runs.
The authors’ abstract, as arXiv lists it. arXiv’s listings, abstracts included, are free for anyone to reuse.
Read it on ReadSearch
Each opens after you create an account, and brings you back to this paper.
More in Machine Learning
Variational learning of energy-stable macroscopic flow equations from microscopic laws
On Stochastic Continuous-Time Linear System Identification for Adaptive Stabilization
Transport for Conditional Sampling: Foundations and Applications
Efficient Proper Agnostic Learning of Halfspaces
Correctness and Predictive Uncertainty in Bayesian TSK Fuzzy Regression: A Reproducible Evaluation on Tabular and Building-Meter Data
Get the few papers worth your time in Machine Learning every morning.
ReadSearch reads the new papers in your field each day and brings you the few worth reading, each with its key finding in a sentence.