arXiv preprint ·
Making Tensor Decoupling More Robust and Less Parametric
- Published
- Source
- arXiv preprint, posted before peer review
- Original
- Read it at arXiv
- Topics
Abstract
Tensor decoupling algorithms recover compact representations of multivariate functions by combining linear factor matrices with univariate internal nonlinearities. We identify two limitations of recent methods. First, their performance is not invariant to mismatched output or gradient magnitudes, and second, their spline internals require manual tuning of their degrees of freedom. We propose two lightweight fixes. The first is a preprocessing step that rescales the input Jacobian tensor, removing both the bias towards larger magnitude outputs and the ill-conditioning of certain factor updates caused by such a mismatch. The second replaces B-spline internals by penalized splines coupled with generalized cross-validation, making the method less dependent on the user selecting the right number of control points. Finally, we release an efficient open-source JAX implementation of our algorithm.
The authors’ abstract, as arXiv lists it. arXiv’s listings, abstracts included, are free for anyone to reuse.
Read it on ReadSearch
Each opens after you create an account, and brings you back to this paper.
More in Machine Learning
When Does Compensating Inherited Quantization Error Help? Mean-Square Guarantees for Layer-Wise Quantization and Their Limits
FedLoRD: Personalized Federated Learning based on Low-Rank Structures of Model Parameter Differences
RFWeave: Learning Wireless Propagation Across Scenes and Transceiver Configurations
ECG-RSI: Recursive Self-Improvement through Targeted Electrocardiogram Waveform Evolution
Universal Distributed Feature Extraction Complementary to Side Information via H-Score
Get the few papers worth your time in Machine Learning every morning.
ReadSearch reads the new papers in your field each day and brings you the few worth reading, each with its key finding in a sentence.