arXiv preprint ·
Classifications in modular restricted Boltzmann machines
- Published
- Source
- arXiv preprint, posted before peer review
- Original
- Read it at arXiv
- Topics
Abstract
We consider a modular associative neural network made of $L$ Hopfield models (HMs), coupled so that intra-module interactions are Hebbian and inter-module interactions are anti-Hebbian; this competitive coupling is known to endow the network with pattern-disentanglement capabilities. The integral representation of this system coincides with an assembly of $L$ restricted Boltzmann machines (RBMs) whose hidden layers are coupled, thereby extending the HM-RBM duality to the modular setting. We then train this modular RBM, via one-step contrastive divergence, to perform a classification task in which a query encoded on the visible layers is mapped onto an $L$-tuple of labels read off the hidden layers. When the query is composed of $L$ patterns that are mutually orthogonal on average, we prove that the RBM weights obtained as empirical means over the training dataset, as suggested by the HM-RBM equivalence, constitute a fixed point of the learning dynamics, and we derive an explicit, non-asymptotic bound on the residual drift. We then turn to a harder classification task in which each module is queried with a mixture of $L$ patterns and we show numerically that the same setting for the RBM weights still provides an effective set-up, letting the network jointly classify and disentangle the mixture.
The authors’ abstract, as arXiv lists it. arXiv’s listings, abstracts included, are free for anyone to reuse.
Read it on ReadSearch
Each opens after you create an account, and brings you back to this paper.
More in Disordered Systems & Neural Networks
Asymptotic Analysis of Empirical Risk Minimization on Entry-wise i.i.d. Heavy-Tailed Data
Structure alone supports efficient visual computation in the Drosophila visual system
Matching of signal, noise and hardware timescales for filtering and forecasting of correlated noise signals
Observing the magic Mpemba effect in localized dynamics on a digital quantum computer
Quantum annealing of the random transverse-field Ising model in one, two, and three dimensions
Get the few papers worth your time in Disordered Systems & Neural Networks every morning.
ReadSearch reads the new papers in your field each day and brings you the few worth reading, each with its key finding in a sentence.