arXiv preprint ·
ENCL: Enabling Efficient Neuromorphic Continual Learning with Local Learning Rule and Sub-Network Freezing Strategy
- Published
- Source
- arXiv preprint, posted before peer review
- Original
- Read it at arXiv
- Topics
Abstract
Continual learning (CL) in neural networks is a promising learning approach for updating the knowledge of deployed systems based on new data encountered at run time, that have not been learned during the offline training phase. To enable their power/energy-efficient CL processing, neuromorphic continual learning (NCL) concept with sparse event-based operations have recently been introduced. However, state-of-the-art NCL methods typically employ the memory-hungry data/experience replay, coupled with costly global learning rule (i.e., backpropagation-based surrogate gradient). To address these limitations, we propose ENCL, a novel methodology that enables efficient NCL by employing a lightweight local learning rule and sub-network freezing strategy. ENCL leverages the temporally- and spatially-local learning rule to perform efficient credit assignment for training new tasks, alleviating the expensive cost of global learning rule. ENCL also employs a sub-network freezing strategy based on the given tasks to avoid the needs for network architecture expansion, hence further improving the efficiency and scalability of the NCL training process. To efficiently support this, we train each task with a small number of epochs as well as adjusted learning rate and timestep. Experiment results on task-incremental learning (TIL) scenario show that, our ENCL consistently achieves strong performance across different datasets as compared to state-of-the-art works. Specifically, ENCL achieves competitive accuracy comparable to that of the state-of-the-art data/experience replay, while reducing memory cost by up to 88%, NCL training time by up to 51%, average power consumption by up to 40%, and energy consumption by up to 65% across different benchmarking datasets (i.e., Split MNIST, Split Fashion MNIST, Split CIFAR-10, and Split CIFAR-100).
The authors’ abstract, as arXiv lists it. arXiv’s listings, abstracts included, are free for anyone to reuse.
Read it on ReadSearch
Each opens after you create an account, and brings you back to this paper.
More in Neural & Evolutionary Computing
AgenticEvolve: Evolutionary Program Search with Persistent Language Model Agents that Design and Run Experiments
A Resource-Efficient and Precision-Scalable FPGA Accelerator for Spiking Neural Networks Using Minifloat Arithmetic
First hardware demonstration of a continuous-state probabilistic computer with programmable energy-defined sampling
Searching with Sparse Histories: Cross-Task Architecture Search from Incomplete Search Logs and Zero-Cost Proxies
PAFS: Perturbation-Aware Firing Scaffolds for Robust Event-Based SNNs
Get the few papers worth your time in Neural & Evolutionary Computing every morning.
ReadSearch reads the new papers in your field each day and brings you the few worth reading, each with its key finding in a sentence.