arXiv preprint ·
ACSAN: All-Convolutional Self-Attention Network Integrating Convolutional Operations with Vision Transformer
- Published
- Source
- arXiv preprint, posted before peer review
- Original
- Read it at arXiv
- Topics
Abstract
In the past few years, Vision Transformer (ViT) has achieved excellent performance in the field of computer vision. However, there still some problems remain. For instance, ViTs heavily rely on large-scale training data since they simply contain a large number of trainable parameters, which slows down their learning efficiency and limits their applicability in resource-constrained scenarios. In this paper, we propose a novel deep learning model, namely All-Convolutional Self-Attention Network (ACSAN), which applies convolution operations to replace fully-connected matrices multiplications in the ViT framework. To implement ACSAN, we firstly design Dynamic Convolutional Self-Attention (DCSA) as the basic module, which employs multi-scale dilated convolutions to generate Query, Key, and Value. We defined spatial self-attention and channel self-attention to generate attention features respectively. The two attention features are finally merged as the output of the DCSA module. Based on DCSA, we then construct the basic ACSAN block. Comparing with ViT, we use DCSA to replace the regular self-attention operations, and use convolution layers to replace MLPs. To evaluate the performance of the proposed method, we train ACSAN on three widely-used image classification databases, i.e. CIFAR-10, CIFAR-100 and ImageNet-1k, from scratch. Experimental results show that ACSAN achieves excellent classification accuracy comparing with standard ViT with comparable or fewer parameters, as well as other state-of-the-art methods. Ablation studies verify that each component contributes positively to the final performance.
The authors’ abstract, as arXiv lists it. arXiv’s listings, abstracts included, are free for anyone to reuse.
Read it on ReadSearch
Each opens after you create an account, and brings you back to this paper.
More in Computer Vision
Regional prediction of hematoma expansion from non-contrast computed tomography in spontaneous intracerebral hemorrhage patients
SO-101 Bench: Accessible Spatial Manipulation
StreamVibe: Proactive Video Agents for Evolving User Intent
FORESEE-MS: Toward Prospective Spatial Forecasting of Future Multiple Sclerosis Lesions from Baseline FLAIR MRI
Video World Models Need Predictable Latents
Get the few papers worth your time in Computer Vision every morning.
ReadSearch reads the new papers in your field each day and brings you the few worth reading, each with its key finding in a sentence.