arXiv preprint ·
LampAttention: Look-Ahead Mixed-Precision FlashAttention for Dedicated Accelerators
- Published
- Source
- arXiv preprint, posted before peer review
- Original
- Read it at arXiv
- Topics
Abstract
While most attention logits can be computed in low precision without degrading numerical stability, current attention kernels fail to exploit this phenomenon. We introduce a novel hardware-algorithm co-design in the form of mixed-precision FlashAttention. Our method accumulates key-query products and evaluates their exponentials in 8-bit formats, then adaptively identifies sensitive sub-blocks and recomputes them in 16-bit formats. We propose the specifications for a dedicated accelerator capable of executing this pipeline efficiently. Simulated experiments with Qwen3 and Gemma 3 show that rerouting a selective minority of sub-blocks to high precision is sufficient to recover the baseline model performance.
The authors’ abstract, as arXiv lists it. arXiv’s listings, abstracts included, are free for anyone to reuse.
Read it on ReadSearch
Each opens after you create an account, and brings you back to this paper.
More in Machine Learning
High-accuracy simulation of Picard HMC, part I: Gaussian cloud correction
On the Pricing of American Options under Stochastic Local Volatility and Stochastic Correlation via the RBSDE Framework
Minimax rates for learning spectral Barron functions by deep ReLU neural networks
Kolmogorov-Arnold Networks for Free-Boundary Partial Differential Equations
Quantum Frozen--Oseen homotopy analysis method with LCHS for solving nonlinear partial differential equations
Get the few papers worth your time in Machine Learning every morning.
ReadSearch reads the new papers in your field each day and brings you the few worth reading, each with its key finding in a sentence.