arXiv preprint ·
The Rise of Low Precision Floating Point
- Published
- Source
- arXiv preprint, posted before peer review
- Original
- Read it at arXiv
- Topics
Abstract
Floating point computing has dominated scientific computations since the 1950s because of its versatile ability to approximate real numbers with both sufficiently high precision and sufficiently great range. Since 1985, two specific formats have become standard across the industry, one with 32 bits which is adequate for many purposes and one with 64 bits which allows numbers with both much higher precision and much greater range for applications where these are needed. However, during the past decade, deep learning has come to dominate computation, and narrower floating point formats with lower precision and reduced range have steadily become more common. In this article we discuss this phenomenon, why it is happening, how various 16-bit, 8-bit and even 4-bit floating point formats have been developed, and efforts to standardize these formats.
The authors’ abstract, as arXiv lists it. arXiv’s listings, abstracts included, are free for anyone to reuse.
Read it on ReadSearch
Each opens after you create an account, and brings you back to this paper.
More in Mathematical Software
Extensible Programmatic Specification of Finite Element Methods
Integer-Quantized Hierarchical Matrices with Adaptive Bit Allocation
A GPU Engine for Sparse Matrix-Vector Products on $k^2$-Tree-Compressed Graphs
Beyond atol and rtol: Sound Test Oracles for Low-Precision GPU Kernels
diff-soup: GPU-native Differentiable Simulation and Inference for Large-scale Stiff Dynamical Systems
Get the few papers worth your time in Mathematical Software every morning.
ReadSearch reads the new papers in your field each day and brings you the few worth reading, each with its key finding in a sentence.