arXiv preprint ·
HBFSim: A Simulator for Exploring HBF in Memory Hierarchies for LLMs
- Published
- Source
- arXiv preprint, posted before peer review
- DOI
- 10.1145/3848591.3856870
- Original
- Read it at arXiv
- Topics
Abstract
High Bandwidth Flash (HBF) has recently emerged as a promising technology that provides terabyte-scale accelerator-local capacity at bandwidths approaching HBM, with a potential cost-per-bit advantage over HBM. However, its NAND foundation brings long latency, out-of-place updates, and finite write endurance directly into the accelerator memory hierarchy, making its effects on performance and lifetime difficult to predict. To study this emerging design space, we present HBFSim, an LLM-aware event-driven simulator for heterogeneous HBM+HBF memory systems that preserves application-level memory semantics while modeling flash-internal behavior. HBFSim models HBM, HBF with host-managed flash translation, and conventional capacity tiers in one simulation driven by a calibrated serving frontend. Using HBFSim, we study decoding of Llama-3.1-70B and Qwen3-235B-A22B under three workloads that differ in their input and output lengths. HBF serves data that does not fit in HBM 11.0--19.0$\times$ faster than host DRAM; HBM is better used to hold the running sequences' KV cache than as a cache for HBF; and at a published peak load of Llama-3.1-70B, HBF lasts 2.0 to 6.9 years, in proportion to the stacks that share the KV writes. The artifact is available at https://github.com/FlashLabAE/HBFSim and will track public HBF device data as it becomes available.
The authors’ abstract, as arXiv lists it. arXiv’s listings, abstracts included, are free for anyone to reuse.
Read it on ReadSearch
Each opens after you create an account, and brings you back to this paper.
More in Hardware Architecture
Replay Pruning and Few-Bit Integer Inference for Hardware-Efficient Speed-Sign Recognition
SENTRY: A Streaming Edge Network Trust Engine for Real-Time Audio Deepfake Detection
qGAMMA: A Framework for Optimal DNN Mapping on Multi-Precision Accelerators
Cryogenic Recharacterization of a Commercial 22 nm FD-SOI Foundry Standard-Cell Library
ConQUESTA: Topology-Aware Post-Training Structural Pruning of Quantum Neural Networks
Get the few papers worth your time in Hardware Architecture every morning.
ReadSearch reads the new papers in your field each day and brings you the few worth reading, each with its key finding in a sentence.