arXiv preprint ·
ARK: Avoiding Routing Collisions for KV Cache Transfer in Disaggregated LLM Inference
- Published
- Source
- arXiv preprint, posted before peer review
- DOI
- 10.1145/3789240.3828750
- Original
- Read it at arXiv
- Topics
Abstract
Disaggregated LLM inference separates the prefill and decode phases across GPU pools, generating massive KV-cache transfers. Because these transfers last hundreds of milliseconds to seconds, they behave as mega elephant flows that dominate link bandwidth utilization. In this regime, stateless ECMP can perform poorly: hash collisions may overload one spine link while leaving others idle, stretching transfer times by seconds. Yet this same persistence makes coordination practical. Since these flows are long-lived, even lightweight one-to-all coordination can be amortized over their lifetime. We present ARK, a distributed elephant-flow path reservation mechanism. ARK coordinates senders to choose source ports whose hashes map concurrent flows onto distinct spines, without requiring switch changes or receiver-side packet reordering. Packet-level RDMA simulations show that ARK reduces mean and P95 FCT by up to 27.3% and 34.0% under moderate load, and further reduces mean TTFT by up to 12.9%.
The authors’ abstract, as arXiv lists it. arXiv’s listings, abstracts included, are free for anyone to reuse.
Read it on ReadSearch
Each opens after you create an account, and brings you back to this paper.
More in Networking & Internet
From Confounding to Counterfactuals: A Cross-Layer Causal Model for RAN Scheduling
Beamfocusing Beyond the Near Field: Frequency Diverse Arrays for OFDM-ISAC Systems
Can Agentic AI Undermine Its Own Assurance Loop? Closed-Loop Dependency Assurance for Cognitive Network Digital Twins
DeepWeave: Weaving Neural and Symbolic Evidence for ML-Based Network Intrusion Detection System
When 200 OK Is Not an Answer: Silent Fetch Failures in Agentic Web Content Acquisition
Get the few papers worth your time in Networking & Internet every morning.
ReadSearch reads the new papers in your field each day and brings you the few worth reading, each with its key finding in a sentence.