TALLOP: Joint Optimization of Local Training and Communication Compression for Time-Constrained Federated Learning

Abstract

Minimizing the wall-clock time required to reach a target accuracy is crucial in federated learning (FL), where many applications require frequent model updates as data distributions evolve over time. Nevertheless, in heterogeneous environments, slow clients with limited computational and communication capabilities can delay global synchronization, forcing faster clients to wait and increasing the overall training time. To mitigate this straggler effect, recent studies have introduced per-round time constraints and adapted client workloads within the available time. However, these approaches primarily focus on local computation, whereas communication can become a major bottleneck in communication-limited environments, motivating the joint optimization of training workload and communication compression. Although previous studies have explored such joint optimization, determining an effective training--communication balance under a fixed per-round time budget remains challenging, as the resulting model performance depends on both training progress and compression-induced degradation. We therefore propose TALLOP, which jointly optimizes the local training workload and communication compression under per-round time constraints. TALLOP predicts the training loss achievable under different computation budgets from the early-stage mini-batch loss trajectory (LENS), and estimates the degradation caused by communication compression using a compression-error model (CLEN). TALLOP combines these predictions to select the training-communication allocation that minimizes the post-compression loss under the given round time constraint. Experiments in heterogeneous FL environments show that TALLOP generally achieves competitive or improved time-to-accuracy compared with approaches that optimize only local training or neither, although the benefit over training-only adaptation depends on the setting.

The authors’ abstract, as arXiv lists it. arXiv’s listings, abstracts included, are free for anyone to reuse.

More in Machine Learning

• arXiv preprint · 16 October 2026

  One Communicator, Three Vendors: Extending the MoA Predictive Theory to a Real NVIDIA+AMD+CPU MPI Ensemble

  Lenore Mullin and Gaetan Hains
• arXiv preprint · 17 October 2026

  Koi: Evidence-Guided, Self-Calibrating Planning for Multi-Cloud, Heterogeneous LLM Inference Fleets

  Mankeerat Sidhu, Gangmuk Lim, Hetarth Chopra and 10 others
• arXiv preprint · 16 October 2026

  FederatedBytes: Tokenizer-Free, Byte-Level Small Language Models for Efficient Multimodal Federated Learning

  Ali Sarabadani, Massimo Piccardi, Mehran Abolhasan and 3 others
• arXiv preprint · 17 October 2026

  Adding Heavy-Ball Momentum to Distributed Constrained Optimization for Energy Resource Management Subject to Time-Delays and Quantization

  Mohammadreza Doostmohammadian and Sergio Pequito
• arXiv preprint · 16 October 2026

  Scalable Levenberg-Marquardt Training Framework for Deep Learning

  Xianghao Wang, Johan Barthélemy and John Taylor

All new papers in Machine Learning