Energy Consumption Analysis of Discrete Diffusion and Autoregressive Language Models

Abstract

As Large Language Models (LLMs) continue to scale, their energy consumption during inference has become a critical concern. While Autoregressive (AR) models are the industry standard, Diffusion Language Models (DLLMs) have emerged as a promising alternative capable of parallel token generation. In this paper, we perform a comparative energy benchmark of diffusion models (LLaDA-8B, Dream-7B) against AR counterparts (Llama-3.1-8B, Qwen-2.5-7B). We analyze the trade-offs between model architecture, KV-cache optimization, and Joules-per-token efficiency. Our results show that at longer sequences (200--600 tokens), diffusion models achieve 2.5--2.7 J/token, approximately 2$\times$ more efficient than AR models with KV-cache (5.3--5.7 J/token). KV-cache provides substantial benefits for AR models at longer sequences (50--70\% energy reduction).

The authors’ abstract, as arXiv lists it. arXiv’s listings, abstracts included, are free for anyone to reuse.

More in Machine Learning

• arXiv preprint · 14 October 2026

  TopicMerge: A Divide and Conquer Approach to Topic Modeling

  Matthew Raffel, Lizhong Chen, Sai Munikoti and 1 other
• arXiv preprint · 15 October 2026

  Efficient Agnostic Learning of Univariate Densities Under Local Differential Privacy

  Mohammad Afzali, Hassan Ashtiani and Sivan Sabato
• arXiv preprint · 14 October 2026

  Why don't Graph Transformers attend to the same token?

  Alexandre Bismuth
• arXiv preprint · 15 October 2026

  INCONVENIENT: A Benchmark to Assess LLM Generalization Capabilities from Convenience Samples

  Sara Rezaeimanesh, Chad Klochko, Kundan Thind and 1 other
• arXiv preprint · 14 October 2026

  Counterfactual Gate-Support Decomposition: Separating Selection from Redistribution in Training-Data Filtering

  Hyojung Han, Jongmin Kim and Seung-Hun Jeon

All new papers in Machine Learning