Score Mismatch and Set-Constrained Sampling in Zero-Shot Conditional Discrete Diffusion

Abstract

Discrete diffusion models have become a powerful framework for modeling complex discrete distributions across a range of applications. While theoretical convergence guarantees depend on consistency between the target and training distributions, such a case rarely holds in practice, making score mismatch a central challenge in discrete diffusion sampling. This is particularly relevant to zero-shot conditional generation, where a model trained on the unconditional distribution is directly reused at inference time to sample from a conditional target distribution. In this paper, we provide the first theoretical analysis for score-mismatched discrete diffusion models and derive a general KL divergence upper bound that clearly identifies the additional bias caused by such a mismatch. Our analysis covers partial-observation constraints, including text infilling, sentence completion, and more general coordinatewise set constraints. Specifically, we analyze the classical Conditional SEDD sampler and show that it can incur a bias term that scales with the dimension, vocabulary size, and compatibility. Based on this analysis, we propose a Randomized Hard Clamping (RHC) sampler for coordinatewise set constraints. We show that the assignment probabilities can be expressed exactly by the unconditional scores with absorbing rate. Finally, numerical and real-data experiments demonstrate the effectiveness of the proposed RHC sampler.

The authors’ abstract, as arXiv lists it. arXiv’s listings, abstracts included, are free for anyone to reuse.

More in Machine Learning (Statistics)

• arXiv preprint · 15 October 2026

  Efficient Agnostic Learning of Univariate Densities Under Local Differential Privacy

  Mohammad Afzali, Hassan Ashtiani and Sivan Sabato
• arXiv preprint · 16 October 2026

  Unit-level aggregation breaks coverage in causal foundation models

  Gabe Schulman, Lucius E. J. Bynum and Yoav Wald
• arXiv preprint · 16 October 2026

  Conformal Policy Selection: When to Deploy AI Decisions for Targeted Intervention

  Junyu Cao, Ruijiang Gao, Maytal Saar-Tsechansky and 1 other
• arXiv preprint · 15 October 2026

  StochLOB: Heavy-Tailed Score Matching for Robust Limit Order Book Trend Prediction

  Arshia Abolghasemi, Seyed Reza Tavakoli, Parsa Naderi and 7 others
• arXiv preprint · 16 October 2026

  Towards Hyperparameter Transfer for Differentially Private Optimization

  Tianze Wang, Zhiqi Bu and Linjun Zhang

All new papers in Machine Learning (Statistics)