Nonparametric autoregressive estimation of probability distributions with parallel gradient boosting

Abstract

Nonparametric estimation of joint probability distributions is a core problem in statistics, which is notoriously challenging with high-dimensional data. Such challenges also arise with conditional distribution estimation and have led to flexible algorithms that aim to tackle the ''curse of dimensionality'' by exploiting predictive structures of lower dimension than the ambient space. In this paper, we study the use of such conditional distribution estimators to estimate joint distributions by factorizing the latter in an ''autoregressive'' way. We show that for loss functions satisfying a subadditivity property, bounds on the convergence rate of an autoregressive estimator can be derived from bounds on the convergence rate of the conditional distribution estimators. These bounds relate the difficulty of estimating the joint distribution to that of estimating the most challenging factor in the autoregressive factorization. When the conditional estimators adapt to sparsity in their conditioning sets, we characterize this difficulty through variable elimination in an undirected dependence graph. This allows bounding the rates of convergence by quantities related to the sparsity of the graph, and provides an additional motivation for the autoregressive approach. We then evaluate the practical performance of an autoregressive estimator based on parallel gradient boosting, a flexible nonparametric method for estimating conditional distributions. The estimator demonstrates competitive estimation performance on various simulated and real datasets, even without optimizing the factorization order. Together, these results support nonparametric autoregressive estimation as a practical approach to flexible joint distribution estimation.

The authors’ abstract, as arXiv lists it. arXiv’s listings, abstracts included, are free for anyone to reuse.

More in Machine Learning (Statistics)

• arXiv preprint · 22 October 2026

  Lifetime-valid decisions by continually evolving AI analysts

  Xiaojing Sun, Yuhan Zeng and Xiao Wang
• arXiv preprint · 23 October 2026

  Rethinking Representation Geometry in Supervised Contrastive Learning under Long-Tailed Data

  Joonhyeok Jeong, Jy-yong Sohn and Kibok Lee
• arXiv preprint · 22 October 2026

  Residual Calibration via Local Feature-Space Refinement

  Xihao Wang, Chengyao Yu and Ruixing Ming
• arXiv preprint · 23 October 2026

  DBCM: Dual-Branch Calibration Module for Test-Time Adaptation in Time Series Forecasting

  Jaebyeong Jeon, Kunwoo Kang, Sang Yup Lee and 1 other
• arXiv preprint · 22 October 2026

  Width and Depth Limits Commute During Training in Linear Residual Networks

  Ruijia Zhang and Soufiane Hayou

All new papers in Machine Learning (Statistics)