arXiv preprint ·
Mutual Information Estimation via Triangular Transport Maps: Finite Sample Guarantees and Sample Allocation
- Published
- Source
- arXiv preprint, posted before peer review
- Original
- Read it at arXiv
- Topics
Abstract
We study finite sample guarantees for transport based mutual information estimation. Mutual information can be estimated by first learning the conditional and marginal densities and then evaluating their log-density ratio by Monte Carlo. Given training samples, we use triangular transport maps to learn these densities and then compute the mutual information estimate on an independent batch of evaluation samples. For conditional density estimation using triangular transport maps, we establish non-asymptotic guarantees over realizable Hölder classes by viewing the conditional density as an ordinary density with respect to the product of the law of the conditioning variable and Lebesgue measure. This allows standard density estimation techniques to be applied directly to the conditional density estimation. These results yield finite sample guarantees for the resulting mutual information estimator and characterize how a fixed sample budget should be allocated between training and evaluation in order to minimize the mean squared error. Numerical experiments support the predicted training and evaluation error rates and illustrate the resulting sample allocation tradeoff.
The authors’ abstract, as arXiv lists it. arXiv’s listings, abstracts included, are free for anyone to reuse.
Read it on ReadSearch
Each opens after you create an account, and brings you back to this paper.
More in Statistics Theory
Bethe Hessian Recovery through the Kesten-Stigum Regime
Non-asymptotic analysis of SGLD with discontinuous stochastic gradients: a new route to portfolio CVaR minimization
On Statistical-Computational Gaps in Graph Combinatorial Testing Problems
The Statistical Geometry of Latent Recovery in Hidden Markov Models
Instance-dependent sequential limits as an exact information ledger
Get the few papers worth your time in Statistics Theory every morning.
ReadSearch reads the new papers in your field each day and brings you the few worth reading, each with its key finding in a sentence.