arXiv preprint ·
Operational Indirect Rate-Distortion Functions for Classification Tasks
- Published
- Source
- arXiv preprint, posted before peer review
- Original
- Read it at arXiv
- Topics
Abstract
In goal-oriented communications, indirect source coding involves two sources: a semantic source $X$ and an observation $Z$, which is the only source available at the encoder. In this paper, we consider this framework for classification, where the decoder only needs to reconstruct $X$. We argue that the gap between the indirect rate-distortion function and the performance of practical learned coding schemes, such as variational information bottleneck (VIB) and variational autoencoder (VAE) architectures, comes from three effects: the latent representation, the choice of the deep neural network (DNN) architecture and training loss, and whether the encoder is aware of the task. To quantify these effects, we introduce four operational rate-distortion functions that account for the DNN architecture and the training loss, and we establish a hierarchy among them. We then derive these functions for a Gaussian mixture source, where $X$ is the mixture state. Numerical results, both for the Gaussian mixture and for practical coding schemes on the CIFAR-100 dataset, quantify the rate loss incurred by each effect.
The authors’ abstract, as arXiv lists it. arXiv’s listings, abstracts included, are free for anyone to reuse.
Read it on ReadSearch
Each opens after you create an account, and brings you back to this paper.
More in Information Theory
Graph Neural Networks for Resource Allocation in Integrated Sensing and Communication
Functional Converses for Private Matrix-Product Retrieval under Library-Wide Decoder-Side Correlation
Energy-Time Optimal Decoding Using Expander-Graph LDPC Codes and Mesh Network Architecture
Design of a Speech Enhancement-Based Deep Semantic Autoencoder for Wireless Communications
Information-Spectrum General Formulas for the Capacity Region of Classical-Quantum Multiple Access Channels
Get the few papers worth your time in Information Theory every morning.
ReadSearch reads the new papers in your field each day and brings you the few worth reading, each with its key finding in a sentence.