arXiv preprint ·
Relative Wasserstein Spatial Depth for Cluster Number Selection in Distributional Data
- Published
- Source
- arXiv preprint, posted before peer review
- Original
- Read it at arXiv
- Topics
Abstract
Contemporary data in many scientific domains, such as images, media streams, biomedical omics, are naturally modeled as probability distributions in Wasserstein space instead of points in Euclidean space. Consequently, clustering methods for distributions are in high demand, as they provide exploratory frameworks to group objects with similarity. Clustering distribution-valued data in Wasserstein space often requires choosing the number of clusters in advance. We propose Relative Wasserstein Spatial Depth (RWSD) as a selection criterion for optimal number of clusters. It compares the depth of each distribution in its assigned cluster with its depth in competing clusters. We pair this criterion with Wasserstein K-means or a proposed K-medians algorithm. Under suitable assumptions, we establish consistency of the cluster centers and of the selected optimal number of clusters. We also give an error rate under two-stage sampling and characterize the robustness of RWSD under any fixed partition. In extensive simulations, RWSD selects the intended number of clusters under unequal separations, heterogeneous dispersion, heavy tails, and outliers, where silhouette score or the Davies-Bouldin Index often fails. Applications to MNIST images and flow cytometry data illustrate the practical value of RWSD. In some cases where a cluster is not geodesically convex and lacks a meaningful depth center, the RWSD can fail.
The authors’ abstract, as arXiv lists it. arXiv’s listings, abstracts included, are free for anyone to reuse.
Read it on ReadSearch
Each opens after you create an account, and brings you back to this paper.
More in Methodology
Trustworthy Method Comparison with AI Judges: Estimation and Design under Order, Batch, and Aggregation Effects
Finite-Rank Logistic Gaussian Processes with Exact Likelihood for Conditional Density Estimation
A Functional Representation of Credit Behavior for Probability of Default Modeling
Cluster-Robust Prediction-Powered Inference
Beyond Marginal Monitoring: Distributed Joint-Distribution Testing for Data Concept Drift in Large Scale E-Commerce Operations
Get the few papers worth your time in Methodology every morning.
ReadSearch reads the new papers in your field each day and brings you the few worth reading, each with its key finding in a sentence.