Asymptotic Properties of Support Vector Machines in High-Dimension, Low-Sample-Size Settings under a Spiked Model

Abstract

In this paper, we consider asymptotic properties of the support vector machine (SVM) in high-dimension, low-sample-size (HDLSS) settings under a spiked model. The existing theory of the SVM in the HDLSS context relies on the geometric representation of HDLSS data, which requires that the eigenvalues of the covariance matrices are not dominant. We first show that the geometric representation does not hold under the spiked model. We show that the Gram matrix of HDLSS data converges in distribution to a random matrix, namely, the HDLSS data converge to a random configuration in a finite-dimensional space whose dimension is given by the number of the spikes. We show that the misclassification rates of the SVM do not tend to zero, that is, the SVM does not hold the consistency property. We also show that the bias-corrected SVM (BC-SVM) does not give preferable performance in this setting because the bias term itself should be modified. In order to overcome such difficulties, we propose a spike-corrected SVM (SC-SVM). We show that the SC-SVM holds the consistency property when the sample size goes to infinity, and that the growth of the sample size is essential in the sense that any projection-based procedure fails when the sample size is fixed. Finally, we check the performance of the classifiers by numerical simulations.

The authors’ abstract, as arXiv lists it. arXiv’s listings, abstracts included, are free for anyone to reuse.

More in Machine Learning (Statistics)

• arXiv preprint · 30 September 2026

  Optimal VC Dimension of Contrastive Learning with Margin

  Dionysis Arvanitakis, Vaggos Chatziafratis, Yiyuan Luo and 1 other
• arXiv preprint · 1 October 2026

  Learning Multiple Timescales for Goal-Conditioned Reinforcement Learning

  Pedro Robles Dutenhefner, Dikshant Shehmar, Wagner Meira and 1 other
• arXiv preprint · 30 September 2026

  Optimal Allocation and Volume under Surface

  Kai Feng, Han Hong, Jessie Li and 1 other
• arXiv preprint · 1 October 2026

  Block Optimism for Nonstationary Bandits with Latent Linear Dynamics

  Taehyun Hwang, Hyunjun Choi, Heesang Ann and 1 other
• arXiv preprint · 30 September 2026

  Sharp Statistical Rates for Asynchronous TD Learning with Markovian Data

  Yang Peng

All new papers in Machine Learning (Statistics)