Learning Kernels by Alignment for Multiclass Bayes Classification

Abstract

Kernel methods separate data representation from decision-making, but typically require the kernel to be chosen in advance. We show that this kernel can instead be learned by alignment, and develop the resulting framework through the recently introduced Collaborative Learning and Inference (CLaI). We show that Collaborative Learning can be viewed as a kernel alignment process, in which an embedding is trained so that its induced similarity matches a label-derived target kernel. We also prove that Collaborative Inference is equivalent to kernel Bayes classification with Parzen-window density estimation. Motivated by these perspectives, we generalise CLaI by replacing cosine similarity with a learned Mahalanobis distance and extend it to multiclass classification. On CIFAR-10, PathMNIST, and SleepEDF, the Mahalanobis formulation improves accuracy, converges faster, and yields lower calibration error than the cosine-based variant. Auxiliary experiments further support these connections, showing that CLaI produces latent signals of the same form as a Gaussian process, while achieving competitive calibration on sepsis prediction. Together, these results establish a principled learned-kernel framework that unifies representation learning, kernel alignment, and Bayesian classification, and extends naturally to the multiclass setting.

The authors’ abstract, as arXiv lists it. arXiv’s listings, abstracts included, are free for anyone to reuse.

More in Machine Learning

• arXiv preprint · 6 September 2026

  Deep learning from the crowd Fundamentals of morphological galaxy classification

  Luis Enrique Sucar, Carlos del Burgo and Jonathan Serrano-Pérez
• arXiv preprint · 7 September 2026

  Contextual Observer Grounding: Evaluating Situated Spatial Reasoning in Vision-Language Models

  Mimo Shirasaka, Haochen Zhang and Yonatan Bisk
• arXiv preprint · 6 September 2026

  One Shared LoRA Weight for MRI Reconstruction across Acceleration Factors

  Zhiwei Zhao, Weikang Gong, Zhongnian Li and 1 other
• arXiv preprint · 7 September 2026

  Learning Adaptive SED for heterogeneous load balancing

  Sanne van Kempen, Jaron Sanders, Fiona Sloothaak and 1 other
• arXiv preprint · 6 September 2026

  Linear Algebra Foundations of Efficient Attention: A Phase Reversal in Rank Collapse Under SVD Compression

  Anjaneya Teja Sarma Kalvakolanu

All new papers in Machine Learning