Interpretable Singing Emotion Recognition Using Acoustic and Vocal Technique Features

Abstract

Singing emotion recognition relies predominantly on deep learning models that achieve strong performance but offer limited insight into the vocal characteristics driving emotion, of- ten overlooking the singing voice as a distinct signal. We propose an interpretable singing emotion recognition framework using a compact set of singing-specific descriptors across prosody, spec- tral features, phonetics, and vocal technique. To decode model decisions and link predictions to measurable acoustic phenomena, we employ SHAP and apply GRAD-CAM to qualitatively assess the learned representations of vocal techniques. Evaluations show that singing-specific features reduce cross-singer variability, while vocal technique descriptors provide interpretable insights into singing expression. Furthermore, out-of-domain evaluation on VocalSet reveals interesting associations between emotional expression and ground-truth technical execution. The analysis highlights vibrato and pharyngeal techniques as salient cues of emotional delivery, with SHAP indicating that vibrato is context-dependent on articulation rate. These findings emphasize the value of domain-specific features for interpretable singing emotion analysis.

The authors’ abstract, as arXiv lists it. arXiv’s listings, abstracts included, are free for anyone to reuse.

More in Sound

• arXiv preprint · 23 October 2026

  MI-MIDI 2: From Observation to Control with Sparse Autoencoders in Text-to-MIDI Generation

  Jakub Poćwiardowski and Mateusz Modrzejewski
• arXiv preprint · 25 October 2026

  EmoNet: A Lightweight Attention-Based CNN for Speech Emotion Recognition

  Muhammad Moiz Khalid, Abdur Rehman Tariq and Muhammad Okasha Yamin
• arXiv preprint · 23 October 2026

  DIAL-S2ST: Learning Latency Control for Simultaneous Speech-to-Speech Translation

  Jeongsoo Choi, Junseok Ahn and Joon Son Chung
• arXiv preprint · 25 October 2026

  AudioFM: Towards General Purpose Audio Representation Learning

  Yamin Li, Ruidong Zhang, Anran Wang and 19 others
• arXiv preprint · 23 October 2026

  Supervised Source Separation of Electric Vehicle Interior Noise Using a Hybrid Transformer Architecture

  Philipp Diel, Zhuo Cao, Arne Hildenbrand and 4 others

All new papers in Sound