arXiv preprint ·
Learnable Classifier-Free Guidance Null Embeddings for Enhanced Controllable Speech Synthesis
- Published
- Source
- arXiv preprint, posted before peer review
- Original
- Read it at arXiv
- Topics
Abstract
Classifier-free Guidance (CFG) is widely adopted in text-to-speech (TTS) systems to enhance generation quality and conditioning fidelity by interpolating between conditioned and unconditioned predictions. A common unconditional technique is to use an empty representation, in the form of a fixed null vector. In this work, we propose replacing this representation with a learnable unconditional embedding, optimized to represent a meaningful unconditional state. Objective and subjective evaluations demonstrate that learnable null embeddings consistently outperform fixed null embeddings across speaker similarity, speech stability, and expressiveness, while exhibiting greater robustness to larger guidance scales. We further show that learning a distinct unconditional embedding for each of the TTS conditioning modalities allows fine-grained control over speaker and text guidance, showcasing the trade-off between similarity and quality, and stability and expressiveness in the generated speech.
The authors’ abstract, as arXiv lists it. arXiv’s listings, abstracts included, are free for anyone to reuse.
Read it on ReadSearch
Each opens after you create an account, and brings you back to this paper.
More in Audio & Speech Processing
AURA: Uncertainty-Routed Activation Editing for Acoustic Grounding in Speech Foundation Models
Synthesis and editing of multi-instrument audio mixtures using scalar-quantised latents with MIDI Span conditioning
MECT: Mixture of Experts with CNN-Transformer Network for Speaker verification
SE-MSB: End-to-End Unpaired Speech Enhancement using Mamba Schrödinger Bridges
P2Flow: Phoneme-aware Progressive Flow Matching for Extreme Speech Super-Resolution
Get the few papers worth your time in Audio & Speech Processing every morning.
ReadSearch reads the new papers in your field each day and brings you the few worth reading, each with its key finding in a sentence.