Continuous-Time Acoustic Modelling with Neural Controlled Differential Equations

Abstract

Text-to-speech (TTS) models commonly address text--speech alignment by expanding phone-level encoder states to frame-level decoder inputs using predicted durations. While this length-regulation step resolves alignment structurally, this use of duration typically changes only where and how often latent states appear, not the values of the states themselves. This paper proposes a continuous-time mechanism for duration-aware acoustic modelling in TTS using neural controlled differential equations (CDEs). We formulate the phone representation as a temporally parameterised control path and use a neural acoustic vector field to produce a continuous-time hidden state whose values evolve with phonetic content and duration-derived timing. The resulting trajectory can be sampled at discrete points and integrated into a standard acoustic decoder pipeline. Objective results contrast CDEs and typical recurrent models. Subjective results suggest that CDE-based models evaluating one phone per step can improve rank-order agreement between synthesised and reference emotion intensity while maintaining comparable emotion-expression quality to a strong baseline. Additional experiments with half-phone step-sizes suggest that temporal resolution changes the trade-off between style tracking and absolute calibration. These results position CDEs as a promising design space for continuous-time and duration-aware style-sensitive TTS.

The authors’ abstract, as arXiv lists it. arXiv’s listings, abstracts included, are free for anyone to reuse.

More in Sound

• arXiv preprint · 10 September 2026

  Less can be More: What Aspects of Speech Drive End-of-Turn Detection

  Rini Sharon, Manickavela A, Kadri Hacioglu and 1 other
• arXiv preprint · 11 September 2026

  TokenMapper: A Step Toward Interoperable Speech Token Translation

  Tal Kozakov, Tal Rosenwein and Eliya Nachmani
• arXiv preprint · 10 September 2026

  Xiaomi-CocktailASR-1 Technical Report

  Yiru Zhang, Hang Su, Lichun Fan and 10 others
• arXiv preprint · 11 September 2026

  Direct Preference Density Alignment for Conversational Audio Equalization

  Ioannis Stylianou, Sven Ewan Shepstone, Jon Francombe and 2 others
• arXiv preprint · 10 September 2026

  Automatic Lyric Transcription for Greek Songs: Scaling and Task Composition Effects in Whisper Adaptation

  Maria Frangiadaki, Dimitrios Damianos, Kosmas Kritsis and 1 other

All new papers in Sound