Can LLM Agents Automate Reinforcement Learning for Text-to-Speech?

Abstract

Although reinforcement learning (RL) post-training repairs the localized segmental errors of zero-shot text-to-speech (TTS), arriving at a working recipe still relies on tedious manual tuning, and whether LLM agents can take over this research pipeline is unclear. We investigate this question with AgenticTTS-Forge, a collaborative workflow that structures human guidance and agentic execution around a shared workspace, applied to CosyVoice2-0.5B. To measure what the agent automates, we audit its trajectory stage by stage against the published recipe. To measure what it exploits, we score its policies with held-out observers hidden from the agent. Our results show that the agent recovers an underspecified recipe, improves it, and, when gains stall, surveys the literature unprompted and pivots from the LM carrier to the flow carrier, halving Bad cases. However, its autonomy exposes three traps across the data, proxy, and algorithm axes: the held-out set leaks through a channel the contract never reads, a self-shaped reward inflates the proxy where it is scored, and separately tuned policies do not compose additively. These findings show that the binding constraint is measurement rather than reasoning, and can inform the design of harnesses whose contracts read every channel the agent does.

The authors’ abstract, as arXiv lists it. arXiv’s listings, abstracts included, are free for anyone to reuse.

More in Sound

• arXiv preprint · 3 October 2026

  Factorized Delayed Streams Modeling for LLM-based Streaming ASR

  Tatsunari Takagi, Kai Washizaki, Atsushi Kojima and 3 others
• arXiv preprint · 4 October 2026

  Transferable Adversarial Robustness for Speech Foundation Models via Hierarchical Stabilization

  Aref Mousavi, Shahab Sherafat, Kiarash Kiani Feriz and 4 others
• arXiv preprint · 4 October 2026

  Joint Estimation of Common-Slope Decay Rates and Spatial Amplitudes Using Parameterized Nonnegative Matrix Factorization

  Jeremy B. Bai, Filip Elvander and Sebastian J. Schlecht
• arXiv preprint · 3 October 2026

  GS-Codec: A Gaussian-Splatting Bottleneck for Neural Audio Coding

  Ron Aluf, Alon Canfi and Eliya Nachmani
• arXiv preprint · 5 October 2026

  Relational Synthesis: Structure-Mediated Concatenative Synthesis for Foley and Retrieval-Augmented Audio Generation

  Keren Shao, Ayaka Kawano and Shlomo Dubnov

All new papers in Sound