arXiv preprint ·
CF-AVSS: A Coarse-to-Fine Audio-Visual Speech Separation Framework with Target-Speaker Refinement and Post-Fusion for Real-World AVSE Challenge
- Published
- Source
- arXiv preprint, posted before peer review
- Original
- Read it at arXiv
- Topics
Abstract
Audio-visual speech enhancement (AVSE) leverages visual cues to recover target speech from mixtures corrupted by reverberation, noise, and overlapping speakers. We propose a coarse-to-fine two-stage AVSE framework. In the first stage, a progressive SPAV-TFGridNet is extended to two synchronized visual streams to produce visually ordered coarse estimates by progressively suppressing interfering speech while preserving target reverberation and shared background noise. In the second stage, a TFGridNet-based target speaker refinement network takes the original mixture and both coarse estimates as input to further recover target speech and improve signal quality. The coarse and refined estimates are finally combined using development-set-calibrated fusion. On the ISCSLP 2026 Real-World AVSE Challenge, our system tied for first place on Track 1 and ranked second on Track 2 in OVRL, while achieving the lowest CER on both tracks at 12.31% and 14.79%, respectively.
The authors’ abstract, as arXiv lists it. arXiv’s listings, abstracts included, are free for anyone to reuse.
Read it on ReadSearch
Each opens after you create an account, and brings you back to this paper.
More in Sound
CoSTA: Compressed Speech Token Adaptation for Efficient Spoken Dialogue Models
Too Emotional to Be Fake? Intra-Utterance Emotion Shifts Expose Failure Modes in Speech Deepfake Detectors
Full-Duplex Interaction with Speech, Expressions and Gestures
Semantic-Critical Regions for Video-to-Audio Generation
DuET: Note-Frame Dual Encoding for Piano Performance-to-Score Transcription
Get the few papers worth your time in Sound every morning.
ReadSearch reads the new papers in your field each day and brings you the few worth reading, each with its key finding in a sentence.