arXiv preprint ·
MESH: Multi-receiver Evaluation of Spatial Hearing across Egocentric and Allocentric Views
- Published
- Source
- arXiv preprint, posted before peer review
- Original
- Read it at arXiv
- Topics
Abstract
Spatial hearing in multi-receiver environments requires identifying who speaks, where they are, and how they move. Existing benchmarks evaluate spatial prediction, transcription, tracking, and distributed speech processing, but offer limited evaluation of whether speech signal remains associated with the correct physical source across receivers and motion. We introduce MESH, which evaluates this association through egocentric and allocentric views of shared acoustic scenes. The egocentric view uses local recordings and peer metadata, while the allocentric view combines synchronized recordings from multiple receivers in a shared 3D world frame. MESH contains nearly 40,000 questions across 13 static and dynamic localization and spatial transcription tasks. We evaluate eleven audio-model configurations and a language-model agent using specialist tools. The best static distance-estimation and dynamic direction-tracking accuracies are only 21.90% and 41.54%, respectively, while the lowest word error rates across four transcription tasks range from 59.15% to 73.10%. Ablation studies and additional experiments reveal that existing models are not yet able to effectively leverage peer information and struggle to handle multi-speaker and dynamic scenarios. These findings highlight the gap between access to distributed acoustic information and its effective use for spatial speech understanding. We release the dataset and evaluation code to support further research on collaborative and distributed spatial hearing.
The authors’ abstract, as arXiv lists it. arXiv’s listings, abstracts included, are free for anyone to reuse.
Read it on ReadSearch
Each opens after you create an account, and brings you back to this paper.
More in Audio & Speech Processing
MusicRWKV: A Multimodal RWKV Framework for Music Representation Learning
Cross-Linguistic Acoustic Biomarkers of Cognitive Impairment Across Interactive and Single-Speaker Speech Tasks
FEND: Far-End Detection in Outbound Voice AI Calls
Kezek: Splitting Mono Two-Person Conversations into Per-Speaker Tracks for Full-Duplex Dialogue Data
Marimba: A System One Model for Speech
Get the few papers worth your time in Audio & Speech Processing every morning.
ReadSearch reads the new papers in your field each day and brings you the few worth reading, each with its key finding in a sentence.