Beyond Pixel Similarity: Task-Aware Evaluation of GAN-Based Synthetic Sonar Data for Robotic Perception

Abstract

Synthetic data can reduce the cost of collecting and annotating training data for robotic perception, but generating sensor observations that preserve the characteristics relevant to downstream perception remains challenging, particularly for sonar imagery. In this work, we investigate whether conventional image-fidelity metrics adequately reflect the downstream perception performance of GAN-generated synthetic sonar data. We employ a Pix2Pix conditional generative adversarial network with four discriminator configurations characterized by different receptive fields: PixelGAN, PatchGAN-16, PatchGAN-70, and ImageGAN. The models are trained using sonar imagery from two datasets and evaluated using conventional image-fidelity metrics, including Structural Similarity Index (SSIM), Peak Signal-to-Noise Ratio (PSNR), and Mean Squared Error (MSE). To complement these pixel-level measures with task-oriented evaluation, YOLOX-S, YOLOX-L, and Faster R-CNN detectors are trained exclusively on real sonar imagery and subsequently evaluated on the GAN-generated images using identical test samples and annotations across all discriminator configurations. The results reveal a discrepancy between image-fidelity and downstream object-detection performance: the configuration achieving the best SSIM, PSNR, and MSE does not consistently yield the best detection performance. In particular, PatchGAN configurations achieve strong downstream detection results despite not achieving the highest pixel-level similarity scores. These findings suggest, for the datasets and models considered, pixel-level image-fidelity metrics alone may not consistently capture the task-relevant realism of synthetic sonar observations and motivate the use of task-aware evaluation for synthetic sensor data intended for robotic perception.

The authors’ abstract, as arXiv lists it. arXiv’s listings, abstracts included, are free for anyone to reuse.

More in Robotics

• arXiv preprint · 16 September 2026

  Feedback-Modulated Harmonic Policies for Quadruped Locomotion

  Yixuan Jia, Steven Roche and Jonathan P. How
• arXiv preprint · 17 September 2026

  CoreSense: Traceable Failure Recall and Conflict-Aware Belief Gating for Auditable Robot Decisions

  Zoe Li
• arXiv preprint · 16 September 2026

  Missing Bridges: Composition-Aware Active Imitation Learning

  Maxwell J. Jacobson, Ahmed H Qureshi and Yexiang Xue
• arXiv preprint · 17 September 2026

  AMB3R-SLAM: Kilometer-scale SLAM with Hierarchical Backend

  Hengyi Wang and Lourdes Agapito
• arXiv preprint · 16 September 2026

  Causal-History Test-Time Scaling for Failure Recovery in Autoregressive World-Action Models

  Lin Li, Long Chen, Kwunhang and 10 others

All new papers in Robotics