arXiv preprint ·
[Re] Boosting the Visual Interpretability of CLIP via Adversarial Fine-Tuning
- Published
- Source
- arXiv preprint, posted before peer review
- Original
- Read it at arXiv
- Topics
Abstract
This paper presents a reproducibility study of ``Boosting the Visual Interpretability of CLIP via Adversarial Fine-Tuning'', published at ICLR 2025, which proposes an unsupervised adversarial fine-tuning (AFT) method with norm regularisation to enhance the visual interpretability of CLIP's image encoder. We attempt to reproduce the key claims regarding improved saliency map quality, increased concept alignment, transferability to out-of-distribution datasets, and the trade-off with zero-shot accuracy. Beyond reproduction, we propose a saliency-guided regularisation extension that introduces an Energy Pointing Game loss, directly supervising the spatial alignment of Simple Gradient saliency maps with target objects. We evaluate our extension across a range of saliency-loss weights and show that explicit saliency supervision \rev{substantially} improves localisation metrics at a modest cost: the reduction in zero-shot accuracy and adversarial robustness is small at the lowest saliency-loss weight ($w_s=0.2$) and moderate at the strongest setting ($w_s=0.6$). We restructure the codebase for configurable, systematic experimentation.
The authors’ abstract, as arXiv lists it. arXiv’s listings, abstracts included, are free for anyone to reuse.
Read it on ReadSearch
Each opens after you create an account, and brings you back to this paper.
More in Computer Vision
Design Choices for Multimodal Transformer Learning with Missing and Longitudinal Healthcare Data
Automated carotid artery atherosclerotic inflammation segmentation in PET using MSR-Net
SceneForesee: Learning Scene Evolution for Spatiotemporal Reasoning
GeoBridge: Evolving Transferable Decoder Topologies for Geometric Vision
FreeCHROMA: Generative Chroma Keying from Unaligned Background References
Get the few papers worth your time in Computer Vision every morning.
ReadSearch reads the new papers in your field each day and brings you the few worth reading, each with its key finding in a sentence.