arXiv preprint ·
Know-Thy-Not: Training-Free Targeted Negation in Dual-Encoder Vision-Language Models
- Published
- Source
- arXiv preprint, posted before peer review
- Original
- Read it at arXiv
- Topics
Abstract
Dual-encoder vision-language models systematically fail at negation, scoring ''a cat without a dog'' nearly identically to ''a cat with a dog.'' We show that this failure is inherent to the single-embedding framework. Under contrastive training, single-embedding methods can at best ignore the negated concept (dilution) but never penalize its presence (rejection). We propose Know-Thy-Not (KTN), a training-free method that introduces an image-side rejection dimension to penalize the negated concept directly in the visual representation. We also introduce COCO-Confusion, the first benchmark requiring genuine rejection to succeed. Across 9 backbones, KTN improves COCO-Confusion mAP by up to +25.6 over vanilla VLMs and +9.6 over the strongest existing method.
The authors’ abstract, as arXiv lists it. arXiv’s listings, abstracts included, are free for anyone to reuse.
Read it on ReadSearch
Each opens after you create an account, and brings you back to this paper.
More in Computer Vision
Meaningful Moments: A Corpus of VLM-Derived Temporal Importance Labels for Video Action Recognition
CÆNL: Collapse-Based Active Neural Learning with Objective-Aligned Representation Control
Adversarial Robustness of Camera-Radar Sensor Fusion for Autonomous Driving
Gaussian Splatting for Real-Time Scene Reconstruction: Methods, Systems, and Trade-offs
Removing the Compute Bottleneck for Event-based Visual Place Recognition with Spiking Neural Networks
Get the few papers worth your time in Computer Vision every morning.
ReadSearch reads the new papers in your field each day and brings you the few worth reading, each with its key finding in a sentence.