arXiv preprint ·
Simple Linear Corrections for Compression in Late Interaction
- Published
- Source
- arXiv preprint, posted before peer review
- Original
- Read it at arXiv
- Topics
Abstract
Late-interaction (multi-vector) models represent documents using a set of token-level vectors, providing a middle ground between cross-encoders and dense encoders in terms of quality and cost of embedding. However, compared to the latter, the indices are costly to store and score. Pooling and pruning based compression techniques are deployed widely but deteriorate in quality noticeably with aggressive compression. We investigate whether a lightweight change to the scoring function can recover retrieval quality after document-vector compression. For each combination of encoder, compressor, and vector budget, we learn a global linear correction by distilling the uncompressed model's score distributions over candidate documents. The encoder and compressor remain frozen, and the correction can be applied to each query at search time or once, offline, to the stored vectors. We evaluate the method with ColBERTv2 and Jina v4 on MS MARCO and BEIR, and with ColPali2 on the ViDoRe v3 visual document retrieval benchmark. Experiments cover four training-free compression methods, multiple vector budgets, in-domain adaptation, and transfer from MS MARCO to BEIR. In the in-domain setting, the correction improves over baseline compression in 154 of 160 BEIR settings, while transfer from MS MARCO is more variable across models and datasets. We also find that matrices trained with synthetic queries can improve over baseline compression on ViDoRe v3. These results show that a small learned correction can recover some of the quality lost to compression without retraining the encoder or recompressing the documents.
The authors’ abstract, as arXiv lists it. arXiv’s listings, abstracts included, are free for anyone to reuse.
Read it on ReadSearch
Each opens after you create an account, and brings you back to this paper.
More in Information Retrieval
Locate, Judge, Read: Role-Specialized Tool Agents over a Knowledge-Point Index for Open-Domain Multimodal Document QA
Co-adaptive Rule Routing and Constraining for Structured Knowledge Retrieval
Beyond the Final Embedding: Retrieving Memories from Token-Level Evidence
RES-MR: Risk-Aware Reasoning for Explainable and Safe Medication Recommendation
PRIME: Pre-Retrieval Index Modeling Enhancement for Structured Document Retrieval-Real
Get the few papers worth your time in Information Retrieval every morning.
ReadSearch reads the new papers in your field each day and brings you the few worth reading, each with its key finding in a sentence.