Which Code Changes Need Re-embedding? CIPBench and the Limits of Multi-Signal Index Maintenance

Abstract

Repository-level retrieval-augmented generation systems must keep vector indexes synchronized with continuously evolving code, yet there is no principled basis for deciding which changed code units actually need new embeddings. We ask whether the retrieval impact of a code change can be anticipated before that embedding is computed. Across 10 open-source repositories in Python, Go, and JavaScript, we characterize how a change propagates from syntactic substantiality through embedding displacement to retrieval impact. The links are informative but not deterministic: syntactic class separates displacement sharply, yet impact depends on a unit's retrieval exposure as much as on how far its representation moves, and recurring counterexamples bound what any single signal can decide. We formalize the decision as CIPBench, a benchmark that scores maintenance policies under a strict pre-embedding information boundary. Prediction works, and one signal exhausts it. A ranker using only a unit's change history substantially reduces retrieval harm relative to random and recency-based scheduling. Combining it with three further signals--individually discriminative and largely unassociated with one another--leaves the decision statistically unchanged across every budget we evaluate, for reasons we trace to the structure of the score rather than to its tuning. What pays off is not ranking but filtering: units whose text is unchanged from the indexed state can be skipped with no risk and, in the most reversion-heavy project, at nearly half the embedding cost. Signal-level informativeness is therefore not evidence of decision-level value, and separating the two requires evaluating the decision a signal produces rather than its association with the outcome.

The authors’ abstract, as arXiv lists it. arXiv’s listings, abstracts included, are free for anyone to reuse.

More in Databases

• arXiv preprint · 18 October 2026

  HCFM: Helmholtz Component Flow Matching for Time Series Anomaly Detection

  Shyam Sundar Saravanan, Tie Luo and Sanjay Madria
• arXiv preprint · 19 October 2026

  MultiCard: Multiple Attribute Cardinality Estimation via Deep Learning

  Mobina Koochaknia, Maryam Ramezani and Mohammad Hasan Shafaat
• arXiv preprint · 19 October 2026

  SweepANN: A GPU-Accelerated Large-K Approximate Vector Search for Drug Discovery

  Zhenyu Zhang, Qi Lin, Nan Zhang and 1 other
• arXiv preprint · 18 October 2026

  RemoteANN: A Native Vector Search Engine for Cloud Object Storage

  Yutong Gou, Silu Huang, Yue Wang and 2 others
• arXiv preprint · 20 October 2026

  TriGeoBench: A Multimodal Mathematical Geometry Benchmark with Figures and TikZ

  Jing Wang, Yitong Yang, Hao Dou and 6 others

All new papers in Databases