arXiv preprint ·
Embedding Drift in Code Vulnerability Models Under Intended Behaviour-Preserving Transformations
- Published
- Source
- arXiv preprint, posted before peer review
- Original
- Read it at arXiv
- Topics
Abstract
Code edits that preserve program behaviour can shift frozen code embeddings across classifier decision boundaries, causing correctly detected vulnerabilities to be predicted as benign. Such instability is problematic because semantically neutral changes, including comment removal, insertion of unreachable code, variable renaming, and loop rewriting, should not alter a model's security judgment. We investigate this issue using 15,000 C/C++ functions organized as 7,500 vulnerable-patched pairs from the Big-Vul dataset. Four mutation tracks are applied to both vulnerable and benign samples: comment removal, unreachable code insertion, variable renaming, loop rewriting, and a combined setting that applies all transformations. Using frozen microsoft/codebert-base embeddings with six downstream classifiers, we evaluate the robustness of vulnerability predictions and introduce a train-only defence that projects clean and mutated representations closer together while preserving vulnerability-class information. Under combined mutations, the baseline Vulnerable Flip Rate (VFR) ranges from 35.55% to 42.15%. The proposed defence reduces the mean common-set VFR from 22.60% to 11.32% and improves mean mutated accuracy from 55.33% to 57.10%. For Logistic Regression, VFR decreases by 17.29 percentage points, although benign-to-vulnerable flips increase by 11.95 percentage points. These results indicate that the defence improves robustness to semantics-preserving code transformations but introduces a false-positive trade-off and does not fully eliminate prediction instability.
The authors’ abstract, as arXiv lists it. arXiv’s listings, abstracts included, are free for anyone to reuse.
Read it on ReadSearch
Each opens after you create an account, and brings you back to this paper.
More in Software Engineering
SWE-Proof: Can Language Models Resolve Real-World Issues with Machine-Checked Proofs?
From Code to Requirements: Agentic Reverse Engineering of Business Rules at Enterprise Scale
Towards the Generalizability of Leveraging ChatGPT in APR via Self-enhancing: An Empirical Study
License Compliance in Open Source Cybersecurity Projects
"It Comes in Notebooks": Changes and Challenges when Operationalizing ML Prototypes
Get the few papers worth your time in Software Engineering every morning.
ReadSearch reads the new papers in your field each day and brings you the few worth reading, each with its key finding in a sentence.