arXiv preprint ·
JevBreak: Your Model, Their Decision
- Published
- Source
- arXiv preprint, posted before peer review
- Original
- Read it at arXiv
- Topics
Abstract
Restricting what a model can say is often mistaken for restricting an attacker's influence over its decision. An attacker can add text to the evidence and steer the model toward a wrong answer that still belongs to the allowed set. We evaluate seven decision models spanning hosted APIs, open-weight models and an encoder on 16 tasks in seven domains from Decision Bench, targeting every wrong option across 240 text examples (990 targets per model). We present Slot Attack, a black-box attack designed for decision models that appends text assembled from named components to steer predictions toward an attacker-selected incorrect option. The method relies solely on returned label probabilities, achieves success rates of 32--99\% on initially correct decisions, and identifies the effective component combinations with a median of 17--72 queries per successful attack. Accuracy and confidence do not reliably reveal this vulnerability. Models with near-identical accuracy on the sampled tasks differ by about 20\%age points in attack success. We demonstrate that an attacker can force not just one wrong answer but every available one: on about 20\% of correct decisions for the JEV models, 77\% for gpt-6-luna and 98\% for GLiNER2.5; a confidence threshold does not reliably screen the attack out. A fixed answer set constrains a decision's form, not its source; robustness must be measured per model, per task and per version rather than assumed from output format.
The authors’ abstract, as arXiv lists it. arXiv’s listings, abstracts included, are free for anyone to reuse.
Read it on ReadSearch
Each opens after you create an account, and brings you back to this paper.
More in Cryptography & Security
Accessed, never disclosed: Silent access to protected health information by medical artificial intelligence agents
ScrambleMark: Zero-Query Watermark-Agnostic Black-Box Evasion of Modern Speech Watermarking
Malware Clustering in the Wild: Taxonomy, Challenges, and Evaluation Metrics
Say the Same, Act Differently: Diagnosing and Steering Text-Action Divergence in Reasoning VLAs
Networked Synthetic Social Engineering on the Web
Get the few papers worth your time in Cryptography & Security every morning.
ReadSearch reads the new papers in your field each day and brings you the few worth reading, each with its key finding in a sentence.