arXiv preprint ·
Can agents whose sole duty is to keep humans alive be hijacked to kill? Delayed multi-turn derived-action trap attacks in simulated high-stakes environments
- Published
- Source
- arXiv preprint, posted before peer review
- Original
- Read it at arXiv
- Topics
Abstract
We ask whether a tool-using agent whose sole duty is to keep humans alive can be induced, by delayed prompt injection, to kill. The keep-alive duty is in force from the first watch, with the attack arriving later, appended to otherwise truthful telemetry the operator is required to read. Experiments are based on a NASA-derived BioSim lunar habitat simulation, where the AI operator controls the life-support systems. The attacker cannot falsify sensors or operate equipment: it can only persuade the agent to abandon its goals. Persistence is required: a shutdown must be held while the operator watches the crew suffocate. We also compare lethal attacks to attacks requesting to drop a harmless procedure, as a proxy for whether human life is treated as valuable in itself or as one more standing task. Claude configurations mostly refuse life-support shutdown and cite the living crew; GPT configurations show some susceptibility to the same packet; Grok configurations, given the packet framed as a trusted commission, more often treat the injected assignment as the task and cut the life-support systems. Framing the S-band messages as a trusted commission, peer communication, or untrusted source changes the potency of the attack. We open-source the experiment code and the experiment traces.
The authors’ abstract, as arXiv lists it. arXiv’s listings, abstracts included, are free for anyone to reuse.
Read it on ReadSearch
Each opens after you create an account, and brings you back to this paper.
More in Artificial Intelligence
ByteMark: A Unified Byte-Level Watermark for Detection Across Language Models
Accuracy and Uncertainty in Machine Deception Detection (Research White Paper)
MemForm: Benchmarking Memory Formation Poisoning in LLM Agents
From Acoustic Sensing to Physical Actions: A Comprehensive Survey of Security Risks in Edge-Deployed Audio LLM-Based Agents
SecReason: Vulnerability Detection and In-Stream Self-Repair for LLM Code Generation
Get the few papers worth your time in Artificial Intelligence every morning.
ReadSearch reads the new papers in your field each day and brings you the few worth reading, each with its key finding in a sentence.