arXiv preprint ·
RetryBench: Separating Retry Permission from Recovery Outcomes in GPT Tool Use
- Published
- Source
- arXiv preprint, posted before peer review
- Original
- Read it at arXiv
- Topics
Abstract
False retry permission (FRP) occurs when an agent retries a timed-out tool call without support from the visible contract. RetryBench separates this error from realized recovery. Successful protected recovery does not imply reliable rejection of unsupported retries. In a 720-response key control, valid keys usually enabled safe progress, but offered changed keys elicited unsupported retries in Sol (1/80), Terra (55/80), and Luna (60/80). A later 240-response control removed the candidate and required models to supply a key. Luna still made 8/80 unsupported dispatches; Sol and Terra made none in 80 responses each. Accepting a displayed bad key therefore cannot explain all observed errors, although changed output requirements and conditions preclude a pure display-effect comparison. Removing the weak-contract label increased retries in Sol and Terra's coded-action conditions in a six-endpoint study. A separate 1,536-response canonical-action block also found a positive Terra contrast (115/128 versus 124/128 retries), concentrated in two task/order conditions; the other five contrasts remained unresolved after multiplicity correction. An explicit-contract cue control elicited deferral in all 762 returned responses, leaving recovery unresolved. These results support reporting permission and recovery together across declared interface variants. The evidence covers four purposive tasks, fixed GPT endpoints, and one recovery decision; it identifies neither internal mechanisms nor episode-level safety.
The authors’ abstract, as arXiv lists it. arXiv’s listings, abstracts included, are free for anyone to reuse.
Read it on ReadSearch
Each opens after you create an account, and brings you back to this paper.
More in Artificial Intelligence
Accuracy and Uncertainty in Machine Deception Detection (Research White Paper)
Adversarial Robustness of Camera-Radar Sensor Fusion for Autonomous Driving
From Acoustic Sensing to Physical Actions: A Comprehensive Survey of Security Risks in Edge-Deployed Audio LLM-Based Agents
When Agents Talk, Secrets Travel: Topology-Driven Leakage and Recipient-Conditioned Sanitization in Multi-Agent LLM Systems
Security in LLM-Assisted Coding: A Systematic Literature Review
Get the few papers worth your time in Artificial Intelligence every morning.
ReadSearch reads the new papers in your field each day and brings you the few worth reading, each with its key finding in a sentence.