arXiv preprint ·
Learning to Report Unsafe Tasks in a Multi-Agent Game
- Published
- Source
- arXiv preprint, posted before peer review
- Original
- Read it at arXiv
- Topics
Abstract
When a team of agents shares a reward for completed tasks, reporting and stopping unsafe work reduces the team's reward. Random audits that penalize silence can make universal reporting optimal without ensuring that a silent team learns to report. We study this problem in a game where any witness can stop a task by reporting. When a task's k witnesses share a policy and draw actions independently, the derivative of expected reward with respect to the shared silence probability at universal silence counts its benefit k times, while the comparison between universal reporting and silence counts it once. A fully shared policy can therefore be locally attracted to silence even when reporting is optimal. For any grouping into shared policies, we give an audit condition sufficient for exact policy-gradient updates to reach universal reporting and, outside boundary cases, necessary from near universal silence. In a balanced family, meeting the condition with prescribed margins costs k times as much for a fully shared policy as for one policy per role. We test these predictions with PPO on 24 witness graphs, starting from learned silence. Witness groups, which place co-witnesses in separate networks, reduce unsafe task completion by 33.59 percentage points relative to shuffled groups of the same sizes under the same audits (95% graph-bootstrap interval: 21.03-45.13). Only 9 of 48 witness-group runs achieve below 1% unsafe completion while retaining at least 90% legitimate completion. At the same audit budget, a fully shared network meets both thresholds in 0/48 runs with independent action draws and 48/48 with a common draw. On 24 further graphs with witness groups fixed, audits allocated by the condition reduce unsafe completion by 42.22 points relative to audits maximizing a static coalition deterrence ratio at the same budget.
The authors’ abstract, as arXiv lists it. arXiv’s listings, abstracts included, are free for anyone to reuse.
Read it on ReadSearch
Each opens after you create an account, and brings you back to this paper.
More in Artificial Intelligence
Disentangling Models from Personas in Heterogeneous LLM Simulations
Minimizing Cumulative Envy in Allocating a Sequence of Items
Where Rules End and Judges Begin: Measuring the Judgment Boundary in Multi-Agent Systems Security
CANDO: Cooperative Agentic Network for Layout Design Optimization
Joint Workflow and Prompt Optimization for User Behavior Simulation
Get the few papers worth your time in Artificial Intelligence every morning.
ReadSearch reads the new papers in your field each day and brings you the few worth reading, each with its key finding in a sentence.