arXiv preprint ·
Knowing When to Escalate: Disagreement Patterns for Uncertainty-Aware Content Moderation
- Published
- Source
- arXiv preprint, posted before peer review
- DOI
- 10.1145/3799682.3840059
- Original
- Read it at arXiv
- Topics
Abstract
Agentic content-moderation systems increasingly rely on collective judgments from LLM agents, while aggregation methods commonly treat inter-agent disagreement as a reliability problem to be removed through consensus or voting. This framing is inappropriate for value-laden tasks, where disagreement may reflect irreducible value pluralism - a distinction standard uncertainty decompositions cannot recover, since they operate on output distributions and discard the reasoning behind each judgment. We propose Reasoning Trace Divergence Analysis (RTDA), which embeds model-generated justifications from perspective-differentiated agents and classifies each case by justification similarity and conclusion agreement. The resulting four-state taxonomy isolates convergent disagreement, where agents reasoning similarly but concluding differently, as a candidate signature of value-laden conflict. On 1,000 items from the Measuring Hate Speech corpus, this structural signal predicts human annotator disagreement better than magnitude-only and output-only baselines. We contribute a design principle: condition human escalation on the structure of disagreement, not its magnitude alone, with RTDA serving as a post-processing layer for systems that already generate reasoning traces.
The authors’ abstract, as arXiv lists it. arXiv’s listings, abstracts included, are free for anyone to reuse.
Read it on ReadSearch
Each opens after you create an account, and brings you back to this paper.
More in Multiagent Systems
SlowLab: Benchmarking Autonomous AI Agents on a Year of Experiments They Cannot Undo
GarmentSegAgent: Autonomous Garment Segmentation for Web Fashion Images via Textual Self-Refinement
Collective Emergent Alignment: Measuring and Amplifying Interaction-Induced Reasoning Alignment in Multi-Agent LLMs
Remembering Is Not Enough: Refining Memories and Learning When to Use Them in Multi-Agent Systems
ClinVerse: A Realistic Multi-Agent Diagnostic Environment for Training and Evaluating Medical Agents
Get the few papers worth your time in Multiagent Systems every morning.
ReadSearch reads the new papers in your field each day and brings you the few worth reading, each with its key finding in a sentence.