arXiv preprint ·
Overtaught, Underlearned: Learner-Centered Distillation from Multi-Agent Discussion
- Published
- Source
- arXiv preprint, posted before peer review
- Original
- Read it at arXiv
- Topics
Abstract
Multi-agent discussion produces reasoning collaboratively, while deployment still relies on a smaller student that must answer alone. Full-graph distillation transfers such discussions by treating each correct solution as a separate training target, but what teachers provide need not match what the student learns. This mismatch appears in two ways: several correct solutions may amount to the same lesson in the student's representation (over-teaching), while some training questions remain missed after learning (under-learning). We quantify both with rates that are high on discussion graphs. To address them, we introduce FAR (Fold--Attempt--Revisit), a learner-centered distillation procedure that organizes supervision around the student's own state. Before training, FAR folds solutions within each discussion state into representative lessons. After training, the student's own attempts identify missed questions, while poorly represented clusters indicate where folding may have lost useful variation; FAR then writes targeted lessons from retained discussion contexts. Across two students and four reasoning tasks, FAR outperforms full-graph distillation with far fewer training targets and lower under-learning. Further analyses examine the behavior of FAR and the sources of its gains. Overall, the results show that organizing discussion supervision around the student can make distillation both more effective and more efficient.
The authors’ abstract, as arXiv lists it. arXiv’s listings, abstracts included, are free for anyone to reuse.
Read it on ReadSearch
Each opens after you create an account, and brings you back to this paper.
More in Artificial Intelligence
Computational Reflective Equilibrium for Agentic AI: A Rawlsian Alignment Scaffold for Governance
Foundation Models for Event Prediction Across Heterogeneous Event Logs
When Does Compensating Inherited Quantization Error Help? Mean-Square Guarantees for Layer-Wise Quantization and Their Limits
YUBI-Real2Sim: 3DGS Scene Verification for Reliable Policy Evaluation
Of Errors and Echoes: Responsibility-Aware Credit Assignment for Tool-Use Agents
Get the few papers worth your time in Artificial Intelligence every morning.
ReadSearch reads the new papers in your field each day and brings you the few worth reading, each with its key finding in a sentence.