arXiv preprint ·
SurgEye: Instruction Tuning-Free Perception-Reasoning Framework for Surgical Scene Understanding
- Published
- Source
- arXiv preprint, posted before peer review
- Original
- Read it at arXiv
- Topics
Abstract
Surgical scene understanding is pivotal for computer-assisted intervention, yet demands compositional reasoning across instruments, anatomy, and workflow. Existing surgical vision-language models rely on instruction tuning with large-scale question-answer pairs, entailing laborious data curation and costly retraining. To bridge this gap, we propose SurgEye, a perception-reasoning framework that grounds reasoning in a vision foundation model without instruction tuning. Specifically, we devise Routed Multi-Task Perception (RMP) that routes the vision foundation model through task-specific pathways learned from public perception labels, producing spatial and temporal evidence. We further introduce Agentic Compositional Reasoning (ACR) that employs a frozen large language model to coordinate RMP specialists and temporal memory as callable tools, composing evidence across tasks and time into grounded answers. Moreover, we construct SurgEye-Bench spanning eight surgical procedures, pairing each question with perception labels for verifiable evaluation. Extensive experiments demonstrate that SurgEye outperforms state-of-the-arts in surgical reasoning. The source code will be available.
The authors’ abstract, as arXiv lists it. arXiv’s listings, abstracts included, are free for anyone to reuse.
Read it on ReadSearch
Each opens after you create an account, and brings you back to this paper.
More in Computer Vision
The DynaCell Evaluation Suite for Dynamic 3D Virtual Staining of Live Cells
CHROMA-MILL: Joint Low-Light Enhancement and White Balance under Colored Illumination
Findings-aware Language-Image Pretraining for CT Volume Interpretation
Is This 3D Asset Usable? Beyond the Beauty Render for Asset Curation
AgiWorld: An Agentic Video World Model for Building Persistent Worlds
Get the few papers worth your time in Computer Vision every morning.
ReadSearch reads the new papers in your field each day and brings you the few worth reading, each with its key finding in a sentence.