OpenAlex ·
RadMatch: Auditable Radiology Report Evaluation via Finding-Level Matching
- Published
- Source
- OpenAlex
- DOI
- 10.48550/arxiv.2609.01470
- Original
- Read it at the publisher
- Topics
Abstract
As AI systems are increasingly used to draft radiology reports, reliably evaluating their clinical quality remains a critical challenge. Large language model (LLM)-based metrics are now the best-correlated with radiologist judgment, yet they output a single opaque score that neither a clinician nor a model builder can easily interpret or audit. We introduce RadMatch, a multi-stage, LLM-based metric that decomposes report comparison into a structured finding-level matching with significance-aware scoring and error characterization across seven clinical attribute dimensions (status, location, severity, morphology, certainty, longitudinal comparison, and measurement). The main score is the actionable-error count, both interpretable and auditable. Candidate findings are graded correct, partial, or incorrect, and unmatched findings are counted as missed or hallucinated. Triage and actionable safety recall/precision and per-subset views add complementary, deployment-oriented lenses. Across two expert benchmarks, RadMatch is the most clinically aligned metric, matching inter-radiologist agreement on ReXVal and more than doubling the best prior metric on the harder RadEvalExpert. Relying only on few-shot prompting, it is designed to extend to other modalities and anatomies. We will release RadMatch as open-source code with an interactive dashboard for inspecting results.
The authors’ abstract, as arXiv lists it. arXiv’s listings, abstracts included, are free for anyone to reuse.
Read it on ReadSearch
Each opens after you create an account, and brings you back to this paper.
More in Computer Vision
Conditional Flow Matching for Cross-Field MRI Harmonisation
The 2024 MRSI Data Processing and Quantification Challenge Synthetic Dataset
Prior-Guided Implicit Neural Representations for Single-Subject Diffusion MRI Super-Resolution
A multicenter benchmark and clinically structured metric for coronary CTA report generation
Learning Task-Specific Antibody Representations via Function-Aware Masking
Get the few papers worth your time in Computer Vision every morning.
ReadSearch reads the new papers in your field each day and brings you the few worth reading, each with its key finding in a sentence.