arXiv preprint ·
Evaluating Synthetic Skin Lesion Images for Classifier Training: A Safety-Aware Algorithm
- Published
- Source
- arXiv preprint, posted before peer review
- Original
- Read it at arXiv
- Topics
Abstract
Skin cancer classifiers need large sets of annotated images, which are scarce, especially for rare lesion types. AI-generated images can fill the gap, but a generator produces images of widely varying quality, so selecting which images to retain is important. That choice usually rests on a single quality score, most often the Fréchet Inception Distance (FID), which has not been tested against what a classifier needs and cannot tell whether the generator memorizes its training photographs. We test that practice with a tournament of 16 quality metrics. Each metric alone selects a training subset from a pool of 275,472 images generated by a diffusion model (EDM2), and a classifier trained on each subset scores the metric per class. The tournament shows that no single metric serves every class: FID computed on Inception features gives the best classifier for one lesion type and no measurable advantage over random sampling for another, and the generation setting it prefers is also the one closest to the training images. We turn these per-class measurements into a safety-aware evaluation algorithm that weights each metric by its measured value in each class and removes images too close to a real training image. On the held-out ISIC-2018 test set, the algorithm improves Macro-$F_2$, a recall-weighted measure of classification performance, over the FID baseline for a Swin-B classifier by +0.0136 to +0.0164 across four safety thresholds (Bonferroni-corrected $p < 0.02$) and over random sampling by about +0.03, though not for ResNet-50. Tightening the threshold improves a memorization-safety score with no measurable change in utility, but every curated set remains less safe than random sampling. The results show a modest but statistically significant gain on the primary classifier, and offer a practical, safety-aware procedure for evaluating and using synthetic data.
The authors’ abstract, as arXiv lists it. arXiv’s listings, abstracts included, are free for anyone to reuse.
Read it on ReadSearch
Each opens after you create an account, and brings you back to this paper.
More in Computer Vision
The DynaCell Evaluation Suite for Dynamic 3D Virtual Staining of Live Cells
CHROMA-MILL: Joint Low-Light Enhancement and White Balance under Colored Illumination
Findings-aware Language-Image Pretraining for CT Volume Interpretation
Is This 3D Asset Usable? Beyond the Beauty Render for Asset Curation
AgiWorld: An Agentic Video World Model for Building Persistent Worlds
Get the few papers worth your time in Computer Vision every morning.
ReadSearch reads the new papers in your field each day and brings you the few worth reading, each with its key finding in a sentence.