arXiv preprint ·
What Remains Without Test Labels? Re-evaluating a Graph-Level Anomaly Detection Benchmark
- Published
- Source
- arXiv preprint, posted before peer review
- Original
- Read it at arXiv
- Topics
Abstract
Graph-level anomaly detectors are trained only on normal graphs, yet benchmarks may use test labels for model selection. We examine whether UB-GOLD results reused by later papers support label-free detection. We audit UB-GOLD and 14 GLAD code bases, re-run 21 detectors on UB-GOLD's folds under selection rules with and without test labels, and evaluate sensitivity to known structural changes. Test labels select checkpoints in UB-GOLD's shared training loop and checkpoints or hyperparameters in 8 of the 14 code bases. UB-GOLD also evaluates its kernel baselines against misaligned labels. Across nine datasets, selecting checkpoints by test AUROC adds an average of 5.12 AUROC points [3.97, 6.24] for the 12 UB-GOLD detectors relative to the final checkpoint and 4.13 [3.32, 4.79] relative to held-out normal graphs. In a post hoc permutation analysis with no truly better checkpoint, picking the best of noisy test evaluations still yields a gain more than half as large as the one observed against the final checkpoint. Configurations tuned on test labels score a further 1.53 points [0.64, 2.42] above repository defaults, a gap that mixes the value of test labels with that of tuning itself. With tuned hyperparameters and label-free checkpoint selection, the two strongest detectors retain their lead, while comparisons below them are less stable. Six of eight detectors evaluated out of sample miss size-preserving composition changes that graphlet counts detect. We recommend reporting results selected without test labels as the main comparisons and release per-run outputs that allow verification without retraining.
The authors’ abstract, as arXiv lists it. arXiv’s listings, abstracts included, are free for anyone to reuse.
Read it on ReadSearch
Each opens after you create an account, and brings you back to this paper.
More in Machine Learning
Validating LLM-generated political networks with survey-reported partisan exposure
YouTube Clips and Broad but Shallow Attention to Candidates: Evidence from the 2024 Tokyo Gubernatorial Election
"You" Is Not Toxic: How LLM Labelers Rewrite the Web's Conversation Laws
Scam Town: Can Scam Education Spread Like a Vaccine?
Graphlet-based Decentralized Finance Transaction Method Classification
Get the few papers worth your time in Machine Learning every morning.
ReadSearch reads the new papers in your field each day and brings you the few worth reading, each with its key finding in a sentence.