arXiv preprint ·
When Disagreement Labels Do Not Identify F1 Rankings
- Published
- Source
- arXiv preprint, posted before peer review
- Original
- Read it at arXiv
- Topics
Abstract
Agreement labels cancel from an accuracy comparison but can change an F1 ranking. We specialize partial identification to an explicit audit after labeling disagreements: two agreement counts give exact bounds, certifying a ranking or a choice within a specified F1 tolerance under every completion. Across 3,200 frozen model pairs, shared completions tighten bounds and a 0.01 tolerance reduces labeling costs. A zero-loss (weak-winner) control attributes most savings to positive tolerance, not ties. Guarantees concern the evaluated batch, not future data.
The authors’ abstract, as arXiv lists it. arXiv’s listings, abstracts included, are free for anyone to reuse.
Read it on ReadSearch
Each opens after you create an account, and brings you back to this paper.
More in Machine Learning
State-Augmented Learning for Post-Deployment Adaptation of Coexisting O-RAN Applications
CytoWave: Perturbation-Centric Pretraining for Single-Cell Response Prediction
Information-Theoretic Design of Sparse Autoencoder-Based Steering
Improving Financial Cybersecurity Through Fraud Transaction Detection with Deep Learning
Toward Executable Multi-framework Front-end Code Generation with Self-Correction
Get the few papers worth your time in Machine Learning every morning.
ReadSearch reads the new papers in your field each day and brings you the few worth reading, each with its key finding in a sentence.