arXiv preprint ·
When Do External Priors Help Low-Resource Structured Prediction? A Ranking-Level Alignment Study
- Published
- Source
- arXiv preprint, posted before peer review
- Original
- Read it at arXiv
- Topics
Abstract
Cold-start credit scoring (scarce labels, immature features, or minimal capacity) motivates injecting external priors (teacher models, LLMs, or domain experts) into training. We study Ranking Prior Alignment (RPA), a model-agnostic objective that aligns a model's ranking distribution to an external prior via a temperature-scaled KL loss with an annealed weight, $\mathcal{L} = \mathcal{L}_{\text{task}} + γ(t)\,\text{KL}(P_{\text{agent}} \| P_{\text{model}})$, and requires no external model at inference. Across tree-ensemble and multiple-instance-learning (MIL) instantiations on the public Amex dataset and an industrial merchant-credit dataset (1.5M+ merchants), we ask when such priors genuinely transfer information rather than merely act as a regularizer. Our central finding: the prior's quality $π$ (its rank correlation with the label) governs transfer only when the alignment granularity matches the granularity at which $π$ is measured. On cross-sample tree alignment (Amex), a high-$π$ teacher yields $+0.028$ AUC over the no-alignment baseline (paired bootstrap 95% CI $[0.022, 0.035]$) whereas zero-information priors (uniform, random) yield none, and $Δ$AUC tracks $π$ (XGBoost Pearson $r = 0.985$); this replicates on a second public dataset (LendingClub). On within-bag MIL-attention alignment, the same high-$π$ teacher transfers nothing: it is significantly worse than its own within-bag shuffle (net $-0.017$, 95% CI $< 0$, sign $p = 0.021$ over 10 seeds), while zero-information priors give the only significant gains: the KL term collapses to pure entropy regularization. We distill these results into a two-factor $N \cdot \mathcal{C}$ decision rule (gains concentrate at low sample size and capacity and vanish as either grows) and a scale-controlled paired protocol that separates information transfer from regularization.
The authors’ abstract, as arXiv lists it. arXiv’s listings, abstracts included, are free for anyone to reuse.
Read it on ReadSearch
Each opens after you create an account, and brings you back to this paper.
More in Machine Learning
Mid-Training Language Models on Raw Video
Inverse Synthesis of Shock Time Series from Shock Response Spectra Using Ensembles of Conditional Variational Autoencoders
A Graph Neural Network for Global Daily Fire Radiative Power Prediction at Medium-Range Lead Times
TreeHCA: Hindsight-Guided Exploration and Credit Assignment for Multi-Turn LLM Agents
NOMOS: Compiling Written Policies into Statically Verified Tool-Call Gates for LLM Agents
Get the few papers worth your time in Machine Learning every morning.
ReadSearch reads the new papers in your field each day and brings you the few worth reading, each with its key finding in a sentence.