Training Negatives Still Matter After Calibrating Answerability Readouts

Abstract

Answerability readouts predict whether a passage supplies an answer, but whether calibration can compensate for removing negative passages during training remains unclear. We present a controlled study of how training-negative construction affects calibrated prediction, ranking, and downstream answer retention. We compare native training, which retains negative passages, with empty training, which removes them, holding questions, labels, positive examples, and model weights fixed. Both are calibrated on the same labeled native examples. On SQuAD 2.0 and RepLiQA, native-trained logistic readouts at selected depths achieve lower Brier loss for Qwen2.5-7B-Instruct and Gemma-2-9B-it, with or without output scores. Native fusion also outperforms calibrated output alone. Post-hoc AUROC analysis shows better ranking under native training, beyond what threshold adjustment alone can explain. On the official SQuAD 2.0 development set, native readout policies improve F1 over empty policies but retain more answers to unanswerable questions. The advantage varies by estimator and depth, while RepLiQA's text shortcuts limit claims about contextual understanding. Unfinished generations prevent complete answer-level evaluation, although worst-case bounds preserve the native-empty F1 gains. Our findings show that training negatives affect ranking and downstream decisions even after common calibration, without establishing a universal advantage for native training.

The authors’ abstract, as arXiv lists it. arXiv’s listings, abstracts included, are free for anyone to reuse.

More in Computation & Language / NLP

• arXiv preprint · 19 October 2026

  Cognitive Maps in Large Language Models

  Albert Qin, Jonathan Cohen and Gautam Reddy
• arXiv preprint · 20 October 2026

  The Router Already Knows

  Andrew Bistras and Marty Johnson
• arXiv preprint · 19 October 2026

  Text Dominance in Paralinguistic Conflict: Tracing Suppression Through Audio LLM Circuits

  Ankur Banerjee, Vishnu Guntakala, Danity Pike and 3 others
• arXiv preprint · 20 October 2026

  Harness Self-Evolution for Long-Horizon Terminal Agents

  Ruhan Wang, Yucheng Shi, Zongxia Li and 6 others
• arXiv preprint · 19 October 2026

  Curriculum Learning Teaches LLMs Long-Horizon Implicit Reasoning, Up To A Limit

  Jimson Huang, Maxwell Nickell, Ruowang Zhang and 1 other

All new papers in Computation & Language / NLP