Calibration as a First-Class Criterion in LLM Evaluation

Abstract

Calibration of language models -- the alignment between expressed or implicit confidence and empirical correctness -- is a well-studied subfield within NLP. Methods to measure it already exist. The problem is adoption: outside this subfield, NLP research regularly introduces new models, datasets, and benchmarks without checking whether the model's confidence scores are meaningful. We argue that this adoption gap is a major obstacle to trustworthy LLM evaluation. Miscalibration causes problems in two distinct areas: at deployment, where overconfident mistakes cause real harm, and inside the research pipeline, where methods like LLM-as-a-judge, synthetic data generation, and active learning rely on calibrated confidence without verifying it. Standard calibration metrics only require two inputs per example: a confidence score and a correctness judgment. Most benchmarks in use today already provide both, meaning calibration can be reported immediately. For open-ended generation, however, defining these two inputs is still an open challenge. We argue that each NLP subfield should pair its main performance metric with a calibration score and call for treating calibration as an essential property of every model rather than a niche topic.

The authors’ abstract, as arXiv lists it. arXiv’s listings, abstracts included, are free for anyone to reuse.

More in Computation & Language / NLP

• arXiv preprint · 22 September 2026

  Matryoshka attribution: Learning to attribute language model outputs to representations and weights

  Aryaman Arora, Kirill Acharya, Nathan Hu and 4 others
• arXiv preprint · 23 September 2026

  Realize What Matters: Principled Context Representation for Large-Scale Reasoning

  Michael Theologitis, Dean Light, Shuyue Stella Li and 3 others
• arXiv preprint · 22 September 2026

  Compressing Long Context into Answer-Aligned Memory Embeddings for LLM Inference

  Md Mostafizer Rahman, Md Faizul Ibne Amin, Md Shahajada Mia and 2 others
• arXiv preprint · 23 September 2026

  Beyond Overlap: Estimating the Causal Effect of Benchmark Exposure

  Divyansh Singh
• arXiv preprint · 22 September 2026

  Rewired or Gated? How Instruction Tuning Shapes Knowledge-Conflict Circuits in LLMs

  Shubham Santosh Pandere, Gautam Ranka, Ritika Varshney and 3 others

All new papers in Computation & Language / NLP