arXiv preprint ·
Constrained Tagging for Span Labeling
- Published
- Source
- arXiv preprint, posted before peer review
- Original
- Read it at arXiv
- Topics
Abstract
Generative LLMs can label spans without task-specific training, but their output is not tied to positions in the input: a returned span may be absent from the input or occur several times. We present constrained tagging, a training-free method in which the model reproduces the input and wraps each span in labeled inline markup. A logits processor using a byte-level token trie and a finite-state machine admits only tokens that continue the input verbatim or form a valid tag, so the output reproduces the input exactly with labels from the given set, and spans may begin or end at any character boundary, even inside a token. Two-phase decoding lets reasoning models reason before the constraint engages. On six datasets and six open-weight models, the guarantee holds in all but 0.022% of 255,312 constrained generations, each failure a reasoning model exhausting its token budget or never closing its reasoning, and the constraint adds no measurable cost. Unlike context-based prompting, which loses 8.5% of its spans, constrained tagging locates every span it proposes on every dataset; it is as accurate on two datasets, and on the other four its errors lie in which spans the model chooses, not where they are.
The authors’ abstract, as arXiv lists it. arXiv’s listings, abstracts included, are free for anyone to reuse.
Read it on ReadSearch
Each opens after you create an account, and brings you back to this paper.
More in Information Retrieval
Baseline and Incremental: User, Item, and Request Debiasing in Instagram Feed Ranking
Our Own Jev: Typed Decisions in One Forward Pass, from an In-House Judge to Open LoRA Models
Factual Access States in Diffusion Language Models: From Mechanism to Knowledge Editing and Retrieval
Progressive Learning from Content to Relevance to Personalization for CTR Prediction
GARRET: Generative Feedback for Region Retrieval in Visual Documents
Get the few papers worth your time in Information Retrieval every morning.
ReadSearch reads the new papers in your field each day and brings you the few worth reading, each with its key finding in a sentence.