arXiv preprint ·
A Token Is Not a Unit of Difficulty: Segmentation Controls Where Adaptive Computation Goes Inside a Word
- Published
- Source
- arXiv preprint, posted before peer review
- Original
- Read it at arXiv
- Topics
Abstract
Early-exit language models decide for each token how much computation to spend, usually by stopping once the prediction is confident. Yet a token is not a fixed unit: tokenizers cut words into different numbers of pieces in different languages. We ask where inside a word such a policy spends its budget. In Qwen3, along two adaptive axes (the timesteps of a spiking conversion and the layers of the original Transformer), confidence stopping gives the continuation pieces of a word less computation than its first piece in English, German, and Turkish, but more in Bangla. Segmentation drives this. Re-segmenting the same parallel passages with BPE-dropout shifts the gap by up to eight exits on both axes, small models trained under two tokenizers shift Bangla's gap by six exits, and BLOOM and mGPT, which split Bangla less finely, give it the English pattern. Characters per token do not predict the gap across languages; within-word surprisal explains 75% of its variance. A learned stopping head beats confidence stopping by 7 to 8 points, but gains less than one point over the best mix of fixed exits at equal cost, and sometimes loses. All results come from offline analysis, not measured speedups.
The authors’ abstract, as arXiv lists it. arXiv’s listings, abstracts included, are free for anyone to reuse.
Read it on ReadSearch
Each opens after you create an account, and brings you back to this paper.
More in Computation & Language / NLP
Spotting the Artificial: Dialogic Humor Improvisation in Humans and LLMs
XDecideBench: Evaluating Typed Decision Models and Confidence Policies Across Languages
Swap, Replicate, Read the Mass: A Protocol for Auditing Demographic Bias in System One Decision Models
Information-Theoretic Design of Sparse Autoencoder-Based Steering
Robustness Starts with the Objective: Designing and Certifying Robust LLM-Generated Heuristics
Get the few papers worth your time in Computation & Language / NLP every morning.
ReadSearch reads the new papers in your field each day and brings you the few worth reading, each with its key finding in a sentence.