arXiv preprint ·
CoTs remain monitorable when being unfaithful requires computation: Replicating and Extending the Necessity Effect Across Eleven Models
- Published
- Source
- arXiv preprint, posted before peer review
- Original
- Read it at arXiv
- Topics
Abstract
While a LLM's chain of thought (CoT) is a promising approach to interpreting model outputs, prior work shows CoT's content does not always accurately reflect the reasoning process. Emmons et al. (2025) finds that models can follow a question hint without revealing so in its chain of thought, but when the hint is sufficiently complex the computation is forced into the CoT. This claim, however, has only been tested on Gemini models, and the datasets and code is closed source. We replicate their proxy task across eleven models from six families and find the necessity effect generalizes. Every model follows simple cues above a no-hint baseline, while concealed complex-cue following falls back near baseline for most models. We also extend the study from Emmons et al. (2025) to measure CoT-necessity and unfaithfulness across five cue families (four of them new) and three difficulty levels. The relationship between when CoT is necessary and unfaithfulness is task-specific and model-specific. For some hint types, models often verbalize their hint-following behavior even when the hint can be decoded in a single forward pass. These results support the case for CoT monitoring when unsafe behavior requires non-trivial computation, and that properties of CoT unfaithfulness are sensitive to model and setting.
The authors’ abstract, as arXiv lists it. arXiv’s listings, abstracts included, are free for anyone to reuse.
Read it on ReadSearch
Each opens after you create an account, and brings you back to this paper.
More in Machine Learning
Power-Law Correlations Without Apparent Temperature-Tuned Criticality in Fixed-Tree Hierarchical Languages
DF-PALM: Dynamic Fusion Prompt Learning for Audio--Language Models
Uniform convergence at optimal rate and scale
Means-Ends Analysis Helps LLMs Plan in Continuous-State Domains
What Matters for NVFP4 Training? A Comparative Study of Low-Precision Pretraining Recipes
Get the few papers worth your time in Machine Learning every morning.
ReadSearch reads the new papers in your field each day and brings you the few worth reading, each with its key finding in a sentence.