OpenAlex ·
In-Context Neurofeedback: Can LLMs Control Their Internal Representations through Privileged Access?
- Published
- Source
- OpenAlex
- DOI
- 10.48550/arxiv.2609.00904
- Original
- Read it at the publisher
- Topics
Abstract
Whether large language models (LLMs) can control their own internal representations matters for both machine metacognition and AI safety. A recent study applied neurofeedback to LLMs and claimed that they can control their internal representations. However, the reported control may rely on superficial mechanisms rather than genuine internal access because the control targets in that study are not privileged, meaning that a third party can infer them from the prompt. We redesign the neurofeedback paradigm for LLMs so that the control target satisfies the privileged access requirement, which is closer to neurofeedback experiments in human cognitive neuroscience. Under this stricter setting, the models do not demonstrate reliable control over privileged internal representations, suggesting that previously reported control cannot exclude the possibility that it relies on superficial mechanisms. Our results indicate that rigorous assessments of metacognition in LLMs require evaluation methods that demand privileged access.
The authors’ abstract, as arXiv lists it. arXiv’s listings, abstracts included, are free for anyone to reuse.
Read it on ReadSearch
Each opens after you create an account, and brings you back to this paper.
More in Artificial Intelligence
Cross-Modal Guidance for Out-of-View Object Search in Simulated Prosthetic Vision
A Roadmap for MEG Foundation Models
A distributed-delay Wilson-Cowan model of sleep-related rhythms in the corticothalamic system
Formation of structural attractors in neuromorphic systems
An Intelligent Decision Support System for Emotion Monitoring using Microscopic Fixational Dynamics
Get the few papers worth your time in Artificial Intelligence every morning.
ReadSearch reads the new papers in your field each day and brings you the few worth reading, each with its key finding in a sentence.