arXiv preprint ·
The (Butter)fly Dream: Linear Separability of World-Model Rollouts in a Connectome-Constrained Visual Model
- Published
- Source
- arXiv preprint, posted before peer review
- Original
- Read it at arXiv
- Topics
Abstract
Zhuangzi dreamt he was a butterfly and, on waking, could not tell which was the dream. We ask a version of his question of a model of the fly brain: placed inside a world generated by a world model, can a fixed brain tell that the world is a dream? We treat it as a question about what a connectome-constrained model trained on one task makes available beyond that task: do its responses make the defects of a dream linearly decodable? The observer is a published model of the fly visual system trained only for optic-flow estimation; reality is the MPI-Sintel movie on the model's 721-column retina; dreams are continuations of five real frames by three small world models (a persistence random walk, an MDN-RNN and an RSSM) at three temperatures. The judge is a classifier two-sample test on temporal statistics of cell-type responses, with sequence-level cross-validation. In the trained observer's responses all nine dreams are separable from reality (AUC 0.85-0.99, mean over 50 trained models), in those of the same network before training much less so (0.58-0.75 for the learned world models; paired difference 0.20-0.37), equally on sequences the observer never saw, and in an order unrelated to pixel error. Because these AUCs are near ceiling, we add graded probes made of each window's own real frames with only their order broken. Swapping one adjacent pair of the 14 generated frames gives 0.73 on the trained responses against 0.54 before training and 0.56 for pixel statistics; the trained curve reaches 0.75 at 1.1 swapped pairs of seven, the pixel statistics at 4.7, and the untrained network not at all. A detection horizon, the test on the first k generated frames, orders the world models where the full-length AUC saturates.
The authors’ abstract, as arXiv lists it. arXiv’s listings, abstracts included, are free for anyone to reuse.
Read it on ReadSearch
Each opens after you create an account, and brings you back to this paper.
More in Machine Learning
Prior Action Control Causally Alters Language-Model Plasticity
Do agents need to sleep?
AVID: A 5T fMRI Dataset for Benchmarking Auditory-induced Visual Mental Imagery Decoding
Modeling Visual Cortex with Layerwise Multiscale Manifold Capacity Maximization
When Model Complexity Matters in Cross-Subject Naturalistic fMRI Prediction
Get the few papers worth your time in Machine Learning every morning.
ReadSearch reads the new papers in your field each day and brings you the few worth reading, each with its key finding in a sentence.