arXiv preprint ·
Understanding the Inner Workings of a State-Space Foundation Model for Limit Order Books
- Published
- Source
- arXiv preprint, posted before peer review
- DOI
- 10.1145/3858063.3858862
- Original
- Read it at arXiv
- Topics
Abstract
Autoregressive foundation models offer state-of-the-art capabilities to simulate the market by generating the message flow of limit order books (LOBs). Yet, without understanding their inner workings, research directions are hard to narrow and practical use in real-world trading environments remains uncertain. This study therefore asks what these models actually learn about market microstructure. We find that, similarly to games where sequence models build an internal representation of the board, an 8.2M-parameter Mamba-3 state-space model trained autoregressively on the tokenized message stream of eight large-cap US equities learns an internal representation of the market and its dynamics. In particular, order-flow imbalance, a central driver of short-horizon price moves, is decodable at $R^2 \approx 0.59$, far above controls and baselines. This representation extends to unseen stocks, suggesting that these models learn general representations rather than ticker-specific patterns. The model also generalizes across metrics beyond order-flow imbalance: results hold across traded volume and volatility. Additionally, the representation is causally steerable: pushing the state along a feature's axis shifts the model's next-message beliefs as that feature implies, monotonically in dose, consistent with the emergence of a world model.
The authors’ abstract, as arXiv lists it. arXiv’s listings, abstracts included, are free for anyone to reuse.
Read it on ReadSearch
Each opens after you create an account, and brings you back to this paper.
More in Trading & Market Microstructure
Speculative Imbalance Positions by Non-Physical Traders: Impacts on Electricity System Balancing in Great Britain
Dynamics of the Liquidity Profile in Concentrated Liquidity Provision: An SPIDE Approach
Equilibria and Learning in Competitive Market Making
Topological Compression of Prediction Market Belief States for Event Forecasting
Intraday Forecasting Ability and Liquidity Provision of Institutional Traders
Get the few papers worth your time in Trading & Market Microstructure every morning.
ReadSearch reads the new papers in your field each day and brings you the few worth reading, each with its key finding in a sentence.