OpenAlex ·
PALM: Point-in-Time Adaptation for Financial Language Models
- Published
- Source
- OpenAlex
- Published in
- arXiv (Cornell University)
- DOI
- 10.48550/arxiv.2609.30316
- Original
- Read it at the publisher
- Topics
Abstract
Language models used in financial backtests suffer from look-ahead bias, as a model trained on text published after the study period has already observed the outcomes it is asked to predict. To handle this issue, point-in-time (PIT) language models are pretrained on chronologically filtered corpora and released as one checkpoint per calendar year, each with a documented cutoff. However, each additional year costs another pretraining run, and whether that run is necessary has never been tested. In this paper, we show that a newer checkpoint is not detectably better on the decision it informs, as its information coefficient on a fixed evaluation window shows no decay with age that we can detect. Motivated by this observation, we propose PALM (Point-in-time Adaptation for financial Language Models), a simple yet effective alternative to annual pretraining that fits a low-rank adapter on financial text the checkpoint is allowed to read, without modifying any pretrained weight. We further find that the gain comes from the financial domain rather than from recency, which leaves the point-in-time constraint without a detectable cost. We validate PALM on a decade of financial news and on three families of PIT models, whose cutoffs span two decades. The pooled gain keeps its sign across experimental settings and comes at a small fraction of the cost of pretraining. Code is available at: https://github.com/seunghan96/palm.
The authors’ abstract, as arXiv lists it. arXiv’s listings, abstracts included, are free for anyone to reuse.
Read it on ReadSearch
Each opens after you create an account, and brings you back to this paper.
More in Artificial Intelligence
Instantaneous shrinking of supports for stochastic PDEs
Correlated initialization of deep residual networks
Directional Optimal Sub-Gamma Scales for Infinitely Divisible Laws
A one-parameter family of local-time conditioned processes : the generalized Brownian burglars
Eliciting ESG Preferences for Reinforcement Learning-Based Portfolio Optimization
Get the few papers worth your time in Artificial Intelligence every morning.
ReadSearch reads the new papers in your field each day and brings you the few worth reading, each with its key finding in a sentence.