OpenAlex ·
LEAP: Likelihood Elicitation and Aggregation for LLM-based Probabilistic Forecasting
- Published
- Source
- OpenAlex
- DOI
- 10.48550/arxiv.2609.01337
- Original
- Read it at the publisher
- Topics
Abstract
LLM-based forecasting systems have improved on real-world tasks such as financial markets and sports outcomes, largely through stronger search and tool use. Many systems still ask an LLM to read all collected evidence together and produce the final forecast. We call this design Monolithic Prediction. It can obscure how individual evidence items affect the result and collapse uncertainty across competing outcomes. We propose LEAP (Likelihood Elicitation and Aggregation for Probabilistic forecasting), which reorganizes how collected evidence is used in the prediction stage. LEAP examines each evidence item separately and elicits likelihood parameters that describe its implications for the target. An explicit prior and a deterministic probabilistic model then combine these likelihoods into a posterior distribution. This procedure supports continuous, single-choice, and multi-choice forecasts while preserving reproducible evidence contributions. We build a benchmark covering forecasting, information-seeking, and browsing tasks, and evaluate LEAP on our own agent loop and several agent CLI frameworks. Given the same evidence, LEAP improves most prediction and calibration metrics across models and remains stronger under controlled comparisons of prior access, inference budget, and aggregation.
The authors’ abstract, as arXiv lists it. arXiv’s listings, abstracts included, are free for anyone to reuse.
Read it on ReadSearch
Each opens after you create an account, and brings you back to this paper.
More in Artificial Intelligence
Drift-Aware LLM Routing with Sparse Contexts and Shared Budgets
Recommender System as Slow and Fast Thinkers
ClaimReceipt: Verifying Evidence Sufficiency and Coverage in Agent Evaluations
Don't You Know, Pump it Up! Investigating Cryptocurrency Manipulation in Telegram-Driven Activity
FOXDEN: FAIR Services for AI-Ready Scientific Datasets
Get the few papers worth your time in Artificial Intelligence every morning.
ReadSearch reads the new papers in your field each day and brings you the few worth reading, each with its key finding in a sentence.