arXiv preprint ·
A Distributional Framework for Behavioral Characterization and Risk Management of Large Language Models
- Published
- Source
- arXiv preprint, posted before peer review
- Original
- Read it at arXiv
- Topics
Abstract
Large language models are stochastic by nature. Repeated executions under the same experimental conditions may produce different responses. We propose a framework that can be applied across the model lifecycle, from task design to residual behavior and continuous monitoring. The method consists of building an empirical response distribution estimated through Monte Carlo simulation. Semantic interventions are introduced to study how these distributions change. We illustrate the framework with two different tasks, borrower classification and music album recommendation, and quantify the resulting distributional changes. The framework is linked to model risk management and governance by measuring robustness and control effectiveness, and characterizing residual behavior. Since it relies only on observed responses, it can also be applied to black-box models accessed through commercial APIs.
The authors’ abstract, as arXiv lists it. arXiv’s listings, abstracts included, are free for anyone to reuse.
Read it on ReadSearch
Each opens after you create an account, and brings you back to this paper.
More in Risk Management
The Financing of Debt Relief: Intermediary Liquidity and the Value of Borrower Information
Conditional value-at-risk under reward-penalty mechanism with applications to robust portfolio management
Enhancing credit risk scorecards with transactional-based behavioural features
A Functional Representation of Credit Behavior for Probability of Default Modeling
Realized Tail-Risk Conditional Autoregressive Expectile Models: Accurate and Stable Forecasting Performance
Get the few papers worth your time in Risk Management every morning.
ReadSearch reads the new papers in your field each day and brings you the few worth reading, each with its key finding in a sentence.