arXiv preprint ·
The Impact of Stochasticity on the Rashomon Effect in Machine Learning
- Published
- Source
- arXiv preprint, posted before peer review
- Published in
- SSRN Electronic Journal
- DOI
- 10.2139/ssrn.7306682
- Original
- Read it at arXiv
- Cited by
- 0 works, as OpenAlex counts them
- Topics
In brief
Unpacking the Rashomon Effect in AI
This paper dives into the Rashomon phenomenon by examining three key dimensions of model variability. It sheds light on how different stochastic factors like weight initialization and data ordering uniquely influence model performance and explanations.
What they found: The study reveals that different sources of stochasticity lead to varying sizes of the empirical Rashomon set, predictive disagreement, and attribution agreement among models. For instance, weight initialization resulted in a larger set of distinct models compared to mini-batch ordering. Importantly, the findings suggest that these dimensions of the Rashomon effect are complementary, not interchangeable, which has implications for model reliability and interpretability.
AI-written summary. Check anything you rely on against the original paper.
Abstract
Neural network training is inherently stochastic, with factors such as weight initialization leading to distinct models despite comparable predictive performance. This phenomenon is commonly associated with the Rashomon effect, which describes the existence of multiple near-optimal models for the same task. Although the Rashomon effect has received increasing attention, it remains unclear whether different sources of training stochasticity contribute similarly or differently to its manifestations. In this work, we present an empirical study of the Rashomon phenomenon along three complementary dimensions: solution-space multiplicity, predictive multiplicity, and decision-basis multiplicity. These dimensions are quantified through the size of the empirical Rashomon set, predictive ambiguity, and agreement between XAI attribution maps, respectively. By independently controlling three standard sources of stochasticity, namely weight initialization, mini-batch data ordering, and dropout, we isolate their respective contributions to each dimension of the Rashomon phenomenon. Experiments on tabular and image classification benchmarks reveal that these sources affect the three dimensions in different ways. In particular, larger empirical Rashomon sets do not necessarily correspond to greater predictive disagreement or lower explanation agreement, indicating that solution-space, predictive, and decision-basis multiplicity capture complementary rather than interchangeable aspects of the Rashomon effect. Overall, our results show that training stochasticity influences not only predictive performance but also the stability of predictions and explanations, highlighting the importance of identifying the specific sources of stochasticity responsible for different manifestations of the Rashomon phenomenon when assessing the reliability, reproducibility, and interpretability of neural network models.
The authors’ abstract, as arXiv lists it. arXiv’s listings, abstracts included, are free for anyone to reuse.
Read it on ReadSearch
Each opens after you create an account, and brings you back to this paper.
More in Machine Learning
Emergent One-Third Scaling Law as Attention Tries to Concentrate
Byzantine-Robust Federated RAG via Aligned Calibration and Fixed-Membership Conformal Prediction
LLM Unlearning Evaluation with TRIAGE
Improving the Diversity of LLM Outputs without a Trade-off
An Attention-Driven Heterogeneous GNN Model for Credit Card Fraud Detection
Get the few papers worth your time in Machine Learning every morning.
ReadSearch reads the new papers in your field each day and brings you the few worth reading, each with its key finding in a sentence.