Best Arm Identification for Bandits with Shifting Means

Abstract

We study the best arm identification problem in a stochastic environment with a novel form of adversarial perturbations, which we coin Shifting Means. While classically the mean rewards of the $K$ arms are stable in time, in Shifting Means only the gaps $\boldsymbolΔ$ between mean rewards are stable, while their common shift may be determined adversarially in each round. The objective of the learner is to identify the best arm with high probability while minimizing sample complexity (the fixed confidence setting). Handling shifts requires new tools: we show that algorithms employing a Generalized Likelihood Ratio Test (GLRT) stopping rule, including the popular Track-and-Stop, fail under time-varying shifts. Instead, we propose Importance Weights for Shifting Means ($\mathsf{ISM}$). Assuming means bounded by $U$ and $σ^2$-sub-Gaussian rewards, we show $\mathsf{ISM}$ to be $δ$-correct and to enjoy a sample complexity bound of order $K (σ^2 + U^2) Δ_{\min}^{-2} \ln \frac{1}δ$. We also present a matching (up to constant factors) worst-case lower bound and evaluate our results empirically.

The authors’ abstract, as arXiv lists it. arXiv’s listings, abstracts included, are free for anyone to reuse.

More in Machine Learning (Statistics)

• arXiv preprint · 7 October 2026

  Efficient Best-of-N policy evaluation for inference-time alignment

  Jonas Schweisthal, Yuxin Wang, Athiya Deviyani and 2 others
• arXiv preprint · 8 October 2026

  Decision-Sufficient Posterior Approximation

  Sean Plummer
• arXiv preprint · 7 October 2026

  The Symbol of the Surrogate: Measuring Numerical Provenance in Neural PDE Solvers

  Ridham Patel
• arXiv preprint · 8 October 2026

  A General $\widetildeΩ(\sqrt{T γ_T})$ Lower Bound for Kernel Bandits

  Chenkai Ma and Jonathan Scarlett
• arXiv preprint · 7 October 2026

  The Geometry of Anisotropic Dilation for Optimal Regularization

  Carson Newman and Oscar Leong

All new papers in Machine Learning (Statistics)