arXiv preprint ·
Adaptive Agent Design
- Published
- Source
- arXiv preprint, posted before peer review
- Original
- Read it at arXiv
- Topics
Abstract
We consider an agent acting against a general non-Markovian environment. The agent maintains its agent states, but is free to choose a transition kernel across those states and optimize its state-feedback control policies. We study the bi-level agent design problem that optimizes the transition kernel and the policy it induces, given said kernel with offline data of observations and actions obtained via a behavioral policy. For general environments, we show that a soft $Q$-learning algorithm converges almost surely to the fixed point of a soft Bellman equation defined by the stationary averages that the behavioral policy and the chosen kernel induce, and we delineate what separates the resulting policy from an optimal one. In partially observed Markov decision problems, we analyze convergence properties of parametrized transition kernel design via zero-th order and Bayesian optimization techniques.
The authors’ abstract, as arXiv lists it. arXiv’s listings, abstracts included, are free for anyone to reuse.
Read it on ReadSearch
Each opens after you create an account, and brings you back to this paper.
More in Systems & Control
Stabilizing Performative Feedback Loops with Minimal Model Deployments
Distortion of AI Alignment Revisited: RLHF is a Decent Utilitarian Aligner
Approximating Optimal Welfare in Complementary Allocation under Decentralized Information
Truncated Noisy Best-Response Algorithms: Toward Game Theoretic Learning with Safety Guarantees
Degree-Parameterized Analysis of Sampling-Based Online Matching
Get the few papers worth your time in Systems & Control every morning.
ReadSearch reads the new papers in your field each day and brings you the few worth reading, each with its key finding in a sentence.