We found a match
Your institution may have access to this item. Find your institution then sign in to continue.
- Title
Strong Simple Policies for POMDPs.
- Authors
Winterer, Leonore; Wimmer, Ralf; Becker, Bernd; Jansen, Nils
- Abstract
The synthesis problem for partially observable Markov decision processes (POMDPs) is to compute a policy that provably adheres to one or more specifications. Yet, the general problem is undecidable, and policies require full (and thus potentially unbounded) traces of execution history. To provide good approximations of such policies, POMDP agents often employ randomization over action choices. We consider the problem of computing simpler policies for POMDPs, and provide several approaches to still ensure their expressiveness. Key aspects are (1) the combination of an arbitrary number of specifications the policies need to adhere to, (2) a restricted form of randomization, and (3) a light-weight preprocessing of the POMDP model to encode memory. We provide a novel encoding as a mixed-integer linear program as baseline to solve the underlying problems. Our experiments demonstrate that the policies we obtain are more robust, smaller, and easier to implement for an engineer than those obtained from state-of-the-art POMDP solvers.
- Subjects
PARTIALLY observable Markov decision processes
- Publication
International Journal on Software Tools for Technology Transfer, 2024, Vol 26, Issue 3, p269
- ISSN
1433-2779
- Publication type
Article
- DOI
10.1007/s10009-024-00747-0