A Minimum Relative Entropy Controller for Undiscounted Markov Decision Processes - Computer Science > Artificial IntelligenceReport as inadecuate




A Minimum Relative Entropy Controller for Undiscounted Markov Decision Processes - Computer Science > Artificial Intelligence - Download this document for free, or read online. Document in PDF available to download.

Abstract: Adaptive control problems are notoriously difficult to solve even in thepresence of plant-specific controllers. One way to by-pass the intractablecomputation of the optimal policy is to restate the adaptive control as theminimization of the relative entropy of a controller that ignores the trueplant dynamics from an informed controller. The solution is given by theBayesian control rule-a set of equations characterizing a stochastic adaptivecontroller for the class of possible plant dynamics. Here, the Bayesian controlrule is applied to derive BCR-MDP, a controller to solve undiscounted Markovdecision processes with finite state and action spaces and unknown dynamics. Inparticular, we derive a non-parametric conjugate prior distribution over thepolicy space that encapsulates the agent-s whole relevant history and wepresent a Gibbs sampler to draw random policies from this distribution.Preliminary results show that BCR-MDP successfully avoids sub-optimal limitcycles due to its built-in mechanism to balance exploration versusexploitation.



Author: Pedro A. Ortega, Daniel A. Braun

Source: https://arxiv.org/







Related documents