M Uday Kumar, Ph.D. student of IEOR, presented his work on Optimal Markov Policies for Finite-Horizon Constrained MDPs With Combined Additive and Multiplicative Utilities at IEEE conference on Decision and Control (CDC) 2023 held at Marina Bay Sands, Singapore from 13-15 Dec 2023.
Abstract: We consider the problem of optimizing a finite-horizon constrained Markov decision process (CMDP) where the objective and constraints are sums of additive and multiplicative utilities. To solve this, we construct another CMDP with only additive utilities whose optimal value over a restricted set of policies is equal to that of the original CMDP. Further, we provide a finite-dimensional bilinear program (BLP) whose value equals the CMDP value and whose solution provides the optimal policy. We also suggest an algorithm to solve the proposed BL.
