Title: Robustness to Incorrect Models and Discrete-Time Approximations for Controlled Diffusions under General Information Structures.
Speaker: Somnath Pradhan (Coleman Postdoctoral Fellow Department of Mathematics and Statistics at Queen’s University, Canada )
Day, Date, and Time: Wednesday, 22nd November, 2023, 09:30 - 10:30 AM (Click here to join )
Abstract: In stochastic control applications, typically only an ideal model is assumed, or learned from available incomplete data, based on which an optimal control is designed and then applied to the actual system. This leads to the problem of robustness to model mismatch. An additional related problem is on discrete time/space approximations of optimal control problems involving continuous space and time, which can also be interpreted as a structured form of robustness.
In this talk, we will address these closely related problems for controlled diffusion models.
We first establish the robustness of optimal policies (under several cost criteria) with respect to functional perturbations of the system. Our approach builds on the regularity properties of optimality equations via a PDE theoretic analysis of HJB equations leading to a unified approach for several cost evaluation criteria. Then, we show the continuity of the expected costs as functions of policies over the space of stationary control policies when the policies are given a topology introduced in [Borkar'89]. The same applies for finite horizon problems when the control policies are Markov, and the topology is revised to include time also as a parameter. We then establish that finite action/piecewise constant stationary policies are dense in the space of stationary Markov policies under this topology and the same holds for continuous policies. Using these, we establish that finite action/piecewise constant policies approximate optimal stationary policies with arbitrary precision.
In the second part of the talk, we focus on discrete-time approximations: By providing an appropriate topology on control policies, showing continuous dependence of expected costs in the policies (via several methods tailored to the information structures), and approximating them with piece-wise constant admissible policies, we show that for a large class of controlled diffusions with full information, partial information and decentralized information, we can obtain a sequence of discrete-time Markov Decision Problems (and their partially observed or decentralized/multi-agent counterparts) whose solutions are asymptotically optimal as the discretization steps decrease to zero. These provide a unified approximation paradigm for controlled diffusions under general information. In particular, numerical or reinforcement learning methods for discrete-time models (under full state information, partial-state information or decentralized information) can be applied for continuous-time models with rigorous near-optimality results.
Bio: Dr. Somnath Pradhan is a Coleman postdoctoral fellow in the Department of Mathematics and Statistics at Queen’s University, Canada. He was a National postdoctoral fellow in the Department of Mathematics at Indian Institute of Science Education and Research, Pune, India from 2019-21. He obtained his Ph.D. in Mathematics from the Indian Institute of Science in 2019. His research interests include stochastic processes and its application to stochastic control theory and related fields such as stochastic systems, filtering theory, learning theory, stochastic games, and dynamical systems.