Title: Sample efficient estimation of the transition kernels of controlled Markov chains
Date and time: 11 June 2025 (Wednesday), 10:30 – 11:30 a.m.
Venue: IEOR Seminar Room
Abstract: We will present estimation bounds on non-parametric estimates of the transition kernels of Controlled Markov chains (CMC's). CMC's are a natural choice for modelling various industrial and medical processes, and are also relevant to reinforcement learning (RL). Therefore, learning the transition dynamics of CMC's in a sample efficient manner is an important question. We will attempt to answer this question when the underlying state-control space is both finite and infinite. Under finiteness, we will develop a Probably Approximately Correct (PAC) bound for estimation error. We will explore the additional challenges when the state-control space is infinite, and tackle them using techniques from adaptive estimation. At the end, we will also posit some open questions.
Biography: Imon Banerjee is the IEMS alumni fellow in the department of Industrial Engineering and Management Science (IEMS) at the McCormick school of engineering at Northwestern University. Prior to this, he completed his PhD in statistics at Purdue University, and did his bachelor's and master's in statistics from Indian Statistical Institute (ISI). While at Purdue, he received the Ross-Lynn research fellowship for promising graduate student research. His research interests involve non-parametric statistics and reinforcement learning (RL), but he is also interested in learning about statistical physics, and geometry.