Title: Stochastic Approximation and its Applications to Network Tomography and Reinforcement Learning (video-conferencing)
Speaker: Gugan Thoppe, Duke University, Durham, USA
Date: Tuesday, May 21 2019, 4:30PM
Venue: IEOR Seminar Room, IEOR 211
Abstract: Stochastic Approximation (SA) refers to the class of iterative stochastic algorithms useful for finding optimal points or zeros of deterministic functions when one has access only to its noisy online observations. Given a SA method, two questions of natural interest are i.) where would this method converge to, and ii.) how fast would it converge. In this talk, we will see some definitive answers to these questions based on our recent advances in SA analysis. Specifically, in the first part, I will present a novel stochastic Kaczmarz algorithm as a solution methodology for network tomography. We will discuss its convergence and prove that it converges to precisely where its deterministic counterpart would have. In the second part of the talk, we shall look at the class of two-timescale SA and their applications to reinforcement learning. Two-timescale methods are extremely useful because they mimic methods with nested loops but, at the same time, are also relatively easier to analyze. Our main result here is a convergence rate estimate for linear two-timescale SA and its dependence on the stepsize choice. We shall end with some promising future directions concerning nonlinear and distributed SA.
About the Speaker: Gugan Thoppe is a Postdoctoral Associate at Duke University, USA with Prof. Sayan Mukherjee. Earlier, he worked with Prof. Robert Adler as an EC Senior Researcher (postdoc) at Technion, Israel. He did his PhD in Systems Science with Prof. Vivek Borkar at TIFR, Mumbai. His work won the TAA-Sasken best thesis award for 2017. He is also a two-time recipient of the IBM PhD fellowship award (2013–14 and 2014-15). His research interests include random topology and stochastic approximation.