IEOR Seminar by Manu Gupta, IRIT
Title: A unifying computation scheme of Whittle's indices for Markovian bandits
Speaker: Manu Gupta, Institut de Recherche en Informatique de Toulouse (IRIT)
Time and Date: 11:30 am, 13th Aug, 2018 (Monday)
Venue: IEOR Seminar Hall, 2nd Floor, IEOR Building
Abstract: The multi-armed restless bandit framework allows to model a wide variety of decision-making problems in areas as diverse as industrial engineering, computer communication, operations research, financial engineering, communication networks, etc. In a seminal work, Whittle developed a methodology to derive well-performing (Whittle’s) index policies that are obtained by solving a relaxed version of the original problem. However, the computation of Whittle’s index itself is a difficult problem and hence researchers focused on calculation of Whittle’s index on a problem-to-problem basis. Our main contribution is the derivation of a closed-form expression for Whittle's index when bandit has Markovian evolution, which is valid as long as the technical condition of indexability is satisfied. Our solution approach provides a unifying expression for Whittle’s index, and as particular cases, we retrieve many known results from the literature including classical machine repairman problem, content delivery network, transmission control protocol (TCP) etc.
Speaker's Bio: Manu Gupta is a recipient of CIMI postdoc fellowship since August 2017 at Institut de recherche en informatique de Toulouse (IRIT) where his current work is related to multi-armed restless bandits. His research is focused on stochastic operations research, in particular on controls of queues and sequential decision making. He received his masters and PhD degree in the Department of Industrial Engineering and Operations Research at IIT Bombay. He also received best thesis award for his PhD work in dynamic priority queues at IIT Bombay.