IE613: Online Machine Learning, Jan-April 2020

Aim of this course is to study bandit algorithms and analyze their performance.

Lecture Hours

Monday 5.30-7pm
Thursday 5.30-7pm


Online using MS Teams

Teaching Assistants

Fehmina Malik
Debamita Ghosh
Hitesh Gudwani
Harshit Pandey

TA Hours

Timings: Every Monday 2-4pm
Venue : Room 201, IEOR building


Introduction to batch learning: Empirical Rsik Minimization, PAC learning.

Online learning: adversarial and stochastic settings, online learning with expert feedback, bandit feedback.

Multi-Armed Bandits: Algorithms for simple and cumulative regret (expected and high probability), and their analysis.

Contextual Bandits

Pure exploration

Course Grades

20 points: Midterm
35 points: 3 Assingments
15 points: Pre-project report
30 points: Final project

Reference texts

  • S. Shalev-Shwartz and S. Ben-David, ``Understanding Machine Learning:From Theory to Algorithms,'' Cambridge University Press, 2014, Download.
  • T. Hastie, R. Tibshirani and J. Friedman, ``Elements of statistical machine learning,'' Springer series in statistics, 2009, Download.
  • S. Shalev-Shwartz, ``Online Learning and Online Convex Optimization,'' NOW publisher, 2012,Download.
  • S. Bubeck and N. Cesa-Bianchi, ``Regret Analysis of Stochastic and Nonstochastic Multi-armed Bandit Problems,'' NOW publisher, 2012, Download.
  • N. Cesa-Bianchi and G. Lugosi, ``Prediction, Learning, and Games, Cambridge University Press, 2006, Download.
  • T. Lattimor and C. Szepesvari, ``Bandit Algorithms, Cambridge University Press, Download.
  • Assignments

  • Assignment 1.
  • Assignment 2.
  • Assignment 3.
  • Project List.