IE712: Selected Application of Stochastic Models, July-November 2016

Aim of this course is to learn modelling and anaylsis of engineering problems using theory of stochastic processes.

Lecture Hours

Location

Teaching Assistants

Syllabus

Markov Decision Process: introduction to Markov chains, finite and infinite horizon models, optimality criteria,
value iteration and policy iteration algorithms, examples from inventory systems, resource allocation, and path planning.

Markov Games: introduction to stochastic Games, discounted and average reward stochastic games, Shapley theorem,
computation of equilibria via linear programming.

Reinforcement learning: approximating cost-to-go function, Temporal Difference learning methods (TD),
Least Square Temporal Difference methods (LSTD), Q-learning.

Queuing Models: birth death process, stability of queues, Little's law, Priority queues,
single class queuing networks, multi-class queuing networks.

Homeworks

Course Grades

Reference texts

Class notes