Skip to main content

IE 712: Selected Applications of Stochastic Models



This Jan-Apr'2019 Spring semester, this course is essentially a `Topics in Machine Learning' course.


These include a selection from these topics: loss functions, risks, etc. for classification/regression; time-series; learning with noise; probabilistic models; nueral nets, reinforcement learning, etc.


Some exposure to ML is desirable; contact me, if interesed.  First meeting is this Friday 11am in Seminar Room, 211, IEOR





Prerequisite: IE 611 or equivalent


The aim of this course is to look at some models that arise in decision making in an uncertain environment. Models are for analysis, or optimization or both. Emphasis will be on computational tools. A possible set of topics is as below.

Markov decision processes: finite and infinite horizon models. Optimality of Markov policies. Computational aspects. Examples from inventory systems, resource allocation, etc.

Learning algorithms: Temporal difference methods. Methods based on approximation functions; TD(lambda); Q-learning.

Stability of queuing models. Little's law and its extensions. Advanced queuing models in discrete and continuous time.

Some classes of stochastic scheduling rules; minimizing mean sum of completion times on a single machine with and without pre-emptions and index policies. Makespan with and without pre-emptions on parallel machines; due date related objectives.

  • D.P. Bertsekas (1995), Dynamic programming and optimal control, Vols 1 and 2. Athena publications, Belmont.
  • D.P. Bertsekas and J. Tsitsiklis (1996), Neuro-dynamic programming, Athena Scientific, Belmont.

  • R W Wolff Stochastic modeling and theory of queues. Prentice-Hall Inc.,Englewood Cliffs, 1989.

  • M Pinedo Scheduling: Theory, algorithms and systems. Prentice-Hall Inc., Englewood Cliffs, 1995.