## IE712: Selected Application of Stochastic Models, July-November 2016Aim of this course is to learn modelling and anaylsis of engineering problems using theory of stochastic processes. ## Lecture HoursTuesday 3.35-5pmFriday 3.35-5pm ## LocationLT205## Teaching AssistantsArko ChatterjeeShashank Mishra ## SyllabusMarkov Decision Process: introduction to Markov chains, finite and infinite horizon models, optimality criteria, Markov Games: introduction to stochastic Games, discounted and average reward stochastic games, Shapley theorem, Reinforcement learning: approximating cost-to-go function, Temporal Difference learning methods (TD), Queuing Models: birth death process, stability of queues, Little's law, Priority queues, ## HomeworksHomework 1Homework 2 Homework 3 A short project ## Course Grades25 points: Homework and Quizzes20 points: Midterm exam 15 points: Project 40 points: Final exam ## Reference texts## Class notesLecture 1 MotivationLecture 2 Review of basic probability Lecture 3 Convergence of stochastic processes Lecture 4 Markov chains Lecture 5 Finite horizon MDPs Lecture 6 Priciple of optimality and Dynamic programming Lecture 7 Continuous time optimal control Lecture 8 Infinite horizon MDPs Lecture 9 Infinite horizon MDPs Lecture 10Infinite horizon MDPs Lecture 11 Infinite horizon MDPs Lecture 12 Competitive MDPs Lecture 13 Competitive MDPs Lecture 14 Competitive MDPs Lecture 15 Reinformcement Learning Lecture 16 Reinformcement Learning Lecture 17 Reinformcement Learning Lecture 18 Reinformcement Learning Lecture 19 Reinformcement Learning Lecture 20 Reinformcement Learning Lecture 21 Reinformcement Learning Lecture 22 Reinformcement Learning Lecture 23 Reinformcement Learning Lecture 24 Continuous chain Markov chains Lecture 25 Continuous chain Markov chains Lecture 26 Continuous chain Markov chains |