The Perceptron, Feed-forward networks and Multi-layer perceptron, Memory based networks like Boltzmann Machines, Hopfield Networks. State based networks like Recurrent Neural Networks, Long Short Term Memory Networks. Convolutional Neural Networks, Bidirectional networks, Concept based networks used for transfer learning, Structural Networks for structured prediction, Attention based networks, Auto encoders for dimension reduction and embedding, Generative Adversarial Networks, Deep Gaussian Processes, Deep Bayesian nets, Deep Search Models, Deep Reinforcement Learning, Deep Neural Recommenders. Non-convex Optimization tools for Deep Networks. Theoretical tools to describe Convolutional Neural Networks and Recurrent Neural Networks. Learning theory for Deep Neural Networks. Several Applications covering operations research, computer vision, natural language processing, multi-media analytics, proof checking.