Optimal Control and Reinforcement Learning - 535.741

This course will explore advanced topics in nonlinear systems and optimal control theory, culminating with a foundational understanding of the mathematical principals behind Reinforcement learning techniques popularized in the current literature of artificial intelligence, machine learning, and the design of intelligent agents like Alpha Go and Alpha Star. Students will first learn how to simulate and analyze deterministic and stochastic nonlinear systems using well-known simulation techniques like Simulink and standalone C++ Monte-Carlo methods. Students will then be introduced to the foundations of optimization and optimal control theory for both continuous- and discrete- time systems. Closed-form solutions and numerical techniques like co-location methods will be explored so that students have a firm grasp of how to formulate and solve deterministic optimal control problems of varying complexity. Discrete-time systems and dynamic programming methods will be used to introduce the students to the challenges of stochastic optimal control and the curse-of-dimensionality. Supervised learning and maximum likelihood estimation techniques will be used to introduce students to the basic principles of machine learning, neural-networks, and back-propagation training methods. The class will conclude with an introduction of the concept of approximation methods for stochastic optimal control, like neural dynamic programming, and concluding with a rigorous introduction to the field of reinforcement learning and Deep-Q learning techniques used to develop intelligent agents like DeepMind’s Alpha Go.

Course Prerequisite(s)

EN.535.641 Mathematical Methods for Engineers.