625.743 - Stochastic Optimization and Control Course Homepage
Instructor Information
James Spall
Email: james.spall@jhuapl.eduWork Phone: (443) 778-4960
James C. Spall is a member of the Principal Professional Staff at the JHU Applied Physics Laboratory, is the Chair of the Applied and Computational Mathematics Program within the JHU School of Engineering, and is a Research Professor in the JHU Department of Applied Mathematics and Statistics. Dr. Spall has published extensively in the areas of control and statistics (including two books) and holds two U.S. patents for inventions in control systems. Among other appointments, he was the Program Chair for the 2007 IEEE Conference on Decision and Control and is a Senior Editor for the IEEE Transactions on Automatic Control and Contributing Editor for the Current Index to Statistics. Dr. Spall has received numerous research and publications awards and is a Fellow of IEEE, a member of the American Statistical Association, and a Fellow of the engineering honor society Tau Beta Pi. He won the EP Excellence in Teaching Award in 2006. Click here for Web site with additional publications (and other) information.
Course Information
Course Description
Stochastic optimization plays an increasing role in the analysis and control of modern systems. This course introduces the fundamental issues in stochastic search and optimization with special emphasis on cases where classical deterministic search techniques (steepest descent, Newton-Raphson, linear and nonlinear programming, etc.) do not readily apply. These cases include many important practical problems, which will be briefly discussed throughout the course (e.g., neural network training, nonlinear control, experimental design, simulation- based optimization, sensor configuration, image processing, discrete-event systems, etc.). Both global and local optimization problems will be considered. Techniques such as random search, least mean squares (LMS), stochastic approximation, simulated annealing, evolutionary computation (including genetic algorithms), and machine learning are discussed.
Prerequisites
Multivariate calculus, linear algebra, and one semester of graduate probability and statistics (e.g., 625.403 Statistical Methods and Data Analysis). Some computer-based homework assignments will be given. It is recommended that this course only be taken in the last half of a student's degree program.
Course Goal
This course introduces fundamental issues in stochastic search and optimization with special emphasis on cases where classical deterministic techniques (linear and nonlinear programming, etc.) are inappropriate. The emphasis will be on rigorous analysis and interpretation and an objective treatment of various approaches. The methods considered are designed to: (i) cope with inherent system noise, (ii) be relatively insensitive to modeling uncertainty, and (iii) be capable of finding a global solution from among multiple local solutions. The advantages and disadvantages of various methods will be discussed.
Course Objectives
- Review and discuss some of the most popular methods for stochastic search and optimization, including random search, recursive least squares, Kalman filtering, stochastic approximation, simulated annealing, evolutionary and genetic algorithms, and reinforcement learning. The course will cover most of the material in Chapters 1 to 11 of the textbook (book information is at http://www.jhuapl.edu/ISSO)
- Develop a rigorous basis for understanding and analyzing algorithms in a stochastic sense.
- Discuss the relationships among algorithms and provide some guidance about which algorithms are appropriate for which problems.
- Provide the background to implement the algorithms and/or critically evaluate the implementations of others in a wide variety of practical problems.
When This Course is Typically Offered
Spring semester, odd years, at the Applied Physics Laboratory.
Syllabus
Topics Covered
- Basic issues in search and optimization; stochastic vs. deterministic methods; no free lunch theorems
- Random search
- Recursive methods for linear systems (LMS, RLS, Kalman filter)
- Stochastic approximation (SA)
- Stochastic gradient methods
- Finite-difference SA (FDSA)
- Simultaneous perturbation SA (SPSA)
- Simulated annealing and related methods
- Evolutionary computation
- Genetic algorithms
- Reinforcement learning (temporal difference)
Student Assessment Criteria
| Homework | 70% |
| Final project | 25% |
| Discussant on other project | 5% |
The assessment criteria and associated percentages above are tentative and may be changed. The final assessment criteria will be provided in the course syllabus to be handed out at the first course session.
Computer and Technical Requirements
Some homework assignments will require the use of a computer; the student is free to use any programming language with which he/she is comfortable (the book's Web site includes some MATLAB code).
Participation Expectations
Homework will be assigned on a weekly basis and will be due the following class. The homework exercises will include hand (“pencil and paper”) problems and computer-based assignments. Unless notified otherwise, students are required to work independently on the homework.
Textbooks
Textbook information for this course is available online through the MBS Direct Virtual Bookstore.
Course Notes
There are no notes for this course.
Final Words from the Instructor
This course is co-listed in the Applied and Computational Mathematics and Electrical and Computer Engineering Programs of JHU Engineering Programs for Professionals; it may serve as an elective in other JHU programs subject to adviser approval. Students should come into the class with a good working knowledge of probability and statistics at the beginning graduate level (at least at the level of 625.403) and knowledge of multivariate calculus and basic matrix algebra. It is recommended that this course be taken in the last half of a student's degree program.
(Last Modified: 07-22-2008 at 11:07:57 AM)