ATI's Optimization, Modeling and Simulation course
This course is an introduction to two closely related
areas: (1) stochastic search methods for system
optimization and (2) the analysis and construction of
Monte Carlo simulations. A few of the many areas where
stochastic optimization and simulation-based approaches
have emerged as indispensable include decision aiding,
prototype development for large-scale control systems,
performance analysis of communication networks,
control and scheduling of complex manufacturing
processes, and computer-based personnel training. The
course focuses on core issues in algorithm design and
mathematical modeling, together with implications for
practical implementation. The course does not dwell on
theoretical details related to the methods; attendees are
directed to the appropriate literature for such details.
Attendees should have a solid working knowledge of
probability and statistics at the beginning graduate level
and knowledge of multivariate calculus, basic matrix
analysis, and linear algebra. To aid understanding, the
course will include a brief review of the prerequisite
mathematical material. Attendees will receive a copy of
the textbook Introduction to Stochastic Search and
Optimization by J. C. Spall (Wiley, 2003), a
comprehensive set of notes, and a CD with Matlab code
of the core algorithms. Although not required, attendees
are encouraged to bring a laptop with MATLAB installed.
The course will include class demonstrations and
opportunities to experiment with the algorithms.
James C. Spall James C. Spall holds three positions at the Johns Hopkins University: Principal Staff engineer at the Applied Physics Laboratory, Research Professor in the Department of Applied Mathematics and Statistics, and Chairman of the Applied and Computational Mathematics Program. Dr. Spall’s 30+ years of engineering and teaching experience includes work on numerous projects for the U. S. Navy and
DARPA. He also has extensive teaching
experience, including credit and non-credit courses for
working professionals. Dr. Spall has many publications,
including two books, one of which is the course text
Introduction to Stochastic Search and Optimization (Wiley,
2003). He holds two U.S. patents for inventions in control systems." to "He holds two U.S. patents, both licensed, for inventions in control systems. Dr. Spall is a Fellow of IEEE.
Stacy Hill joined the Johns Hopkins University Applied
Physics Laboratory in 1983. Dr. Hill has extensive
theoretical and practical experience in systems
modeling and analysis. He has led systems
analysis and modeling projects that
evaluated the performance of strategic
defense systems, and has published papers
and given invited talks on stochastic
simulation and optimization. He
received a Best Paper Award for
“Optimization of Discrete Event Dynamic Systems via
Simultaneous Perturbation Stochastic Approximation”
from the Institute of Industrial Engineers. Dr. Hill teaches
in the Johns Hopkins University School of Engineering
Program in Applied and Computational Mathematics and
serves on its Advisory Board.
Contact these instructors (please mention course name in the subject line)
What you will learn:
- Popular methods for stochastic optimization.
- To recognize when stochastic optimization techniques are necessary or beneficial.
- Advantages and disadvantages of popular methods for system optimization.
- Essential theoretical principles and assumptions underlying optimization and Monte
Carlo simulation and the implications for practical implementation.
- Basics of mathematical modeling and the link to Monte Carlo simulation.
- State-of-the-art methods for using Monte Carlo simulations to improve real system
- Brief Mathematical Review. Relevant multivariate analysis, matrix algebra,
probability, and statistics.
- Background on Search and Optimization. Basic issues and definitions.
Stochastic vs. deterministic methods. No free lunch theorems for optimization.
Summary of classical methods of optimization and their limitations.
- Direct Search Techniques. Introduction to direct random search. Monte Carlo
methods. Nonlinear simplex (Nelder-Mead) algorithms.
- Least-Squares-Type Methods. Recursive methods for linear systems.
Recursive least squares (RLS). Least mean squares (LMS). Connection to
- Stochastic Approximation for Linear and Nonlinear Systems. Root-finding
and gradient-based stochastic approximation (Robbins-Monro). Gradient-free
stochastic approximation: finite-difference (FDSA) and simultaneous
perturbation (SPSA) methods.
- Search Methods Motivated by Physical Processes. Simulated annealing and
related methods. Evolutionary computation and genetic algorithms.
Discrete stochastic optimization.
Statistical methods (e.g., ranking and selection, multiple comparisons), general random search methods, and discrete simultaneous perturbation SA (DSPSA).
- Model Building. Issues particular to Monte Carlo simulation models. Bias-variance
tradeoff. Selecting “best” model via cross-validation. Fisher
information matrix as summary measure.
- Simulation-Based Optimization. Use of Monte Carlo simulations to improve
performance of real-world system performance. Gradient-based methods
(infinitesimal perturbation analysis and likelihood ratio) and non-gradient-based
methods (FDSA, SPSA, etc.). Common random numbers.
- Markov Chain Monte Carlo. Monte Carlo methods for difficult calculations;
Metropolis-Hastings and Gibbs sampling. Applications to numerical
ion and statistical estimation.
- Input Selection and Experimental Design. Classical vs. optimal design. A
practical criterion for optimal design (D-optimality). Input selection in linear
and nonlinear models.
The course provides a rigorous introduction to popular stochastic methods for system optimization and Monte Carlo simulation.
This course is not on the current schedule of open enrollment courses. If you are interested in attending this or another course as open enrollment, please contact us at (410) 956-8805 or at firstname.lastname@example.org and indicate the course name and number of students who wish to participate. ATI typically schedules open enrollment courses with a lead time of 3-5 months. Group courses can be presented at your facility at any time. For on-site pricing, request an on-site quote. You may also call us at (410) 956-8805 or email us at email@example.com.