ATI's Optimization, Modeling and Simulation course

Summary: This course is an introduction to two closely related areas: (1) stochastic search methods for system optimization and (2) the analysis and construction of Monte Carlo simulations. A few of the many areas where stochastic optimization and simulation-based approaches have emerged as indispensable include decision aiding, prototype development for large-scale control systems, performance analysis of communication networks, control and scheduling of complex manufacturing processes, and computer-based personnel training. The course focuses on core issues in algorithm design and mathematical modeling, together with implications for practical implementation. The course does not dwell on theoretical details related to the methods; attendees are directed to the appropriate literature for such details. Attendees should have a solid working knowledge of probability and statistics at the beginning graduate level and knowledge of multivariate calculus, basic matrix analysis, and linear algebra. To aid understanding, the course will include a brief review of the prerequisite mathematical material. Attendees will receive a copy of the textbook Introduction to Stochastic Search and Optimization by J. C. Spall (Wiley, 2003), a comprehensive set of notes, and a CD with Matlab code of the core algorithms. Although not required, attendees are encouraged to bring a laptop with MATLAB installed. The course will include class demonstrations and opportunities to experiment with the algorithms. Instructors: James C. Spall is a Principal Staff engineer at the Johns Hopkins University, Applied Physics Laboratory and is Chairman of the Applied and Computational Mathematics Program in the Johns Hopkins School of Engineering. Dr. Spall’s 20+ years of engineering experience includes work on numerous projects for the U. S. Navy and DARPA. He also has extensive teaching experience, including credit and non-credit courses for working professionals. Dr. Spall has many publications, including two books, one of which is the course text Introduction to Stochastic Search and Optimization (Wiley, 2003). He holds two U.S. patents for inventions in control systems. Dr. Spall is a Fellow of IEEE. Stacy Hill joined the Johns Hopkins University Applied Physics Laboratory in 1983. Dr. Hill has extensive theoretical and practical experience in systems modeling and analysis. He has led systems analysis and modeling projects that evaluated the performance of strategic defense systems, and has published papers and given invited talks on stochastic simulation and optimization. He received a Best Paper Award for “Optimization of Discrete Event Dynamic Systems via Simultaneous Perturbation Stochastic Approximation” from the Institute of Industrial Engineers. Dr. Hill teaches in the Johns Hopkins University School of Engineering Program in Applied and Computational Mathematics and serves on its Advisory Board. What you will learn: Popular methods for stochastic optimization. To recognize when stochastic optimization techniques are necessary or beneficial. Advantages and disadvantages of popular methods for system optimization. Essential theoretical principles and assumptions underlying optimization and Monte Carlo simulation and the implications for practical implementation. Basics of mathematical modeling and the link to Monte Carlo simulation. State-of-the-art methods for using Monte Carlo simulations to improve real system performance. Course Outline: Brief Mathematical Review. Relevant multivariate analysis, matrix algebra, probability, and statistics. Background on Search and Optimization. Basic issues and definitions. Stochastic vs. deterministic methods. No free lunch theorems for optimization. Summary of classical methods of optimization and their limitations. Direct Search Techniques. Introduction to direct random search. Monte Carlo methods. Nonlinear simplex (Nelder-Mead) algorithms. Least-Squares-Type Methods. Recursive methods for linear systems. Recursive least squares (RLS). Least mean squares (LMS). Connection to Kalman filtering. Stochastic Approximation for Linear and Nonlinear Systems. Root-finding and gradient-based stochastic approximation (Robbins-Monro). Gradient-free stochastic approximation: finite-difference (FDSA) and simultaneous perturbation (SPSA) methods. Search Methods Motivated by Physical Processes. Simulated annealing and related methods. Evolutionary computation and genetic algorithms. Model Building. Issues particular to Monte Carlo simulation models. Bias-variance tradeoff. Selecting “best” model via cross-validation. Fisher information matrix as summary measure. Simulation-Based Optimization. Use of Monte Carlo simulations to improve performance of real-world system performance. Gradient-based methods (infinitesimal perturbation analysis and likelihood ratio) and non-gradient-based methods (FDSA, SPSA, etc.). Common random numbers. Markov Chain Monte Carlo. Monte Carlo methods for difficult calculations; Metropolis-Hastings and Gibbs sampling. Applications to numerical integrat ion and statistical estimation. Input Selection and Experimental Design. Classical vs. optimal design. A practical criterion for optimal design (D-optimality). Input selection in linear and nonlinear models. Focus Sentence: The course provides a rigorous introduction to popular stochastic methods for system optimization and Monte Carlo simulation. Tuition: Tuition for this three-day course is $1490 per person at one of our scheduled public courses. Onsite pricing is available. Please call us at 410-956-8805 or send an email to ati@ATIcourses.com. Register Now Without Obligation