1 / 11

CHAPTER 6 S TOCHASTIC A PPROXIMATION AND THE F INITE- D IFFERENCE M ETHOD

Slides for Introduction to Stochastic Search and Optimization ( ISSO ) by J. C. Spall. CHAPTER 6 S TOCHASTIC A PPROXIMATION AND THE F INITE- D IFFERENCE M ETHOD. Organization of chapter in ISSO Contrast of gradient-based and gradient-free algorithms Motivating examples

viho
Download Presentation

CHAPTER 6 S TOCHASTIC A PPROXIMATION AND THE F INITE- D IFFERENCE M ETHOD

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. Slides for Introduction to Stochastic Search and Optimization (ISSO)by J. C. Spall CHAPTER 6STOCHASTIC APPROXIMATION AND THE FINITE-DIFFERENCE METHOD Organization of chapter in ISSO Contrast of gradient-based and gradient-free algorithms Motivating examples Finite-difference algorithm Convergence theory Asymptotic normality Selection of gain sequences Numerical examples Extensions and segue to SPSA in Chapter 7

  2. Motivation for AlgorithmsNot Requiring Gradient of Loss Function • Primary interest here is in optimization problems for which we cannot obtain direct measurements of L/q • cannotuse techniques such as Robbins-Monro SA, steepest descent, etc. • can (in principle) use techniques such as Kiefer and Wolfowitz SA (Chapter 6), genetic algorithms (Chapters 9–10),… • Many such “gradient-free” problems arise in practice • Generic difficult parameter estimation • Model-free feedback control • Simulation-based optimization • Experimental design: sensor configuration

  3. Model-Free Control Setup (Example 6.2 in ISSO)

  4. Finite Difference SA (FDSA) Method • FDSA has standard “first-order” form of root-finding (Robbins-Monro) SA • Finite difference approximation replaces direct gradient measurement (Chap. 5) • Resulting algorithm sometimes called Kiefer-Wolfowitz SA • Let denote FD estimate of g() at kth iteration (next slide) • Let denote estimate for  at kth iteration • FDSA algorithm has form where ak is nonnegative gain value • Under conditions,  in stochastic sense (a.s.)

  5. Finite Difference Gradient Approximation • Classical method for approximating gradients in Kiefer-Wolfowitz SA is by finite differences • FD gradient approximation used in SA recursion as gradient measurement (previous slide) • Standard two-sided gradient approximation at iteration k is where j is p-dimensional with 1 in jth entry, 0 elsewhere • Each computation of FD approximation takes 2p measurements y(•)

  6. Shaded Triangle ShowsValid Coefficient Values  and  in Gain Sequences ak = a/(k+1+A) and ck = c/(k+1) (Sect. 6.5 of ISSO) Solid line indicates non-strict border ( or ) and dashed line indicates strict border (>)

  7. Example: Wastewater Treatment Problem (Example 6.5 in ISSO) • Small-scale problem with p = 2 • Aim is to optimize water cleanliness and methane gas byproduct • Evaluated algorithms with 50 realizations of N = 2000 measurements • Used FDSA with gains ak = a/(1 + k) and ck = 1/(1 + k)1/6 • Asymptotically optimal decay rates found “best” • Gain tuning chooses a; naïve gain sets a = 1 • Also compared with random search algorithm B from Chapter 2 • Algorithms use noisy loss measurements (same level as in Example 2.7 in ISSO)

  8. Mean values ofL() with 95% Confidence Intervals

  9. Example: Skewed-Quartic Loss Function(Examples 6.6 and 6.7 in ISSO) • Larger-scale problem with p = 10: ()iis the ith component of B, and pB is an upper triangular matrix of ones • Used N = 1000 measurements; 50 replications • Used FDSA with gains ak = a/(1+k+A) and ck = c/(1+k) • “Semi-automatic” and manual gain tuning • Also compared with random search algorithm B

  10. Algorithm Comparison with Skewed-Quartic Loss Function (p = 10) (Example 6.6 in ISSO)

  11. Example with Skewed-Quartic Loss: Mean Terminal Values and 95% Confidence Intervals for

More Related