1 / 67

Part II.3 Evaluation of algorithms

Part II.3 Evaluation of algorithms. max. Scalar solution methods Population based methods Evaluation of algorithms. B. A. D. C. max. Performance assessment for Pareto optimization algorithms. Limit behavior of stochastic optimizers.

donnelly
Download Presentation

Part II.3 Evaluation of algorithms

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. Part II.3 Evaluation of algorithms max • Scalar solution methods • Population based methods • Evaluation of algorithms B A D C max

  2. Performance assessment for Pareto optimization algorithms

  3. Limit behavior of stochastic optimizers Viewpoint 1: Randomized search heuristics Qualitative: Limit behavior for t → ∞ Probability{Optimum found} 1 Quantitative: Expected Running Time E(T) Algorithm A applied to Problem B 1/2 ∞ Computation Time (number of iterations)

  4. Limit behavior of stochastic optimizers Viewpoint 2: Optimum approximation algorithms Qualitative: Limit behavior for t → ∞ Quality of solution Qmax Quantitative: Trade-off E(Solution Quality) vs. Time Algorithm A applied to Problem B ∞ Computation Time (number of iterations)

  5. Limit Behavior of Multiobjective EA: Related Work • Requirements for archive: • Convergence • Diversity • Bounded Size [Rudolph 98,00] [Veldhuizen 99] [Rudolph 98,00] [Hanne 99] [Thiele et al. 02] convergence to whole Pareto front (diversity trivial) “store all” “store m” convergence to Pareto front subset (no diversity control) (impractical) (not sufficient)

  6. The concept of archiving optimization archiving finitememory generate update, truncate finitearchive A

  7. Unbounded archives

  8. Bounded archive of size M

  9. Bounded archive with diverse solutions     0 0 y2 y1

  10. Lemma on functional representation of Pareto fronts

  11. Theoretical Running Time Analysis for EA problem domain type of results • expected RT (bounds) • RT with high probability (bounds) [Mühlenbein 92] [Rudolph 97] [Droste, Jansen, Wegener 98,02][Garnier, Kallel, Schoenauer 99,00] [He, Yao 01,02] discrete search spaces Single-objective EAs • asymptotic convergence rates • exact convergence rates continuous search spaces [Beyer 95,96,…] [Rudolph 97] [Jagerskupper 03] [Laumanns, Thiele, Deb, Zitzler: GECCO2002] [Laumanns, Thiele, Zitzler, Welzl, Deb: PPSN-VII] Multiobjective EAs discrete search spaces

  12. Theoretical Running Time Analysis

  13. Which technique is suited for which problem class?  Theoretically (by analysis): difficult Limit behavior (unlimited run-time resources) Running time analysis Empirically (by simulation): standard Problems: randomness, multiple objectives Issues:quality measures, statistical testing, benchmark problems, visualization, …

  14. Comparison of non-dominated sets

  15. Quality measures A A B B Is A better than B? independent ofuser preferences Yes (strictly) No dependent onuser preferences How much? In what aspects? Ideal: quality measures allow to make both type of statements

  16. Unary quality indicators

  17. Unary quality indicators

  18. Unary quality indicators

  19. Comparisons in practise From: M. Emmerich, Single- and Multiobjective Optimization, ElDorado 2005

  20. Comparison of sets

  21. Diversity measures

  22. Some notation

  23. Comparison of non-dominated sets

  24. Comparison methods

  25. Comparison methods

  26. Comparison methods

  27. Linking comparison methods and dominance relations

  28. Linking comparison methods and dominance relations

  29. Completeness and Compatibility for the binary e-indicator

  30. Combined binary e-indicator

  31. Compatibility and completeness of unary operators and their combinations

  32. Compatibility and completeness of unary operators and their combinations

  33. Proof by contradiction

  34. Proof by contradiction

  35. Proof by contradiction

  36. Details for proof and further results

  37. Power of unary operators

  38. Averaging Pareto Front Approximation sets

  39. Averaging Pareto Fronts

  40. Example for a median attainment surface

  41. Averaging Pareto Fronts Plotting attainment surfaces: http://dbk.ch.umist.ac.uk/knowles/plot_attainments/ Viviane Grunert da Fonseca, Carlos M. Fonseca, and Andreia O. Hall. Inferential Performance Assessment of Stochastic Optimisers and the Attainment Function. In Eckart Zitzler, Kalyanmoy Deb, Lothar Thiele, Carlos A. Coello Coello, and David Corne, editors, First International Conference on Evolutionary Multi-Criterion Optimization, pages 213-225. Springer-Verlag. Lecture Notes in Computer Science No. 1993, 2001

  42. Test Function Construction: Deb ‘98a

  43. Convex function by Deb

  44. Construction of multimodal Pareto-fronts

  45. Construction of multi-global Pareto-fronts

  46. ED-Function, taking its optima at the naturals

  47. Zitzler Thiele Deb (ZDT) Problems

  48. ZDT 1 Problem

  49. ZDT1

  50. ZDT2 Problem

More Related