Effective gradient-free methods for inverse problems

1. Effective gradient-free methods for inverse problems Jyri Leskinen FiDiPro DESIGN project

2. Introduction • Current research • Evolutionary algorithms • Inverse problems • Case study: Electrical Impedance Tomography (EIT) • Future

3. Current research • Inverse problems • Shape reconstruction • Electrical Impedance Tomography (EIT) • Methods • Evolutionary algorithms (GA, DE) • Memetic algorithms • Parallel EAs • Implementation of the Game Theory • Nash GAs, MAs, DEs

4. Evolutionary algorithms • Based on the idea of natural selection (Darwin) • Operate a population of solution candidates (“individuals”) • New solutions by variation (crossover, mutation) • Convergence by selection (parent selection, survival selection)

5. Evolutionary algorithms • Several methods • Genetic algorithms (Holland, 1960s; Goldberg, 1989) • Evolutionary strategies (Rechenberg, 1960s) • Differential evolution (Price & Storn, 1995)

6. Evolutionary algorithms • Simple EA • Generate initial population • Until termination criteria met, • Select parents • Produce new individuals by crossing over the parents • Mutate some of the offspring • Select fittest individuals for the next generation

7. Evolutionary algorithms • Pros: • Global search methods • Easy to implement • Allows difficult objective functions • Cons: • Slow convergence rate • Many objective function evaluations needed

8. Local search methods • Operate on neighborhoods using certain moves • Pros: • Fast convergence rate • Less resource-intensive • Cons: • Converges to the nearest optimum • Gradient methods need “nice” objective function

9. Memetic algorithms • Hybridization of EAs and LSs • Global method • Improved convergence rate • Memetic algorithms • A class of hybrid EAs • Based on the idea of memes (Dawkins) • LS applied during the evolutionary process

10. Memetic algorithms • Simple MA • Generate initial population • Until termination criteria met, • Select parents • Produce new individuals by crossing over the parents • Mutate some of the offspring • Improve offspring by local search • Select fittest individuals for the next generation

11. Memetic algorithms • Typically Lamarckian • Acquired properties inherited • Unnatural • MAs not limited to that! • Parameter tuning • Local search operators as memes • Parameters encoded in chromosomes • Meme populations • etc.

12. Inverse problems • Inverse problem: • Data from a physical system • Construct the original model using available data and simulations • Typical IPs: • Image reconstruction • Electromagnetic scattering • Shape reconstruction

13. Inverse problems • Objective function for example a sum of squares min F(x) = ∑ |x(i) – x*(i)|2 • x: the vector of values from a simulated solution (forward problem) • x*: the vector of target values

14. Inverse problems • Often difficult to solve because of ill-posedness: the acquired data is not sufficient → the solution is not unique! • Extra information needed; regularization

15. Electrical Impedance Tomography • Used in • Medicine (experimental) • Geophysics • Industrial process imaging • Simple, robust, cost-effective • Poor spatial, good temporal resolution

16. Electrical Impedance Tomography • Data from electrodes on the surface of the object • Inject small current using two of the electrodes • Measure voltages using the other electrodes • Reconstruct internal resistivity distribution from voltage patterns

17. Electrical Impedance Tomography Source: Margaret Cheney et al. (1999)

18. Electrical Impedance Tomography Source: Margaret Cheney et al. (1999)

19. Electrical Impedance Tomography Source: The Open Prosthetics Project (http://openprosthetics.org)

20. Electrical Impedance Tomography • PDE: Complete Electrode Model • Forward problem: calculate voltage values Ul using FEM

21. Electrical Impedance Tomography • Inverse problem: minimize F(σh) by varying the piecewise constant conductivity distribution σh

22. Electrical Impedance Tomography • Mathematically hard, non-linear ill-posed problem • Typically solved using Newton-Gauss method + regularization (Tikhonov, …) • Resulting image smoothed, image artifacts

23. Electrical Impedance Tomography

24. Electrical Impedance Tomography • Solution: Reconstruct the image using discrete shapes? • Resulting objective function multimodal, non-smooth • Solution: Use global methods

25. Electrical Impedance Tomography • Simple test case: Recover circular homogeneity (6 control parameters) • Two different memetic algorithms proposed: • Lifetime Learning Local Search (LLLSDE) • Variation Operator Local Search (VOLSDE)

26. Electrical Impedance Tomography • Evolutionary framework based on the self-adaptive control parameter differential evolution (SACPDE) • LLLSDE: • Lamarckian MA • Local search operator Nelder-Mead simplex method • VOLSDE: • Weighting factor F improved by one-dimensional local search

27. Electrical Impedance Tomography • Five algorithms tested (GA, DE, SACPDE, LLLSDE, VOLSDE) • Result: • GA performed poorly • DE better, some failures • LLLSDE best, but the difference to other adaptive methods minimal

28. Electrical Impedance Tomography

29. Now & future • Improve diversity using multiple populations (“island model”) • EAs can be used to find Nash equilibria • Improve convergence rate with virtual Nash games? • Can competitive games sometimes produce better solutions than cooperative games in multi-objective optimization?

30. Thank you for your attention! Questions?