Presented by: Mingyuan Zhou Duke University, ECE June 11, 2009

1. From Sparse Solutions of Systems of Equations to Sparse Modeling of Signals and ImagesAlfred M. Bruckstein (Technion), David L. Donoho (Stanford), Michael Elad (Technion) SIAM REVIEW 2009 Presented by: Mingyuan Zhou Duke University, ECE June 11, 2009

2. Outline • Introduction • The sparsest solution of Ax = b • Variations on P0 • Sparsity-seeking methods in signal processing • Processing of sparsely generated signals • Applications in image processing

3. Introduction • Under-determined linear system equation • L2 norm • L0 norm How can uniqueness of a solution be claimed? How to verify a candidate solution? How to efficiently solve the problem (the exhaustive search is a NP-hard problem)? What kind of approximations will work and how accurate can those be?

4. Current achievements • Conditions under which has a unique solution • Conditions under which has the unique solution as • Conditions under which the solution can be found by some “pursuit” algorithm • Less restrictive notions of sparsity, impact of noise, the behavior of approximate solutions, and the properties of problem instances defined by ensembles of random matrices…

5. The signal processing perspective • JPEG, DCT • JPEG-2000, DWT • The sparsity of representation under given basis is key to many important signal and image processing problems: Image compression, Image denoising, image deblurring, speech compression, audio compression… Measuring sparsity

6. The sparsest solution of Ax = b

7. Uniqueness • Uniqueness via the Spark • Uniqueness via the Mutual Coherence

8. Pursuit Algorithms: Practice • Greedy Algorithms • Convex Relaxation Techniques

9. Pursuit Algorithms: Performance • Greedy Algorithms • Convex Relaxation Techniques

10. Variations on P0 Uncertainty Principles and Sparsity

11. From Exact to Approximate Solutions • Relaxed constraint: • Stability: • Pursuit algorithms: OMP Iteratively reweighted least squares (IRLS) Iterative thresholding Stepwise algorithms: LARS and Homotopy

12. Performance of pursuit algorithms

13. Beyond Coherence Arguments • Empirical evidence: The column of A is drawn at random from a Gaussian distribution, , Without noise With noise

15. Phase transitions in typical behavior:

16. Phase transitions in typical behavior:

17. Restricted isometry property (RIP):

18. The sparsest solution of Ax = b: A summary • Uniqueness • Solvability • Approximate solutions • Beyond coherence

19. Sparsity-Seeking Methods in Signal Processing • Non-Gaussian Prior • Combined representation

20. Processing of Sparsely Generated Signals • Applications Analysis Compression Denoising Inverse problems Compressive sensing Morphological Component Analysis

21. The quest for a dictionary Reconstructed dictionaries Dictionaries learned from training data Dictionaries learned from data under test Learning Methods: MOD, K-SVD, BPFA

22. Applications in Image Processing • Compression of Facial Images

24. Denoising of Images

25. Denoising of Images

26. Summary