1 / 96

Short-course Compressive Sensing of Videos Venue CVPR 2012, Providence, RI, USA June 16, 2012

Short-course Compressive Sensing of Videos Venue CVPR 2012, Providence, RI, USA June 16, 2012. Organizers : Richard G. Baraniuk Mohit Gupta Aswin C. Sankaranarayanan Ashok Veeraraghavan. Part 2: Compressive sensing Motivation, theory, recovery. Linear inverse problems

kaspar
Download Presentation

Short-course Compressive Sensing of Videos Venue CVPR 2012, Providence, RI, USA June 16, 2012

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript


  1. Short-course Compressive Sensing of Videos Venue CVPR 2012, Providence, RI, USA June 16, 2012 Organizers: Richard G. Baraniuk Mohit Gupta Aswin C. Sankaranarayanan Ashok Veeraraghavan

  2. Part 2: Compressive sensingMotivation, theory, recovery • Linear inverse problems • Sensing visual signals • Compressive sensing • Theory • Hallmark • Recovery algorithms • Model-based compressive sensing • Models specific to visual signals

  3. Linear inverse problems

  4. Linear inverse problems • Many classic problems in computer can be posed as linear inverse problems • Notation • Signal of interest • Observations • Measurement model • Problem definition: given , recover measurement matrix measurement noise

  5. Linear inverse problems • Problem definition: given , recover • Scenario 1 • We can invert the system of equations • Focus more on robustness to noise via signal priors

  6. Linear inverse problems • Problem definition: given , recover • Scenario 2 • Measurement matrix has a (N-M) dimensional null-space • Solution is no longer unique • Many interesting vision problem fall under this scenario • Key quantity of concern: Under-sampling ratio M/N

  7. Image super-resolution Low resolution input/observation 128x128 pixels

  8. Image super-resolution 2x super-resolution

  9. Image super-resolution 4x super-resolution

  10. Image super-resolution Super-resolution factor D Under-sampling factor M/N = 1/D2 General rule: The smaller the under-sampling, the more the unknowns and hence, the harder the super-resolution problem

  11. Many other vision problems… • Affine rank minimizaiton, Matrix completion, deconvolution, Robust PCA • Image synthesis • Infilling, denoising, etc. • Light transport • Reflectance fields, BRDFs, Direct global separation, Light transport matrices • Sensing

  12. Sensing visual signals

  13. High-dimensional visual signals Fog Volumetric scattering Reflection Human skin Sub-surface scattering Refraction Electron microscopy Tomography

  14. The plenopticfunction Collection of allvariationsof light in a scene space (3D) spectrum (1D) time (1D) angle (2D) Different slices reveal different scene properties Adelson and Bergen (91)

  15. The plenopticfunction space (3D) time (1D) spectrum (1D) angle (2D) High-speed cameras Hyper-spectral imaging Lytro light-field camera

  16. Sensing the plenoptic function • High-dimensional • 1000 samples/dim == 10^21 dimensional signal • Greater than all the storage in the world • Traditional theories of sensing fail us!

  17. Resolution trade-off • Key enabling factor: Spatial resolution is cheap! • Commercial cameras have 10s of megapixels • One idea is the we trade-off spatial resolution for resolution in some other axis

  18. Spatio-angular tradeoff [Ng, 2005]

  19. Spatio-angular tradeoff [Levoy et al. 2006]

  20. Spatio-temporal tradeoff [Bub et al., 2010] Stagger pixel-shutter within each exposure

  21. Spatio-temporal tradeoff [Bub et al., 2010] Rearrange to get high temporal resolution video at lower spatial-resolution

  22. Resolution trade-off • Very powerful and simple idea • Drawbacks • Does not extend to non-visible spectrum • 1 Megapixel SWIR camera costs 50-100k • Linear and global tradeoffs • With today’s technology, cannot obtain more than 10x for video without sacrificing spatial resolution completely

  23. Compressive sensing

  24. Sense by Sampling sample

  25. Sense by Sampling too much data! sample

  26. Sense then Compress sample compress JPEG JPEG2000 … decompress

  27. Sparsity largewaveletcoefficients (blue = 0) pixels

  28. Sparsity largewaveletcoefficients (blue = 0) largeGabor (TF)coefficients pixels widebandsignalsamples frequency time

  29. Concise Signal Structure • Sparse signal: only K out of N coordinates nonzero sorted index

  30. Concise Signal Structure • Sparse signal: only K out of N coordinates nonzero • model: union of K-dimensional subspaces aligned w/ coordinate axes sorted index

  31. Concise Signal Structure • Sparse signal: only K out of N coordinates nonzero • model: union of K-dimensional subspaces • Compressible signal: sorted coordinates decay rapidly with power-law power-lawdecay sorted index

  32. What’s Wrong with this Picture? • Why go to all the work to acquire N samples only to discard all but K pieces of data? sample compress decompress

  33. What’s Wrong with this Picture? • nonlinear processing • nonlinear signal model • (union of subspaces) linear processing linear signal model (bandlimited subspace) sample compress decompress

  34. Compressive Sensing • Directly acquire “compressed” data via dimensionality reduction • Replace samples by more general “measurements” compressive sensing recover

  35. Sampling • Signal is -sparse in basis/dictionary • WLOG assume sparse in space domain sparsesignal nonzeroentries

  36. Sampling • Signal is -sparse in basis/dictionary • WLOG assume sparse in space domain • Sampling sparsesignal measurements nonzeroentries

  37. Compressive Sampling • When data is sparse/compressible, can directly acquire a condensed representation with no/little information loss through linear dimensionality reduction sparsesignal measurements nonzero entries

  38. How Can It Work? • Projection not full rank…… and so loses information in general • Ex: Infinitely many ’s map to the same(null space)

  39. How Can It Work? • Projection not full rank…… and so loses information in general • But we are only interested in sparse vectors columns

  40. How Can It Work? • Projection not full rank…… and so loses information in general • But we are only interested in sparse vectors • is effectively MxK columns

  41. How Can It Work? • Projection not full rank…… and so loses information in general • But we are only interested in sparse vectors • Design so that each of its MxK submatrices are full rank (ideally close to orthobasis) • Restricted Isometry Property (RIP) columns

  42. Restricted Isometry Property (RIP) Preserve the structure of sparse/compressible signals K-dim subspaces

  43. Restricted Isometry Property (RIP) K-dim subspaces • “Stable embedding” • RIP of order 2K implies: for all K-sparse x1 and x2

  44. RIP = Stable Embedding An information preserving projection preserves the geometry of the set of sparse signals RIP ensures that

  45. How Can It Work? • Projection not full rank…… and so loses information in general • Design so that each of its MxK submatrices are full rank (RIP) • Unfortunately, a combinatorial, NP-complete design problem columns

  46. Insight from the 70’s [Kashin, Gluskin] • Draw at random • iid Gaussian • iid Bernoulli … • Then has the RIP with high probability provided columns

  47. Randomized Sensing • Measurements = random linear combinationsof the entries of the signal • No information loss for sparse vectors whp sparsesignal measurements nonzero entries

  48. CS Signal Recovery • Goal: Recover signal from measurements • Problem: Randomprojection not full rank(ill-posed inverse problem) • Solution: Exploit the sparse/compressiblegeometry of acquired signal

  49. CS Signal Recovery • Random projection not full rank • Recovery problem:givenfind • Null space • Search in null space for the “best”according to some criterion • ex: least squares (N-M)-dim hyperplaneat random angle

  50. Signal Recovery • Recovery: given(ill-posed inverse problem) find (sparse) • Optimization: • Closed-form solution:

More Related