1 / 41

A T ransform-based V ariational F ramework

Pixel Club, November, 2013. A T ransform-based V ariational F ramework. Guy Gilboa. In a Nutshell. Fourier inspiration:. Fourier Scale. Fourier Scale. Spectral. TV Scale. TV Scale. TV Flow. Relations to eigenvalue problems. General linear: ( L linear operator)

cruz
Download Presentation

A T ransform-based V ariational F ramework

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript


  1. Pixel Club, November, 2013 A Transform-based Variational Framework Guy Gilboa

  2. In a Nutshell Fourier inspiration: Fourier Scale Fourier Scale Spectral TV Scale TV Scale TV Flow

  3. Relations to eigenvalue problems • General linear: (L linear operator) • Functional based Linear Nonlinear

  4. What can a transform-based approach give us? • Scale analysis based on the spectrum. • New types of filtering– otherwise hard to design: nonlinear LPF, BPF, HPF. • Nonlinear spectral theory – relation to eigenfunctions and eigenvalues. • Deeper understandingof the regularization, optimal design with respect to data, noise and artifacts.

  5. Examples of spectral applications today:Eigenfunctions for 3D processing Taken from L Cai, F Da, “Nonrigid deformation recovery..”, 2012. Taken from Zhang et al, “Spectral mesh processing”, 2010.

  6. Image Segmentatoin Eigenvectors of the graph Laplacian [Taken from I. Tziakos et al, “Color image segmentation using Laplacianeigenmaps”, 2009 ]

  7. Some Related Studies • Andreu, Caselles, Belletini, Novagaet al 2001-2012– TV flow theory. • Steidl et al 2004 – Wavelet – TV relation • Brox-Weickert 2006 – scale through TV-flow • Luo-Aujol-Gousseau 2009 – local scale measures • Benning-Burger 2012 – ground states (nonlinear spectral theory) • Szlam-Bresson – Cheeger cuts. • Meyer, Vese, Osher, Aujol, Chambolle, G. and many more – structure-texture decomposition. • Chambolle-Pock 2011, Goldstein-Osher2009– numerics.

  8. Scale Space – a Natural Way to Define Scale We’ll talk specifically about total-variation (TV-flow, Andreuet al - 2001): Scale space as a gradient descent:

  9. TV-Flow:A behavior of a disk in time[Andreu-Caselleset al–2001,2002, Bellettini-Caselles-Novaga-2002, Meyer-2001] Center of disk, first and second time derivatives: t … …

  10. Spectral TV basic framework Phi(t) definition

  11. Reconstruction Reconstruction formula Th. 1: The reconstruction formula recovers

  12. Spectral response Spectrum S(t) as a function of time t: f S(t) t

  13. Spectrum example f S(t)

  14. Dominant scales

  15. Eigenvalue problem The nonlinear eigenvalue problem with respect to a functional J(u) is defined by: We’ll show a connection to the spectral components .

  16. Solution of eigenfunctions Th. 2: For is an eigenfunction with eigenvalue then:

  17. What are the TV eigenfunctions? In 2D, is a characteristic function of a convex set. I then is an eigenfunction. [Giusti-1978], [Finn-1979],[Alter-Caselles-Chambolle-2003].

  18. Filtering H(t) Let H(t) be a real-valued function of t. The filtered spectral response is The filtered spatial response is

  19. Filtering, example 1: TV Band-Pass and Band-Stop filters f S(t) Band-pass Band-stop

  20. Disk band-pass example S(t)

  21. We have the basic framework

  22. Numerics • Many ways to solve. • Variational approach was chosen: • Currently use Chambolle’s projection algorithm (some spikes using Split-Bregman, under investigation). • In time: • 2nd derivative - central difference • 1st derivative - forward differnce • Discrete reconstruction algorithm proved for any regularizing scale-space (Th. 4).

  23. TV-Flow as a LPF Th. 3: The solution of the TV-flow is equivalent to spectral filtering with:

  24. Nonlocal TV • Reminder: NL-TV (G.-Osher2008): Gradient Functional

  25. Spectral NL-TV? • The framework can fit in principle many scale-spaces, like NL-TV flow. We can obtain a one-homogeneous regularizer. • What is a generalized nonlocal disk? • What are possible eigenfunctions? • It is expected to be able to process better repetitive textures and structures.

  26. Sparseness in the TV sense • Sparse spectrum – the signal has only a few dominant scales. • Can be a large objects • Or many small ones (here TV energy is large) • Natural images – are not very sparse in general

  27. Noise Spectrum Various standard deviations: S(t)

  28. Noise + signal • Not additive. • Spreads original image spectrum. • Needs to be investigated. Band-pass filtered u f f-u

  29. Spectral Beltrami Flow? Original Beltrami Flow Spectral Beltrami Initial trials on Beltrami flow with parameterization such that it is closer to TV Difference images: Spectral Beltrami • Keeps sharp contrast • Breaks extremum principle Values along one line (Green channel)

  30. Segmentation priors • Swoboda-Schnorr2013 – convex segmentation with histogram priors. • We can have 2D spectrum with histograms • Use it to improve segmentation S(t,h)

  31. Texture processing • Many texture bands • We can filter and manipulate certain bands and reconstruct a new image. • Generalization of structure-texture decomposition.

  32. Processing approach • Deconstruct the image into bands • Identify salient textures • Amplify / attenuate / spatial process the bands. • Reconstruct image with processed bands

  33. Color formulation • Vectorial TV – all definitions can be generalized in a straightforward manner to vector-valued images. • Bresson-Chan (2008) definition and projection algorithm is used for the numerics.

  34. Orange example

  35. Orange – close up Original Modes 2,3=0 Modes 2-5=x1.5

  36. Selected phi(t) modes (1, 5, 15, 40) f residual

  37. Old man

  38. Old man – close up Original 2 modes attenuated 7 modes attenuated

  39. Old Man - First 3 Modes Modes: 1 2 3

  40. Take Home Messages • Introduction of a new TV transformand TV spectrum. • Alternative way to understand and visualize scales in the image. • Highly selective scale separation, good for processing textures. • Can be generalized to other functionals.

  41. Thanks! • Refs. Google “Guy Gilboa publications” • Preliminary ideas are in SSVM 2013 paper. • Most material is in CCIT Tech report 803. • Up-to-date and organized - submitted journal version – contact me.

More Related