1 / 86

Matting

Matting. Roey Izkovsky Yuval Kaminka. Helping Superman fly since 1978. Outline. The matting problem Previous work New approaches: The iterative approach Jue Wang, Michael F.Cohen Closed form solution Anat Levin, Dani Lischinski,Yair Weiss Comparison and summary Bonus?. Outline.

fairfax
Download Presentation

Matting

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. Matting Roey Izkovsky Yuval Kaminka Helping Superman fly since 1978

  2. Outline • The matting problem • Previous work • New approaches: • The iterative approach Jue Wang,Michael F.Cohen • Closed form solutionAnat Levin, Dani Lischinski,Yair Weiss • Comparison and summary • Bonus?

  3. Outline • The matting problem • Previous work • New approaches: • The iterative approach Jue Wang,Michael F.Cohen • Closed form solutionAnat Levin, Dani Lischinski,Yair Weiss • Comparison and summary • Bonus?

  4. The matting problem - Motivation  Image and video editing New background Composite image

  5. The matting problem - Motivation  Image and video editing Input image New image

  6. The matting problem • The separation of an image I into • Foreground object image F • Background image B • Alpha matte α – the opacity • Problem: extract F, B, α from image hair fur

  7. Why is matting challenging? • Under constrained problem:One equation, 3 unknowns  We need to constrain the problem!

  8. Outline • The matting problem • Previous work • New approaches: • The iterative approach Jue Wang,Michael F.Cohen • Closed form solutionAnat Levin, Dani Lischinski,Yair Weiss • Comparison and summary • Bonus?

  9. Previous work Two types: Known background Natural image matting matting

  10. Known Background • Blue screen Matting • Still under-constrained • Solution: make more assumptions • “Foreground contains no blue” • Other foreground distribution assumption… • Use two different backgrounds • Main flaw: need to know the background… Blue background Composite image

  11. Natural Image Matting • The assumptions: • Smoothness of the alpha matte • GMM for the Background and Foreground colors • Initial estimate:trimap provided by the user Background Foreground Unknown Input image Trimap

  12. Natural Image Matting • The algorithms framework: • Estimate F, B distributions from close pixels • Find best α by some method

  13. Knockout • Extrapolate F,B from close neighborhood • Estimate α from calculated F, B values

  14. Bayesian • Estimate F, B distributions in area • Find best α matching distributions

  15. Bayesian • P(F), P(B) from image samples • P(C|F,B,α) using a distribution for C

  16. Natural Image Matting • Main flaw: Accurate trimap required • Tedious to provide manually • Hard to extract automatically  In particular, not feasible to videos Input image Trimap Binary segmentation Adding unknown region

  17. Great.So let’s get started…

  18. Outline • The matting problem • Previous work • New approaches: • The iterative approach Jue Wang,Michael F.Cohen • Closed form solutionAnat Levin, Dani Lischinski,Yair Weiss • Comparison and summary • Bonus?

  19. New Approach to Matting Trimap reduces to scribbles Two new methods • Iterative optimization approach • Heuristic algorithmic optimization • A closed form solution • Mathematical approach Trimap Scribbles

  20. Iterative optimization approach Jue Wang Michael F. Cohen

  21. Iterative approach

  22. Iterative approach • Score: fit to image data +alpha matte smoothness • Iteratively propagating estimated results.

  23. Iterative optimization - outline • Initialize “work pixels” from scribbles • Repeatedly: • Expand work pixels • Find best alpha matte • Stop when finished

  24. Uc = {user scribbles} ui = 0 α = 0 ui = 0 α = 1 ui = 1 α = 0.5 Initialization • Introducing: • ui - uncertainty variable • Uc – work pixels

  25. Optimization Uc = {user scribbles + 15 pixel radius} Our goal: find α matte for Uc that minimizes the energy - Smoothness Data

  26. N Possible values for F N Possible values for B Vd Score for αp = α Image color Ip

  27. Vd • Fit measure of αp to Ip • Score for αp = α : Fi , Bj – possible values for F, B in the pixel wFi, wBj – corresponding weights

  28. α = 0.4 u = 0.5 α = 0.8 u = 0.3 α = 0.4 u = 0.4 α = 0.2 u = 0.3 α = 0.9 u = 0.2 α = 0.3 u = 0.3 Vd Fi , Bj – possible values for F, B in the pixel wFi, wBj – corresponding weights F Samples B Samples What happens when there are not enough F/B samples? p α = 0.5 u = 1.0

  29. Vd • Score for αp = α : • Discretize • and normalize

  30. Vs • Matte smoothness :

  31. Iterative optimization – step 2 Uc = {user scribbles + 15 pixel radius} Our goal: find α matte for Uc that minimizes the energy - Uc Graph Nodes = Pixels, Edges by 4-connectivity

  32. Iterative optimization – step 2 GOAL: Minimize BELIEF PROPAGATION

  33. Iterative optimization – step 2 GOAL: Minimize BELIEF PROPAGATION t=0 y mpq – message from p to q q p Vector: p’s “opinion” for each possible α for q

  34. Iterative optimization – step 2 GOAL: Minimize BELIEF PROPAGATION t=1 y mpq – new message pq myp – previous message yp q p

  35. Iterative optimization – step 2 GOAL: Minimize BELIEF PROPAGATION t=2,3,4… y q p

  36. Iterative optimization – step 2 GOAL: Minimize BELIEF PROPAGATION t=T (stopping time) y q p

  37. Iterative optimization – step 2 GOAL: Minimize BELIEF PROPAGATION t=T (stopping time) y q p Best state calculated for each node:

  38. Iterative optimization – step 3 Found α matte for Uc that minimizes the energy - Update F, B and uncertainty:

  39. Iterative optimization - algorithm • Initialize Uc, F, B, u and alpha matte from scribbles • Repeatedly: • Expand Uc by another 15 pixel radius • Find best alpha matte (BP) • Update F,B,u for new matte • Stop when total uncertainty is minimal Initial matte Propagation of α matte Final matte

  40. Iterative optimization - Results Input image Extracted matte

  41. Iterative optimization - Results Input image Composite image Extracted matte

  42. Iterative optimization - Results The ambiguity bunny

  43. Iterative optimization - Results Scribbles result Trimap result Ambiguity bunny with trimap Ambiguity bunny with scribbles

  44. Iterative optimization - Summary • Minimal user input • Applicable to video • Sensitive to ambiguity in F, B • Uses simple color-model • Performance: • 15-20 min. on a 640x480 image • Factor 50 reported by better implementation

  45. Fantastic.Let’s go on…

  46. Outline • The matting problem • Previous work • New approaches: • The iterative approach Jue Wang,Michael F.Cohen • Closed form solutionAnat Levin, Dani Lischinski,Yair Weiss • Comparison and summary • Bonus?

  47. Closed form solution Anat Levin Dani Lischinski Yair Weiss

  48. Closed form solution • Assumption: local smoothness in F, B  cancel out unknowns from the matte equs. • Solve for F,B and alphausing algebraic tricks.

  49. Closed form solution Assumptions: • F,B locally smooth.  treat F,B as constant in a small window w

  50. Closed form solution GOAL: Minimize - • Numerical stability • Bias to smoother matte wj

More Related