1 / 74

Discrete Optimization Lecture 2

Discrete Optimization Lecture 2. M. Pawan Kumar pawan.kumar@ecp.fr. Image Segmentation. How ?. Cost/Energy function. Models our knowledge about natural images. Optimize energy function to obtain the segmentation. Image Segmentation. Graph G = (V,E).

moesha
Download Presentation

Discrete Optimization Lecture 2

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. Discrete OptimizationLecture 2 M. Pawan Kumar pawan.kumar@ecp.fr

  2. Image Segmentation How ? Cost/Energy function Models our knowledge about natural images Optimize energy function to obtain the segmentation

  3. Image Segmentation Graph G = (V,E) Object - white, Background - green/grey Each vertex corresponds to a pixel Edges define a 4-neighbourhood grid graph Assign a label to each vertex from L = {obj,bkg}

  4. Image Segmentation Graph G = (V,E) Object - white, Background - green/grey Per Vertex Cost Cost of a labelling f : V  L Cost of label ‘bkg’ high Cost of label ‘obj’ low

  5. Image Segmentation Graph G = (V,E) Object - white, Background - green/grey Per Vertex Cost Cost of a labelling f : V  L Cost of label ‘bkg’ low Cost of label ‘obj’ high UNARY COST

  6. Image Segmentation Graph G = (V,E) Object - white, Background - green/grey Per Edge Cost Cost of a labelling f : V  L Cost of same label low Cost of different labels high

  7. Image Segmentation Graph G = (V,E) Object - white, Background - green/grey Per Edge Cost Cost of a labelling f : V  L Cost of same label high PAIRWISE COST Cost of different labels low

  8. Image Segmentation Graph G = (V,E) Object - white, Background - green/grey Problem: Find the labelling with minimum cost f*

  9. Object Detection How ? By defining an energy function !!

  10. Object Detection H T 1 L1 L2 L3 L4 Graph G = (V,E) Each vertex corresponds to a part - ‘Head’, ‘Torso’, ‘Legs’ Edges define a TREE Assign a label to each vertex from L = {positions}

  11. Object Detection H T 2 L1 L2 L3 L4 Graph G = (V,E) Each vertex corresponds to a part - ‘Head’, ‘Torso’, ‘Legs’ Edges define a TREE Assign a label to each vertex from L = {positions}

  12. Object Detection H T 3 L1 L2 L3 L4 Graph G = (V,E) Each vertex corresponds to a part - ‘Head’, ‘Torso’, ‘Legs’ Edges define a TREE Assign a label to each vertex from L = {positions}

  13. Object Detection H T 3 L1 L2 L3 L4 Graph G = (V,E) Cost of a labelling f : V  L Unary cost : How well does part match image patch? Pairwise cost : Encourages valid configurations Find best labelling f*

  14. Object Detection H T 3 L1 L2 L3 L4 Graph G = (V,E) Cost of a labelling f : V  L Unary cost : How well does part match image patch? Pairwise cost : Encourages valid configurations Find best labelling f*

  15. The General Problem 1 2 b c Graph G = ( V, E ) 1 Discrete label set L = {1,2,…,h} a d 3 Assign a label to each vertex f: V  L f e 2 2 Cost of a labelling Q(f) Unary Cost Pairwise Cost Find f* = arg min Q(f)

  16. Outline • Problem Formulation • Energy Function • MAP Estimation • Computing min-marginals • Reparameterization • MAP Estimation on Trees

  17. Energy Function Label l1 Label l0 Vb Vc Vd Va Db Dc Dd Da Random Variables V= {Va, Vb, ….} Labels L= {l0, l1, ….} Data D Labelling f: {a, b, …. }  {0,1, …}

  18. Energy Function 6 3 2 4 Label l1 Label l0 5 3 7 2 Vb Vc Vd Va Db Dc Dd Da Easy to minimize Q(f) = ∑a a;f(a) Neighbourhood Unary Potential

  19. Energy Function 6 3 2 4 Label l1 Label l0 5 3 7 2 Vb Vc Vd Va Db Dc Dd Da E : (a,b)  E iff Va and Vb are neighbours E = { (a,b) , (b,c) , (c,d) }

  20. Energy Function 0 6 1 3 2 0 4 Label l1 1 2 3 4 1 1 Label l0 1 0 5 0 3 7 2 Vb Vc Vd Va Db Dc Dd Da Pairwise Potential Q(f) = ∑a a;f(a) +∑(a,b) ab;f(a)f(b)

  21. Energy Function 0 6 1 3 2 0 4 Label l1 1 2 3 4 1 1 Label l0 1 0 5 0 3 7 2 Vb Vc Vd Va Db Dc Dd Da Q(f; ) = ∑a a;f(a) +∑(a,b) ab;f(a)f(b) Parameter

  22. Outline • Problem Formulation • Energy Function • MAP Estimation • Computing min-marginals • Reparameterization • Belief Propagation • Tree-reweighted Message Passing

  23. MAP Estimation 0 6 1 3 2 0 4 Label l1 1 2 3 4 1 1 Label l0 1 0 5 0 3 7 2 Vb Vc Vd Va Q(f; ) = ∑a a;f(a) + ∑(a,b) ab;f(a)f(b)

  24. MAP Estimation 0 6 1 3 2 0 4 Label l1 1 2 3 4 1 1 Label l0 1 0 5 0 3 7 2 Vb Vc Vd Va Q(f; ) = ∑a a;f(a) + ∑(a,b) ab;f(a)f(b) 2 + 1 + 2 + 1 + 3 + 1 + 3 = 13

  25. MAP Estimation 0 6 1 3 2 0 4 Label l1 1 2 3 4 1 1 Label l0 1 0 5 0 3 7 2 Vb Vc Vd Va Q(f; ) = ∑a a;f(a) + ∑(a,b) ab;f(a)f(b)

  26. MAP Estimation 0 6 1 3 2 0 4 Label l1 1 2 3 4 1 1 Label l0 1 0 5 0 3 7 2 Vb Vc Vd Va Q(f; ) = ∑a a;f(a) + ∑(a,b) ab;f(a)f(b) 5 + 1 + 4 + 0 + 6 + 4 + 7 = 27

  27. MAP Estimation 0 6 1 3 2 0 4 Label l1 1 2 3 4 1 1 Label l0 1 0 5 0 3 7 2 Vb Vc Vd Va q* = min Q(f; ) = Q(f*; ) Q(f; ) = ∑a a;f(a) + ∑(a,b) ab;f(a)f(b) f* = arg min Q(f; )

  28. MAP Estimation f* = {1, 0, 0, 1} 16 possible labellings q* = 13

  29. Computational Complexity |L||V| |L| = number of pixels ≈ 153600 Can we do better than brute-force?

  30. Outline • Problem Formulation • Energy Function • MAP Estimation • Computing min-marginals • Reparameterization • MAP Estimation on Trees

  31. Min-Marginals 0 6 1 3 2 0 4 Label l1 1 2 3 4 1 1 Label l0 1 0 5 0 3 7 2 Vb Vc Vd Va such that f(a) = i f* = arg min Q(f; ) Min-marginal qa;i

  32. Min-Marginals qa;0 = 15 16 possible labellings

  33. Min-Marginals qa;1 = 13 16 possible labellings

  34. Min-Marginals and MAP • Minimum min-marginal of any variable = • energy of MAP labelling qa;i mini ) mini ( such that f(a) = i minf Q(f; ) Va has to take one label minf Q(f; )

  35. Summary Energy Function Q(f; ) = ∑a a;f(a) + ∑(a,b) ab;f(a)f(b) MAP Estimation f* = arg min Q(f; ) Min-marginals s.t. f(a) = i qa;i = min Q(f; )

  36. Outline • Problem Formulation • Reparameterization • MAP Estimation on Trees

  37. Reparameterization 2 + - 2 2 0 4 1 1 2 + - 2 5 0 2 Vb Va Add a constant to all a;i Subtract that constant from all b;k

  38. Reparameterization 2 + - 2 2 0 4 1 1 2 + - 2 5 0 2 Vb Va Add a constant to all a;i Subtract that constant from all b;k Q(f; ’) = Q(f; )

  39. Reparameterization + 3 - 3 2 0 4 - 3 1 1 5 0 2 Vb Va Add a constant to one b;k Subtract that constant from ab;ik for all ‘i’

  40. Reparameterization + 3 - 3 2 0 4 - 3 1 1 5 0 2 Vb Va Add a constant to one b;k Subtract that constant from ab;ik for all ‘i’ Q(f; ’) = Q(f; )

  41. 3 1 3 1 3 0 1 1 1 2 2 4 2 4 2 4 0 1 2 1 5 0 5 5 2 2 2 Vb Vb Vb Va Va Va Reparameterization - 4 + 4 + 1 - 4 - 2 - 1 + 1 - 2 - 4 + 1 - 2 + 2 + Mab;k + Mba;i ’b;k = b;k ’a;i = a;i Q(f; ’) = Q(f; ) ’ab;ik = ab;ik - Mab;k - Mba;i

  42. Equivalently 2 + - 2 2 0 4 + Mba;i ’a;i = a;i 1 1 + Mab;k 2 + - 2 5 0 2 Vb Va ’ab;ik = ab;ik - Mab;k - Mba;i Reparameterization ’ is a reparameterization of , iff ’   Q(f; ’) = Q(f; ), for all f Kolmogorov, PAMI, 2006 ’b;k = b;k

  43. Recap MAP Estimation f* = arg min Q(f; ) Q(f; ) = ∑a a;f(a) + ∑(a,b) ab;f(a)f(b) Min-marginals s.t. f(a) = i qa;i = min Q(f; ) Reparameterization ’   Q(f; ’) = Q(f; ), for all f

  44. Outline • Problem Formulation • Reparameterization • MAP Estimation on Trees

  45. Belief Propagation • Some MAP problems are easy • Belief Propagation gives exact MAP for chains • Exact MAP for trees • Clever Reparameterization

  46. Two Variables 2 2 0 4 1 1 5 0 5 2 Vb Vb Va Va Add a constant to one b;k Subtract that constant from ab;ik for all ‘i’ ’b;k = qb;k Choose the right constant

  47. Two Variables 2 2 0 4 1 1 5 0 5 2 Vb Vb Va Va = 5 + 0 a;0 + ab;00 Mab;0 = min = 2 + 1 a;1 + ab;10 ’b;k = qb;k Choose the right constant

  48. Two Variables 2 2 0 4 1 -2 5 -3 5 5 Vb Vb Va Va ’b;k = qb;k Choose the right constant

  49. Two Variables f(a) = 1 2 2 0 4 1 -2 5 -3 5 5 Vb Vb Va Va ’b;0 = qb;0 Potentials along the red path add up to 0 ’b;k = qb;k Choose the right constant

  50. Two Variables 2 2 0 4 1 -2 5 -3 5 5 Vb Vb Va Va = 5 + 1 a;0 + ab;01 Mab;1 = min = 2 + 0 a;1 + ab;11 ’b;k = qb;k Choose the right constant

More Related