1 / 34

Submodular Function Minimization

Submodular Function Minimization . Collette Coullard IMA and Northwestern University November 25, 2002. Outline. Introduction to Submodular Function Minimization (SFM) Definitions Applications- Why is SFM interesting? History Separation-optimization equivalence Developments

adamdaniel
Download Presentation

Submodular Function Minimization

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript


  1. Submodular Function Minimization Collette Coullard IMA and Northwestern University November 25, 2002

  2. Outline • Introduction to Submodular Function Minimization (SFM) • Definitions • Applications- Why is SFM interesting? • History • Separation-optimization equivalence • Developments • Recent developments • Iwata, Fleischer, Fugishige algorithm • Lex Schrijver’s algorithm • Schrijver’s Algorithm • Conclusions

  3. subsets of E E A,B E A B  f(A) + f(B)  f(AB) + f(AB) Equivalently: SR E, e E E R S e  f(S {e}) – f(S)  f(R  {e}) – f(R) marginal value of e wrt S (cost) marginal value of e wrt R (cost) Submodular Function Definitions f2Eℝ is submodular if

  4. Example: Number of Different Colors e a marginal value of g g d k 0 b i f({a,b,d,e,k})= 5 < c 1 h f({e,k})= 2 j E f(E)=5 f({a,b,d,e,k,g})= 5 f({e,k,g})= 3

  5. Example: Number of Connections (Edges)) marginal value of g e a g d k 3 f’({a,b,d,e,k})= 3 b i < c 2 h f’({e,k})= 0 j E f’(E)= 11 f’({a,b,d,e,k,g})= 6 f’({e,k,g})= 2

  6. marginal value of g Example: - Number of Connections (Edges)) e a g d f’({a,b,d,e,k,g})= -6 k -3 f’({a,b,d,e,k})= -3 b i < f’({e,k,g})= -2 c -2 h f’({e,k})= 0 j E f’(E)= -11

  7. marginal value of g Example: Number of Colors - Number of Edges: f’’ = f + f’ e a g d f’’({a,b,d,e,k,g})= 5-6=-1 k -3 f’’({a,b,d,e,k})= 5-3=2 b i < f’’({e,k,g})= 3-2=1 c -1 h f’’({e,k})= 2-0=2 j E f’’(E)= 5 -11= -6

  8. G = digraph with node set V A A A  V\{s,t} s f(A) + f(B)f(AB) + f(AB) t f(A) := number of arcs leaving node set A  {s} B A B Example: Directed Cuts

  9. DC DC DC S R Benefit term Risk term Net cost marginal risk-marginal benefit wrt S marginal risk-marginal benefit wrt R Example: Benefit and Risk Tradeoff E: customers i

  10. Example: Find minimum (cardinality or capacity) (s,t)-cut (Note: ƒnot monotone) s t Example: Find minimum net value (colors-edges) node set Submodular Function Minimization:Find S E with minimum f(S)

  11. Example: Find minimum (cardinality or capacity) (s,t)-cut DC (Note: ƒnot monotone) s t Example: Find minimum net cost (max net profit) customer set Submodular Function Minimization:Find S E with minimum f(S)

  12. x* x0 P P Optimizationand Separation Given c and P, find x*P such that cx* cx, for every x P. Given x0 and P, either conclude x0P or return valid inequality axb with ax0 > b. Theorem(Grötschel,Lovasz, Schrijver 1981) The optimization problem for Pcan be solved in polynomial time if and only if the separation problem can be solved for Pin polynomial time.

  13. Greedy Algorithm (Edmonds): ce1 ce2 x*e1 = f({e1}) x*e2= f({e1,e2})-f({e1}) x* x0 xe1 P P xe2 P: x(A)  f(A), A  E x(E) = f(E) Optimization and Separation Given x0 and P, either conclude x0P or return valid inequality x(A)  f(A), with x0(A) > f(A) or min{ f(A) - x0(A) : A  E } submodular submodular function minimization

  14. Challenge(Grötschel,Lovasz, Schrijver 1981) Since Optimizationand Separation are polynomially equivalent, and the optimization problem for Phas a strongly polynomial combinatorial algorithm (GA), there should be a strongly polynomial combinatorial algorithm for the separation problem for P. There should be a strongly polynomial algorithm for Submodular Function Minimization (SFM).

  15. Developments • Submodular function and matroids • Rado (1942) • Ingleton (1959) • Submodular functions and optimization: Greedy algorithm maximizes linear functions over polymatroids • Edmonds (1970) • Polynomial solvability of submodular function minimization (SFM) via ellipsoid algorithm (separation-optimization equivalence) • Grötschel, Lovász, Schrijver (1981,1988) • Partial order of polymatroid extreme points +finite combinatorial algorithm for SFM • Bixby, Cunningham, Topkis (1985) • Pseudopolynomial combinatorial algorithm for SFM • Cunningham (1985) • Strongly polynomial algorithm for symmetric SFM • Queyranne (1998) • Strongly polynomial combinatorial algorithms for SFM • Iwata, Fleischer, Fugishige (1999), Iwata, Fleischer (2000), Fugishige, Iwata(2001) • Schrijver (1999)

  16. a collection of subsets of E (called independent sets) satisfying r is a submodularfunction on E Submodular function and matroids a finite set A matroid M=(E,I) I 1: empty set is independent I 2: subsets of independent sets are independent I 3: All maximal independent subsets of A  E have same size, called rank of A, r(A)

  17. submodular Matroid Examples Forest Matroid of a Graph • G=(V,E) • M=(E,I) edge sets with no cycles Rank functionon 2E:

  18. submodular Matroid Examples • G=(V,E) Cutset Matroid of a Graph • M=(E,I) edge sets with no cutsets Rank functionon 2E:

  19. a submodularfunction on E Matroid Partition • G=(V,E) • Can E be partitioned • into at most k forests? • (k independent sets) • This is a special case • of SFM.

  20. a submodularfunction on E Matroid Intersection • Given matroids M1=(E,I1) and M2=(E,I2), • max {|I|: II1 I2} = min {f1(A) + f2(E\A) : A  E} • Special case of SFM. Maximum matching in bipartite graph is a special case. • G=(UV,E)

  21. Theorem. (Edmonds) This system is TDI, which implies D always has Integral optimal solutions.  • D models the submodular function minimization problem • Feasible solutions to P provide lower bounds onf(S*) • Optimal solutions to P provide certificate of optimality for S* Mathematical Programming Formulationof SFM P: D:

  22. Schrijver’s Algorithm for SFM • Maintains a feasible solution x to P’: • Maintains feasibility of x by keeping: b is an extreme point of P’ • Iteratively improves:

  23. choice of A submodularity of f Extreme Points b is an extreme point of P’: Feasibility proof for b : Suppose smallest last j Then Lower ideals of the ordering are tight sets for b.

  24. Show examples in Excel.

  25. Optimality Condition: We have an xRE and an R E with x is feasible for P’ R has all the negative components and no positive components of x Ris the set of initial elements in the order, for each bi

  26. Optimality condition examples 2 4 1

  27. + + - - + R + - Auxiliary Network G=(E,A) case 1: no path from + to -.

  28. Auxiliary Network G=(E,A) + + - s - + t - + case 2: There is a path from + to -. Use last arc (s,t) to “improve x”.

  29. x = 1b1 + 2b2 + ... + kbk; 1 , ... , k  0; 1 + ... + k = 1 s -1 : . . +1 = 1 ( ’1b’1+ ... + ’kb’l) + 2b1 + ... + kbk x’ = x+ 1 s -1 : . +1 t = ’1b’1 + ’2b’2 + ... + ’kb’l; b1 +  t Improving x: An Iteration, algebraically

  30. s -1 : . +1 = ’1b’1 + ’2b’2 + ... + ’kb’l; b1 +  t Schrijver’s clever idea

  31. x = 1b1 + 2b2 + ... + kbk; 1 , ... , k  0; 1 + ... + k = 1 s -1 : . +1 = 1 ( ’1b’1+ ... + ’kb’l) + 2b1 + ... + kbk x+ 1 t x’ = ... s -1 : . +1 = ’1b’1 + ’2b’2 + ... + ’kb’l; b1 +  t An Iteration, geometrically

  32. Complexity of O(n6)   n n n n

  33. Conclusions and future work • Submodular Function Minimization (SFM) generalizes several combinatorial optimization problems. • Schrijver’s elegant algorithm • known basic ideas from ‘80s • new idea on step direction • clever tie-breaking for O(n6) complexity bound • C++ implementations in the works • Schrijver Algorithm • IFF Algorithms • possibly Queyranne’s column-generation idea • others?

More Related