Approximation Algorithms

1. Approximation Algorithms

2. Motivation • By now we’ve seen many NP-Complete problems. • We conjecture none of them has polynomial time algorithm.

3. Motivation • Is this a dead-end? Should we give up altogether? ?

4. Motivation • Or maybe we can settle for good approximation algorithms?

5. Introduction • Objectives: • To formalize the notion of approximation. • To demonstrate several such algorithms. • Overview: • Optimization and Approximation • VERTEX-COVER, SET-COVER

6. Optimization • Many of the problems we’ve encountered so far are really optimization problems. • I.e - the task can be naturally rephrased as finding a maximal/minimal solution. • For example: finding a maximal clique in a graph.

7. Approximation • An algorithm which returns an answer C which is “close” to the optimal solution C* is called an approximation algorithm. • “Closeness” is usually measured by the ratio bound (n) the algorithm produces. • Which is a function that satisfies, for any input size n, max{C/C*,C*/C}(n).

8. VERTEX-COVER • Instance: an undirected graph G=(V,E). • Problem: find a set CV of minimal size s.t. for any (u,v)E, either uC or vC. Example:

9. Minimum VC NP-hard Proof: It is enough to show the decision problem below is NP-Complete: • Instance: an undirected graph G=(V,E) and a number k. • Problem: to decide if there exists a set V’V of size k s.t for any (u,v)E, uV’ or vV’. This follows immediately from the following observation.

10. Minimum VC NP-hard Observation: Let G=(V,E) be an undirected graph. The complement V\C of a vertex-cover C is an independent-set of G. Proof: Two vertices outside a vertex-cover cannot be connected by an edge. 

11. COR(B) 523-524 VC - Approximation Algorithm • C   • E’  E • whileE’   • do let (u,v) be an arbitrary edge of E’ • C  C  {u,v} • remove from E’ every edge incident to either u or v. • return C.

12. Demo Compare this cover to theone from the example

13. O(n2) O(1) O(n2) O(n) Polynomial Time • C   • E’  E • whileE’  do • let (u,v) be an arbitrary edge of E’ • C  C  {u,v} • remove from E’ every edge incident to either u or v • return C

14. Correctness The set of vertices our algorithm returns is clearly a vertex-cover, since we iterate until every edge is covered.

15. no common vertices! How Good an Approximation is it? Observe the set of edges our algorithm chooses  any VC contains 1 in each our VC contains both, hence at most twice as large

16. The Traveling Salesman Problem

17. The Mission: A Tour Around the World

18. The Problem: Traveling Costs Money 1795$

19. Introduction • Objectives: • To explore the Traveling Salesman Problem. • Overview: • TSP: Formal definition & Examples • TSP is NP-hard • Approximation algorithm for special cases • Inapproximability result

20. TSP • Instance: a complete weighted undirected graph G=(V,E) (all weights are non-negative). • Problem: to find a Hamiltonian cycle of minimal cost. 3 2 10 1 3 4 5

21. Polynomial Algorithm for TSP? What about the greedy strategy: At any point, choose the closest vertex not explored yet?

22. The Greedy $trategyFails 10   5 12 2  3 1 0

23. The Greedy $trategyFails 10   5 12 2  3 1 0

24. TSP is NP-hard The corresponding decision problem: • Instance: a complete weighted undirected graph G=(V,E) and a number k. • Problem: to find a Hamiltonian path whose cost is at most k.

25. TSP is NP-hard verify! Theorem:HAM-CYCLE p TSP. Proof: By the straightforward efficient reduction illustrated below: 0 1 0 0 1 k=0 0 HAM-CYCLE TSP

26. What Next? • We’ll show an approximation algorithm for TSP, • which yields a ratio-bound of 2 • for cost functions which satisfy a certain property.

27. The Triangle Inequality Definition: We’ll say the cost function c satisfies the triangle inequality, if u,v,wV : c(u,v)+c(v,w)c(u,w)

28. COR(B) 525-527 Approximation Algorithm 1. Grow a Minimum Spanning Tree(MST) for G. 2. Return the cycle resulting from a preorder walk on that tree.

29. Demonstration and Analysis The cost of a minimal hamiltonian cycle  the cost of a MST 

30. Demonstration and Analysis The cost of a preorder walk is twice the cost of the tree

31. Demonstration and Analysis Due to the triangle inequality, the hamiltonian cycle is not worse.

32. COR(B) 528 What About the General Case? • We’ll show TSP cannot be approximated within any constant factor 1 • By showing the corresponding gap version is NP-hard.

33. gap-TSP[] • Instance: a complete weighted undirected graph G=(V,E). • Problem: to distinguish between the following two cases: There exists a hamiltonian cycle, whose cost is at most |V|. The cost of every hamiltonian cycle is more than |V|. YES NO

34.  1  1  1  0 1 |V| |V| 0 +1 0 Instances min cost

35. NO! YES! |V| |V| gap What Should an Algorithm for gap-TSP Return? min cost DON’T-CARE...

36. gap-TSP & Approximation Observation: Efficient approximation of factor  for TSP implies an efficient algorithm for gap-TSP[].

37. gap-TSP is NP-hard Theorem: For any constant 1, HAM-CYCLE p gap-TSP[]. Proof Idea: Edges from G cost 1. Other edges cost much more.

38. The Reduction Illustrated 1 |V|+1 1 1 |V|+1 1 HAM-CYCLE gap-TSP Verify (a) correctness (b) efficiency

39. Approximating TSP is NP-hard gap-TSP[] is NP-hard Approximating TSP within factor  is NP-hard

40.  Summary • We’ve studied the Traveling Salesman Problem (TSP). • We’ve seen it is NP-hard. • Nevertheless, when the cost function satisfies the triangle inequality, there exists an approximation algorithm with ratio-bound 2.

41.  Summary • For the general case we’ve proven there is probably no efficient approximation algorithm for TSP. • Moreover, we’ve demonstrated a generic method for showing approximation problems are NP-hard.

42. SET-COVER • Instance: a finite setX and a family F of subsets of X, such that • Problem: to find a set CF of minimal size which coversX, i.e -

43. SET-COVER: Example

44. SET-COVER is NP-Hard Proof: Observe the corresponding decision problem. • Clearly, it’s in NP(Check!). • We’ll sketch a reduction from (decision) VERTEX-COVER to it:

45. SC VC VERTEX-COVER p SET-COVER one element for every edge one set for every vertex, containing the edges it covers

46. O(|F|·|X|) min{|X|,|F|} COR(B) 530-533 Greedy Algorithm • C   • U  X • whileU  do • select SF that maximizes |SU| • C  C  {S} • U  U - S • return C

47. compare to the optimal cover Demonstration 0 1 2 3 4 5

48. Is Being Greedy Worthwhile? How Do We Proceed From Here? • We can easily bound the approximation ratio by logn. • A more careful analysis yields a tight bound of lnn.

49. the n elements >½ what we covered Loose Ratio-Bound Claim: If  cover of size k, then after k iterations the algorithm covered at least ½ of the elements. Suppose it doesn’t and observe the situation after k iterations:

50. the n elements Loose Ratio-Bound Claim: If  cover of size k, then after k iterations the algorithm covered at least ½ of the elements. Since this part  can also be covered by k sets... >½ what we covered