1 / 67

Divide-and-Conquer

Modified by: Daniel Gomez-Prado, University of Massachusetts Amherst. 7 2  9 4  2 4 7 9. 7  2  2 7. 9  4  4 9. 7  7. 2  2. 9  9. 4  4. Divide-and-Conquer. Outline and Reading. Divide-and-conquer paradigm (§5.2) Recurrence Equations (§5.2.1)

lore
Download Presentation

Divide-and-Conquer

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. Modified by: Daniel Gomez-Prado, University of Massachusetts Amherst 7 2  9 4 2 4 7 9 7  2 2 7 9  4 4 9 7 7 2 2 9 9 4 4 Divide-and-Conquer Divide-and-Conquer

  2. Outline and Reading • Divide-and-conquer paradigm (§5.2) • Recurrence Equations (§5.2.1) • Iterative substitution • Recursion trees • Guess-and-test • The master method • Merge-sort (§4.1.1) • Algorithm • Merging two sorted sequences • Merge-sort tree • Execution example • Analysis • Generic merging and set operations (§4.2.1) • Quick-sort (§4.3) • Algorithm • Partition step • Quick-sort tree • Execution example • Analysis of quick-sort (4.3.1) • In-place quick-sort (§4.8) • Summary of sorting algorithms Divide-and-Conquer

  3. Divide-and-Conquer • Divide-and conquer is a general algorithm design paradigm: • Divide the input data S in two or more disjoint subsets S1, S2, … • Conquer the subproblems by solving them recursively • base case for the recursion: If the subproblems are small enough just solve them • Combine the solutions for S1,S2, …, into a solution for S • Analysis can be done using recurrence equations Divide-and-Conquer

  4. Modified from: Mukund Narasimhan, University of Washington Base case Subproblem size 1 # subproblems Work dividing and combining Example: Sum of Queue SumQueue(Q) if (Q.length == 0 ) return 0 else return Q.dequeue() + SumQueue(Q) One subproblem Linear reduction in size (decrease by 1) Combining: constant (cost of 1 add) T(0)  b T(n)  c + T(n – 1) for n>0 Divide-and-Conquer

  5. Modified from: Mukund Narasimhan, University of Washington Iterative substitution method Sum of Queue Solution Equation: T(0)  b T(n)  c + T(n – 1) for n>0 Solution: (finding the closed form solution) T(n)  c + c + T(n-2)  c + c + c + T(n-3)  kc + T(n-k) for all k  nc + T(0) for k=n  cn + b = O(n) Divide-and-Conquer

  6. Modified from: Mukund Narasimhan, University of Washington Find: 9 3 5 7 8 9 12 15 Example: Binary Search Search a sorted array for a given item, x • If x == middle array element, return true • Else, BinarySearch lower (x<mid) or upper (x>mid) sub-array • 1 subproblem, half as large BinarySearch(A, x) if (A.size == 1) return (x == A[0]) mid = A.size / 2 if (x == A[mid]) return true else if (x < A[mid]) return BinarySearch( A_LowerHalf, x) else if (x > A[mid]) return BinarySearch( A_UpperHalf, x) Divide-and-Conquer

  7. Modified from: Mukund Narasimhan, University of Washington Base case Subproblem size 1 Work dividing and combining # subproblems Example: Binary Search Search a sorted array for a given item, x • If x == middle array element, return true • Else, BinarySearch lower (x<mid) or upper (x>mid) sub-array • 1 subproblem, half as large BinarySearch(A, x) if (A.size == 1) return (x == A[0]) mid = A.size / 2 if (x == A[mid]) return true else if (x < A[mid]) return BinarySearch( A_LowerHalf, x) else if (x > A[mid]) return BinarySearch( A_UpperHalf, x) Equation: T(1)  b T(n)  T(n/2) + c for n>1 Divide-and-Conquer

  8. Modified from: Mukund Narasimhan, University of Washington Iterative substitution method Binary Search Solution Equation: T(1)  b T(n)  T(n/2) + c for n>1 Solution: (finding the closed form solution) T(n)  T(n/2) + c T(n/4) + c + c T(n/8) + c + c + c T(n/2k) + kc T(1) + c log n where k = log n b + c log n = O(log n) Divide-and-Conquer

  9. Modified from: Mukund Narasimhan, University of Washington Recursion Tree for Binary Search Problem size Cost per stage n O(1) n/2 O(1) n/4 O(1) log n … … 1 O(1) Θ(log n) Divide-and-Conquer

  10. Example: Recursion Tree • The recursion tree is of the form: • where d > 0 is constant. • And the base case has a running time b Divide-and-Conquer

  11. bn b bn + dn logn Drawing the Recursion Tree With: Divide-and-Conquer

  12. Iterative Substitution forThe recursion tree • In the iterative substitution, or “plug-and-chug,” technique, we iteratively apply the recurrence equation to itself and see if we can find a pattern: • Note that base, T(n)=b, case occurs when 2i=n. That is, i = log n. • So, • Thus, T(n) is O(n log n). Divide-and-Conquer

  13. Guess-and-Test Method,Part 1 • In the guess-and-test method, we guess a closed form solution and then try to prove it is true by induction: • Guess: T(n) < cn log n. • Wrong: we cannot make this last line be less than cn log n Divide-and-Conquer

  14. Guess-and-Test Method, Part 2 • Recall the recurrence equation: • Guess #2: T(n) < cn log2 n. • if c > b. • So, T(n) is O(n log2 n). • In general, to use this method, you need to have a good guess and you need to be good at induction proofs. Divide-and-Conquer

  15. Modified from: Carola Wenk, University of Texas at San Antonio The divide-and-conquerdesign paradigm • Dividethe problem (instance) into subproblems. • asubproblems, eachof size n/b • Conquerthe subproblems by solving them recursively. The base case has a runtime c • Combinesubproblem solutions. Runtime is f(n) Divide-and-Conquer

  16. Master Method • Many divide-and-conquer recurrence equations have the form: • The Master Theorem: Divide-and-Conquer

  17. Master Method, Example 1 • The form: • The Master Theorem: • Example: Solution: logba=2, so case 1 says T(n) is O(n2). Divide-and-Conquer

  18. Master Method, Example 2 • The form: • The Master Theorem: • Example: Solution: logba=1, so case 2 says T(n) is O(n log2 n). Divide-and-Conquer

  19. Master Method, Example 3 • The form: • The Master Theorem: • Example: Solution: logba=0, so case 3 says T(n) is O(n logn). Divide-and-Conquer

  20. Master Method, Example 4 • The form: • The Master Theorem: • Example: Solution: logba=3, so case 1 says T(n) is O(n3). Divide-and-Conquer

  21. Master Method, Example 5 • The form: • The Master Theorem: • Example: Solution: logba=2, so case 3 says T(n) is O(n3). Divide-and-Conquer

  22. Master Method, Example 6 • The form: • The Master Theorem: • Example: (binary search) Solution: logba=0, so case 2 says T(n) is O(log n). Divide-and-Conquer

  23. Master Method, Example 7 • The form: • The Master Theorem: • Example: (heap construction) Solution: logba=1, so case 1 says T(n) is O(n). Divide-and-Conquer

  24. Iterative “Proof” of the Master Theorem • Using iterative substitution, let us see if we can find a pattern: • We then distinguish the three cases as • The first term is dominant • Each part of the summation is equally dominant • The summation is a geometric series Divide-and-Conquer

  25. Divide-and-Conquer,Algorithm Examples: • Merge SortQuick Sort Divide-and-Conquer

  26. 7 2  9 4 2 4 7 9 7  2 2 7 9  4 4 9 7 7 2 2 9 9 4 4 Merge Sort Divide-and-Conquer

  27. Outline • Merge-sort (§4.1.1) • Algorithm • Merging two sorted sequences • Merge-sort tree • Execution example • Analysis • Generic merging and set operations (§4.2.1) Divide-and-Conquer

  28. Merge-Sort • Merge-sort is a sorting algorithm based on the divide-and-conquer paradigm • Like heap-sort • It uses a comparator • It has O(n log n) running time • Unlike heap-sort • It does not use an auxiliary priority queue • It accesses data in a sequential manner (suitable to sort data on a disk) Divide-and-Conquer

  29. 2T(n/2) f(n)=bn Merge-Sort AlgorithmmergeSort(S, C) Inputsequence S with n elements, comparator C Outputsequence S sorted • according to C ifS.size() > 1 (S1, S2)partition(S, n/2) mergeSort(S1, C) mergeSort(S2, C) Smerge(S1, S2) • Merge-sort on an input sequence S with n elements consists of three steps: • Divide: partition S into two sequences S1and S2 of about n/2 elements each • Recur: recursively sort S1and S2 • Conquer: merge S1and S2 into a unique sorted sequence Base case: merge(elem1,elem2) Running time = b Divide-and-Conquer

  30. Merging Two Sorted Sequences Algorithmmerge(A, B) Inputsequences A and B withn/2 elements each Outputsorted sequence of A  B S empty sequence whileA.isEmpty() B.isEmpty() ifA.first().element()<B.first().element() S.insertLast(A.remove(A.first())) else S.insertLast(B.remove(B.first())) whileA.isEmpty() S.insertLast(A.remove(A.first())) whileB.isEmpty() S.insertLast(B.remove(B.first())) return S • The conquer step of merge-sort consists of merging two sorted sequences A and B into a sorted sequence S containing the union of the elements of A and B • Merging two sorted sequences, each with n/2 elements and implemented by means of a doubly linked list, takes O(n) time Divide-and-Conquer

  31. Modified from: Monica Nicolescu, University of Nevada, Remo Example: MERGE(L,R) Divide-and-Conquer

  32. Modified from: Monica Nicolescu, University of Nevada, Remo Example: MERGE(L,R) Divide-and-Conquer

  33. Modified from: Monica Nicolescu, University of Nevada, Remo Example (cont.) Divide-and-Conquer

  34. Modified from: Monica Nicolescu, University of Nevada, Remo Example (cont.) Divide-and-Conquer

  35. Modified from: Monica Nicolescu, University of Nevada, Remo Example (cont.) Done! Divide-and-Conquer

  36. Merge-Sort Tree • An execution of merge-sort is depicted by a binary tree • each node represents a recursive call of merge-sort and stores • unsorted sequence before the execution and its partition • sorted sequence at the end of the execution • the root is the initial call • the leaves are calls on subsequences of size 0 or 1 7 2  9 4 2 4 7 9 7  2 2 7 9  4 4 9 7 7 2 2 9 9 4 4 Divide-and-Conquer

  37. 7 2 9 4  2 4 7 9 3 8 6 1  1 3 8 6 7 2  2 7 9 4  4 9 3 8  3 8 6 1  1 6 7  7 2  2 9  9 4  4 3  3 8  8 6  6 1  1 Execution Example • Partition 7 2 9 4  3 8 6 11 2 3 4 6 7 8 9 Divide-and-Conquer

  38. 7 2  2 7 9 4  4 9 3 8  3 8 6 1  1 6 7  7 2  2 9  9 4  4 3  3 8  8 6  6 1  1 Execution Example (cont.) • Recursive call, partition 7 2 9 4  3 8 6 11 2 3 4 6 7 8 9 7 2  9 4 2 4 7 9 3 8 6 1  1 3 8 6 Divide-and-Conquer

  39. 7  7 2  2 9  9 4  4 3  3 8  8 6  6 1  1 Execution Example (cont.) • Recursive call, partition 7 2 9 4  3 8 6 11 2 3 4 6 7 8 9 7 2  9 4 2 4 7 9 3 8 6 1  1 3 8 6 7  2 2 7 9 4  4 9 3 8  3 8 6 1  1 6 Divide-and-Conquer

  40. 7  2 2 7 9 4  4 9 3 8  3 8 6 1  1 6 Execution Example (cont.) • Recursive call, base case 7 2 9 4  3 8 6 11 2 3 4 6 7 8 9 7 2  9 4 2 4 7 9 3 8 6 1  1 3 8 6 77 2  2 9  9 4  4 3  3 8  8 6  6 1  1 Divide-and-Conquer

  41. Execution Example (cont.) • Recursive call, base case 7 2 9 4  3 8 6 11 2 3 4 6 7 8 9 7 2  9 4 2 4 7 9 3 8 6 1  1 3 8 6 7  2 2 7 9 4  4 9 3 8  3 8 6 1  1 6 77 22 9  9 4  4 3  3 8  8 6  6 1  1 Divide-and-Conquer

  42. Execution Example (cont.) • Merge 7 2 9 4  3 8 6 11 2 3 4 6 7 8 9 7 2  9 4 2 4 7 9 3 8 6 1  1 3 8 6 7  22 7 9 4  4 9 3 8  3 8 6 1  1 6 77 22 9  9 4  4 3  3 8  8 6  6 1  1 Divide-and-Conquer

  43. Execution Example (cont.) • Recursive call, …, base case, merge 7 2 9 4  3 8 6 11 2 3 4 6 7 8 9 7 2  9 4 2 4 7 9 3 8 6 1  1 3 8 6 7  22 7 9 4  4 9 3 8  3 8 6 1  1 6 77 22 9 9 4 4 3  3 8  8 6  6 1  1 Divide-and-Conquer

  44. Execution Example (cont.) • Merge 7 2 9 4  3 8 6 11 2 3 4 6 7 8 9 7 2  9 42 4 7 9 3 8 6 1  1 3 8 6 7  22 7 9 4  4 9 3 8  3 8 6 1  1 6 77 22 9 9 4 4 3  3 8  8 6  6 1  1 Divide-and-Conquer

  45. Execution Example (cont.) • Recursive call, …, merge, merge 7 2 9 4  3 8 6 11 2 3 4 6 7 8 9 7 2  9 42 4 7 9 3 8 6 1  1 3 6 8 7  22 7 9 4  4 9 3 8 3 8 6 1  1 6 77 22 9 9 4 4 33 88 66 11 Divide-and-Conquer

  46. Execution Example (cont.) • Merge 7 2 9 4  3 8 6 11 2 3 4 6 7 8 9 7 2  9 42 4 7 9 3 8 6 1  1 3 6 8 7  22 7 9 4  4 9 3 8 3 8 6 1  1 6 77 22 9 9 4 4 33 88 66 11 Divide-and-Conquer

  47. Analysis of Merge-Sort • The height h of the merge-sort tree is O(log n) • at each recursive call we divide in half the sequence, • The overall amount or work done at the nodes of depth i is O(n) • we partition and merge 2i sequences of size n/2i • we make 2i+1 recursive calls • Thus, the total running time of merge-sort is O(n log n) Divide-and-Conquer

  48. Recurrence Equation Analysis • The conquer step of merge-sort consists of merging two sorted sequences, each with n/2 elements and implemented by means of a doubly linked list, takes at most bn steps, for some constant b. • Likewise, the basis case (n< 2) will take at b most steps. • Therefore, if we let T(n) denote the running time of merge-sort: • We can therefore analyze the running time of merge-sort by finding a closed form solution to the above equation. • That is, a solution that has T(n) only on the left-hand side. Divide-and-Conquer

  49. Iterative Substitution • In the iterative substitution, or “plug-and-chug,” technique, we iteratively apply the recurrence equation to itself and see if we can find a pattern: • Note that base, T(n)=b, case occurs when 2i=n. That is, i = log n. • So, • Thus, T(n) is O(n log n). Divide-and-Conquer

  50. 7 4 9 6 2 2 4 6 7 9 4 2 2 4 7 9 7 9 2 2 9 9 2. Quick-Sort Divide-and-Conquer

More Related