280 likes | 435 Views
Divide-and-Conquer. Recursive in structure Divide the problem into several smaller sub-problems that are similar to the original but smaller in size Conquer the sub-problems by solving them recursively. If they are small enough, just solve them in a straightforward manner.
E N D
Divide-and-Conquer • Recursive in structure • Divide the problem into several smaller sub-problems that are similar to the original but smaller in size • Conquer the sub-problems by solving them recursively. If they are small enough, just solve them in a straightforward manner. • Combine the solutions to create a solution to the original problem
An Example: Merge Sort • Divide: Divide the n-element sequence to be sorted into two subsequences of n/2 elements each • Conquer: Sort the two subsequences recursively using merge sort. • Combine: Merge the two sorted subsequences to produce the sorted answer.
Merge-Sort (A, p, r) INPUT: a sequence of n numbers stored in array A OUTPUT: an ordered sequence of n numbers 1. if p < r 2. then q [(p+r)/2] 3. Merge-Sort (A, p, q) 4. Merge-Sort (A, q+1, r) 5. Merge (A, p, q, r)
Analysis of Merge Sort • Divide: computing the middle takes (1) • Conquer: solving 2 sub-problem takes 2T(n/2) • Combine: merging n-element takes (n) • Total: T(n) = (1) if n = 1 T(n) = 2T(n/2) + (n) if n > 1 T(n) = (n lg n) (CLRS/Chapter 4)
Recurrence Relations • Recurrences (Chapter 4) • Substitution Method • Iteration Method • Master Method • Arising from Divide and Conquer (e.g. MERGE-SORT) T(n) = (1) if n c T(n) = a T(n/b) + D(n) + C(n) otherwise
Substitution Method • Guessing the form of the solutions, then using mathematical induction to find the constants and show the solution works. • It works well when it is easy to guess. But, there is no general way to guess the correct solution.
An Example • Solve: T(n) = 3T(n/3) + n T(n) 3c n/3 lg n/3 + n c n lg (n/3) + n = c n lg n - c n lg3 + n = c n lg n - n (c lg 3 - 1) c n lg n * The last step istrue for c 1 / lg3.
Making a Good Guess • Guessing a similar solution to the one that you have seen before • T(n) = 3T(n/3 + 5) + n similar to T(n) = 3T(n/3) + n when n is large, the difference between n/3 and (n/3 + 5) is insignificant • Another way is to prove loose upper and lower bounds on recurrence and then reduce the range of uncertainty. • Start with T(n) = (n) & T(n) = O(n2)T(n) = (n log n)
Subtleties • When the math doesn’t quite work out in the induction, try to adjust your guess with a lower-order term. For example: • We guess T(n) O(n) for T(n) = 3T(n/3)+ 4, but wehave T(n) 3c n/3 + 4 = c n + 4 • New guess isT(n) c n - b, where b 0 T(n) 3(c n/3 - b)+4 = c n - 3b + 4 = c n - b - (2b-4) Therefore, T(n) c n - b, if 2b - 4 0or if b 2
Changing Variables • Use algebraic manipulation to turn an unknown recurrence similar to what you have seen before. • Consider T(n) = 2T(n1/2) + lg n • Renamem = lg nand we have T(2m) = 2T(2m/2) + m • SetS(m) = T(2m)and we have S(m) = 2S(m/2) + m S(m) = O(m lg m) • Changing back from S(m) to T(n), we have T(n) = T(2m) = S(m) = O(m lg m) = O(lg n lg lg n)
Avoiding Pitfalls • Be careful not to misuse asymptotic notation. For example: • We can falsely proveT(n) = O(n) by guessing T(n) c n for T(n) = 2T(n/2) + n T(n) 2c n/2 + n c n + n = O(n) Wrong! • The err is that we haven’t proved T(n) c n
Exercises • Solution of T(n) = T(n/2) + 1 isO(lg n) • Solution of T(n) = 2T(n/2+ 17) + n isO(n lg n) • Solve T(n) = 2T(n1/2) + 1 by making a change of variables. Don’t worry whether values are integral.
Iteration Method • Expand (iterate) the recurrence and express it as a summation of terms dependent only on n and the initial conditions • The key is to focus on 2 parameters • the number of times the recurrence needs to be iterated to reach the boundary condition • the sum of terms arising from each level of the iteration process • Techniques for evaluating summations can then be used to provide bounds on solution.
An Example • Solve: T(n) = 3T(n/4) + n T(n) = n + 3T(n/4) = n + 3[ n/4 + 3T(n/16) ] = n + 3[n/4] + 9T(n/16) = n + 3[n/4] + 9[n/16] + 27T(n/64) T(n) n + 3n/4 + 9n/16 + 27n/64 + … + 3log4n(1) n (3/4)i+ (nlog43) = 4n+ o(n) = O(n)
Recursion Trees • Keep track of the time spent on the subproblems of a divide and conquer algorithm • A convenient way to visualize what happens when a recursion is iterated • Help organize the algebraic bookkeeping necessary to solve the recurrence
n/2 n/2 n/4 n/4 n/4 n/4 Merge Sort Running times to merge two sublists n Running time to sort the left sublist
Running Time n=n n 2¢(n/2) = n n/2 n/2 lg n 4¢(n/4) = n n/4 n/4 n/4 n/4 Total: n lg n
Recursion Trees and Recurrences • Useful even when a specific algorithm is not specified • For T(n) = 2T(n/2) + n2, we have
Recursion Trees T(n) = (n2)
Recursion Trees • For T(n) = T(n/3) + T(2n/3) + n T(n) = O(n lg n)
Master Method • Provides a “cookbook” method for solving recurrences of the formT(n) = a T(n/b) + f(n) • Assumptions: • a 1 and b 1 are constants • f(n)is an asymptotically positive function • T(n) is defined for nonnegative integers • We interpret n/b to mean either n/bor n/b
The Master Theorem With the recurrence T(n) = a T(n/b) + f(n) as in the previous slide, T(n) can be bounded asymptotically as follows: 1. If f(n)=O(nlogba-)for some constant > 0, then T(n)= (nlogba). 2. If f(n) = (nlogba), then T(n) = (nlogba lg n). 3. If f(n) = ( nlogba+ )for some constant > 0, and if a f(n/b) c f(n)for some constant c < 1and all sufficiently large n, then T(n)= (f(n)).
Simplified Master Theorem Let a 1 and b >1 be constants andlet T(n) be the recurrence T(n) = a T(n/b) + c nk defined for n 0. 1. If a>bk, then T(n) = ( nlogba ). 2. Ifa=bk, then T(n) = ( nk lg n ). 3. If a< bk, then T(n) = ( nk ).
Examples • T(n) = 16T(n/4) + n • a = 16, b = 4, thus nlogba = nlog416= (n2) • f(n) = n = O(nlog416-)where = 1 case 1. • Therefore, T(n) = (nlogba)=(n2) • T(n) = T(3n/7) + 1 • a = 1, b=7/3, and nlogba = nlog7/31 = n0 = 1 • f(n) = 1 = (nlogba)case 2. • Therefore, T(n) = (nlogba lg n) = (lg n)
Examples (Cont.) • T(n) = 3T(n/4) + n lg n • a = 3, b=4, thus nlogba = nlog43 = O(n0.793) • f(n) = n lg n = (nlog43 + )where 0.2case 3. • Therefore, T(n) = (f(n)) = (n lg n) • T(n) = 2T(n/2) + n lg n • a = 2, b=2, f(n) = n lg n, and nlogba = nlog22 = n • f(n)is asymptotically larger thannlogba, but not polynomially larger. The ratio lg n is asymptotically less than n for any positive . Thus, the Master Theorem doesn’t apply here.
Exercises • Use the Master Method to solve the following: • T(n) = 4T(n/2) + n • T(n) = 4T(n/2) + n2 • T(n) = 4T(n/2) + n3