Chap 5: Decrease & conquer

Chap 5: Decrease & conquer

Objectives • To introduce the decrease-and-conquer mind set • To show a variety of decrease-and-conquer solutions: • Depth-First Graph Traversal • Breadth-First Graph Traversal • Fake-Coin Problem • Interpolation Search • To discuss the strengths and weaknesses of a decrease-and-conquer strategy

Decrease and Conquer A PROBLEM OF SIZE n • Reduce problem instance to smaller instance of the same problem and extend solution • Solve smaller instance • Extend solution of smaller instance to obtain solution to original problem • Also called inductive orincremental SUBPROBLEM OF SIZE n-1 A SOLUTION TO SUBPROBLEM A SOLUTION TO THE ORIGINAL PROBLEM

Divide-and-Conquer Illustrated A PROBLEM OF SIZE n SUBPROBLEM 1 OF SIZE n/2 SUBPROBLEM 2 OF SIZE n/2 A SOLUTION TO SUBPROBLEM 1 A SOLUTION TO SUBPROBLEM 2 A SOLUTION TO THE ORIGINAL PROBLEM

Flavours of Decrease and Conquer • Decrease by a constant (usually 1): instance is reduced by the same constant on each iteration • Insertion sort • Graph Searching: DFS, BFS • Generating combinatorials • Decrease by a constant factor (usually 2): instance is reduced by same multiple on each iteration • Binary search • Fake-coin problem • Variable-size decrease: size reduction pattern varies from one iteration to the next • Euclid’s algorithm • Interpolation Search

Flashback: Knapsack by Exhaustive Search • Efficiency: Ω(2n)

Flashback:Generating Subsets • Combinatorics uses Decrease (by one) and Conquer Algorithms • Generateall 2n subsets of A = {a1, …, an} • Divide into subsets of {a1, …, an-1} that contain an and those that don’t • Sneaky Solution: establish a correspondence between bit strings and subsets. Bit n denotes presence (1) or absence (0) of element an • Generate numbers from 0 to 2n-1 convert to bit strings  interpret as subsets • Examples: 000 = Ø , 010 = {a2} , 110 = {a1, a2}

Flashback:Generating Permutations • Generate all n! reorderings of {1, …, n} • Generate all (n-1)! permutations of {1, …, n-1} • Insert n into each possible position (starting from the right or left, alternately) • Implemented by the Johnson-Trotter algorithm • Satisfies Minimal-Change requirement • Next permutation obtained by swapping two elements of previous • Useful for updating style algorithms • Example: • Start: 1 • Insert 2: 12 21 • Insert 3: 123 132 312 321 231 213

Decrease and Conquer A PROBLEM OF SIZE n • Reduce problem instance to smaller instance of the same problem and extend solution • Solve smaller instance • Extend solution of smaller instance to obtain solution to original problem • Also called inductive orincremental SUBPROBLEM OF SIZE n-1 A SOLUTION TO SUBPROBLEM A SOLUTION TO THE ORIGINAL PROBLEM

Exercise: Spot the Difference • Problem: Derive algorithms for computing an using: • Brute Force • Divide and conquer • Decrease by one • Decrease by constant factor (halve the problem size) • Hint: each can be described in a single line

Exercise: Spot the Difference • Problem: Derive algorithms for computing an using: • Brute Force : an= a*a*a*a*...*a n times • Divide and conquer : an= (an/2 )* (an/2) • Decrease by one : an= ( an-1)* a • Decrease by constant factor (halve the problem size) : an= (an/2)2

Graph Traversal • Many problems require processing all graph vertices in a systematic fashion • Data Structures Reminder: • Graph traversal strategies: • Depth-first search (traversal for the Brave) • Breadth-first search (traversal for the Cautious) a b d b c a d a d c d c a b

Depth-First Search • Explore graph always moving away from last visited vertex • Similar to preorder tree traversal DFS(G): G = (V, E) count  0 mark each vertex as 0 for each vertex v∈ V ifv is marked as 0 dfs(v) dfs(v): count  count + 1 mark v with count for each vertex w adjacent to vDO ifw is marked as 0 dfs(w)

Example: DFS a b c d e f g h Traversal Stack: (pre = push, post = pop) 1a8 2b73f24e1 5g66c5 7d48h3 Push order: a b f e g c d h Pop order: e f h d c g b a

a b c d e f g h DFS Forest • DFS Forest: A graph representing the traversal structure • Types of Edges: • Tree edges: edge to next vertex in traversal • Back edges: edge in graph to ancestor nodes • Forward edges: edge in graph to descendants • Cross edges: none of the above b f e a Backedge g c d h

Notes on Depth-First Search • Applicable to different graph structures: • Adjacency matrices: (V2) • Adjacency linked lists: (V+E) • Yields two orderings: • preorder: as vertices are 1st encountered (pushed) • postorder: as vertices become dead-ends (popped) • Applications: • Checking connectivity, finding connected components • Checking a-cyclicity • Searching state-space of problems for solution (AI)

Breadth-First Search bfs(v): count  count + 1 mark v with count initialize queue with v while queue not empty DO a front of queue for each vertex w adjacent to a ifw is marked as 0 count  count + 1 mark w with count add w to queue remove a from queue • Move across to all neighbours of the last visited vertex • Similar to level-by-level tree traversals • Instead of a stack, breadth-first uses a queue BFS(G):G = (V, E) count  0 mark each vertex as 0 for each vertex v∈ V if v is marked as 0 bfs(v)

Example: BFS a b c d e f g h Traversal Queue: a1 b2 e3 f4 g5 c6 h7 d8 Order: a b e f g c h d a f b e g c h d Crossedge

Notes on Breadth First Search • BFS has same efficiency as DFS and can be implemented with: • Adjacency matrices: (V2) • Adjacency linked lists: (V+E) • Yields single ordering of vertices • Applications: • Same as DFS, but can also find paths from a vertex to all other vertices with the smallest number of edges

Decrease and Conquer A PROBLEM OF SIZE n • Reduce problem instance to smaller instance of the same problem and extend solution • Solve smaller instance • Extend solution of smaller instance to obtain solution to original problem • Also called inductive orincremental SUBPROBLEM of smaller size A SOLUTION TO SUBPROBLEM A SOLUTION TO THE ORIGINAL PROBLEM

bfs(v): count  count + 1 mark v with count initialize queue with v while queue not empty DO a front of queue for each vertex w adjacent to a ifw is marked as 0 count  count + 1 mark w with count add w to queue remove a from queue DFS(G): G = (V, E) count  0 mark each vertex as 0 for each vertex v∈ V ifv is marked as 0 dfs(v) BFS(G):G = (V, E) count  0 mark each vertex as 0 for each vertex v∈ V if v is marked as 0 bfs(v)

bfs(v): count  count + 1 mark v with count initialize queue with v while queue not empty DO a front of queue for each vertex w adjacent to a ifw is marked as 0 count  count + 1 mark w with count add w to queue remove a from queue dfs(v): count  count + 1 mark v with count for each vertex w adjacent to vDO ifw is marked as 0 dfs(w)

Example: BFS a b c d e f g h Traversal Queue: a1 b2 e3 f4 g5 c6 h7 d8 Order: a b e f g c h d a f b e g c h d Crossedge

a b c d e f g h DFS Forest • DFS Forest: A graph representing the traversal structure • Types of Edges: • Tree edges: edge to next vertex in traversal • Back edges: edge in graph to ancestor nodes • Forward edges: edge in graph to descendants • Cross edges: none of the above b f e a Backedge g c d h

Objectives • To introduce the decrease-and-conquer mind set • To show a variety of decrease-and-conquer solutions: • Depth-First Graph Traversal • Breadth-First Graph Traversal • Fake-Coin Problem • Interpolation Search • To discuss the strengths and weaknesses of a decrease-and-conquer strategy

The Fake-Coin Problem: Decrease by a Constant Factor • Problem: • Among n identical looking coins, one is a fake (and weighs less) • We have a balance scale which can compare any two sets of coins • Algorithm: • Divide into two size n/2 piles (keeping a coin aside if n is odd) • If they weigh the same then the extra coin is fake • Otherwise proceed recursively with the lighter pile • Efficiency: • W(n) = W(n/2 ) + 1 for n > 1 • W(n) = log2 n = (log2 n) • But there is a better (log3 n) algorithm

Euclid’s GCD: Variable-Size Decrease • Problem: • Greatest Common Divisor of two integers m and n is the largest integer that divides both exactly • Alternative Solutions: • Consecutive integer checking (brute force) • Identify common prime factors (transform and conquer) • Euclid’s Solution: • gcd(m, n) = gcd(n, m mod n) • gcd(m, 0) = m • Right-side args are smaller by neither a constant size nor factor • Example: • gcd(60, 24) = gcd(24, 12) = gcd(12, 0) = 12

Interpolation Search: Variable-Size Decrease • Mimics the way humans search through a phone book (look near the beginning for ‘Brown’) • Assumes that values between the leftmost (A[l]) and rightmost (A[r]) elements increase linearly • Algorithm (key = v, find search index = i): • Binary search with floating variable at index i • Setup straight line through (l, A[l]) and (r, A[r]) • Find point P = (x, y) on line at y = v, then i = x • Efficiency: • Average = (log log n + 1) • Worst = (n) value A[r] v A[l] index l i r

Strengths and Weaknesses of Decrease-and-Conquer • Strengths: • Can be implemented either top down (recursively) or bottom up (without recursion) • Often very efficient (possibly (log n) ) • Leads to a powerful form of graph traversal (Breadth and Depth First Search) • Weaknesses: • Less widely applicable (especially decrease by a constant factor)

Chap 5: Decrease & conquer