Introduction to Computer Science I Topic 6: Generative Recursion

Introduction to Computer Science ITopic 6: Generative Recursion Prof. Dr. Max Mühlhäuser Dr. Guido Rößling

Outline • Introduction to generative recursion • Sorting with generative recursion • Guidelines for the design of generative recursive procedures • Structural versus generative recursion • Backtracking: Traversing graphs

Generative Recursion • So far, we have used structural recursion to process structurally recursive data • We have decomposed the data to their immediate structural components • We have processed these components and combined the results • However: • Not all problems can be solved using structurally recursive functions • Even if there is a structurally recursive solution for a problem, it might not be optimal • Now we will consider a new programming style: Generative recursion

Generative Recursion • Divide and Conqueris an important principle underlying generative recursion • If the problem is trivial to solve, the solution for the trivial case is returned • Otherwise: • Divide the problem in new smaller problems (generate new sub-problems) • Conquer the sub-problems by applying the same technique recursively • Combine the solutions for the sub-problems into a solution for the original problem • The design of generative recursive programs is more an ad-hoc activity as compared to the design of the structural recursive programs that needs an insight – a “Eureka!"

Modeling a rolling ball on a table Task Description: • The ball rolls at a constant speed until it drops off the edge of the table • We can model the table as a canvas/surface with a pre-defined length and width • The ball can be represented as a disc that moves across the canvas • The disc movement can be represented by repeating the following steps: • Draw the disc at the current position on the canvas • Wait for a certain pre-defined time • Erase the disc at the current position • Move it to a new position

Ball Structure and Operations ;;TeachPack: draw.ss ;; structure: (make-ball numbernumbernumbernumber) (define-structball (x y delta-x delta-y)) ;; draw-and-clear : a-ball -> true (define (draw-and-clear a-ball) (and (draw-solid-disk (make-posn (ball-x a-ball) (ball-y a-ball)) 5 'red) (sleep-for-a-while DELAY) (clear-solid-disk (make-posn (ball-x a-ball) (ball-y a-ball)) 5 'red)))

Ball Structure and Operations ;; move-ball : ball -> ball (define (move-ball a-ball) (make-ball (+ (ball-x a-ball) (ball-delta-x a-ball)) (+ (ball-y a-ball) (ball-delta-y a-ball)) (ball-delta-x a-ball) (ball-delta-y a-ball))) ;; Dimension ofsurface (defineWIDTH100) (defineHEIGHT100) ;; Delay constant (defineDELAY.1)

To move the ball multiple times we can write: This gets tedious after a while. We need a function that moves the ball until it is out of bounds. Ball Structure and Operations (definethe-ball (make-ball10 20 -5 +17)) (and (draw-and-clear the-ball) (and (draw-and-clear (move-ball the-ball)) ...))

Rolling the Ball Determine whether a-ball is outside of the bounds: Template for moving the ball until it is out of bounds: ;; out-of-bounds? : a-ball -> boolean (define (out-of-bounds? a-ball) (not (and (<=0 (ball-x a-ball) WIDTH) (<=0 (ball-y a-ball) HEIGHT)))) The trivial case: returntrue ;; move-until-out : a-ball -> true (define (move-until-out a-ball) (cond [(out-of-bounds? a-ball) ... ] [else...])) true ?

Rolling the Ball After drawing and moving the ball, we apply move-until-outagain, which means the function is recursive: (define (move-until-out a-ball) (cond [(out-of-bounds? a-ball) true] [else (and (draw-and-clear a-ball) (move-until-out (move-ball a-ball)))])) We can now test the function as follows: The code creates a canvas of proper size and a ball that moves to the bottom left of the canvas. (startWIDTHHEIGHT) (move-until-out (make-ball1020-5+17)) (stop)

A New Type of Recursion • The procedure move-until-out uses a new type of recursion: • Conditions have nothing to do with the input data • Recursive application in the body does not use part of the input • move-until-outgenerates an entirely new and different ball structure and uses it for the recursion We do not have a design recipe for this (define (move-until-out a-ball) (cond [(out-of-bounds? a-ball) true] [else (and (draw-and-clear a-ball) (move-until-out (move-ball a-ball)))]))

Sorting: Quicksort and Mergesort • We are once more concerned with sorting the elements of a list … • We have seen insertion sort: • A structurally recursive procedure • Now we will see two other algorithms for sorting: Quicksort and Mergesort • Classic examples of generative recursion • Based on the idea of “divide and conquer”

[Reminder: insertionsort] ;; insertion-sort: list-of-numbers -> list-of-numbers ;; creates a sortedlistofnumb. fromnumbers in alon (define (insertion-sortalon) (cond [(empty? alon) empty] [else (insert (firstalon) (insertion-sort (restalon)))])) unsorted sorted an an

Quicksort: The Idea • An arbitrary intermediate step: sorting an arbitrary sub-list:L0=(list elp…elr) • Divide: PartitionL0 in two (possibly empty) lists, L1=(list elp…elq-1) and L2=(list elq+1…elr), such that • each element of L1 is smaller than elq, and • the latter is in turn smaller than each element in L2 • Conquer: Apply the same procedure recursively to sort L1 and L2 • Combine: simply concatenate the sorted lists L1 and L2 Turningpoint elq <= elq > elq

Quicksort: The Idea • Two open questions so far: • How do we select the pivot element? • We always use the first element • When do we stop? That is: what is the trivial case of Quicksort? • The empty list is always sorted!

Quicksort: Approach • Select thefirstelementfromthelistaspivot item • Determinethefirst sub-listwithelements <= pivot • Sortthis sub-listrecursivelyusingQuicksort • Determinethesecond sub-listwithelements > pivot • Sortthis sub-listrecursivelyusingQuicksort • Concatenatethesorted sub-liststo a newlist

Sort(list 11 8 7 14): Select thefirstelementfrom '(11 8 7 14) aspivot item:11 Determinethefirst sub-list <= 11: '(8 7) Sortthis sub-list Select thefirstelementfrom '(8 7) aspivot item: 8 Determinethefirst sub-listwithelements <= 8: '(7) Sortthis sub-list Select thefirstelementfrom '(7) aspivot item: 7 Determinethefirst sub-listwithelements <= 7: empty Sortthis sub-list Resultempty Determinethesecond sub-listwithelements > 7: empty Sortthis sub-list Resultempty Concatenate (empty 7 empty) to a newlist -> (list 7) Determinethesecond sub-listwithelements > 8:  empty Sortthis sub-list  Resultempty Concatenate((list 7) 8 empty) to a newlist  (list 7 8) Determine… Quicksort @ Work

QuicksortatWork 11 (list 14) (list 8 7) empty 14 empty 8 empty (list 7) empty 7 empty (list 14) (list 7) (list 7 8) (list 7 8 11 14)

Quicksortschematically pivot item

quick-sort distinguishes two cases: If the input is empty, it returns empty. Otherwise, it performs recursion. Each sub-list is sorted separately using quick-sort The sorted versions of the two lists are then combined using append Quicksort Algorithm ;; quicksort2: (listofnumber) -> (listofnumber) (define (quicksort2 alon) (cond [(empty? alon) empty] [else (append (quicksort2 (less-or-equal (restalon) (firstalon))) (list (firstalon)) (quicksort2 (greater-than (restalon) (firstalon)))) ]))

Auxiliary Functions of Quicksort • greater-than filters out the items that are larger than threshold: • less-than filters out the items that are smaller than threshold: (define (greater-than alonthreshold) (filter1 > alon threshold))) (define (less-than alonthreshold) (filter1 < alon threshold)))

Quicksort Evaluation Example (quick-sort (list11 8 14 7)) = (append (quick-sort (list8 7)) (list11) (quick-sort (list14))) = (append (append (quick-sort (list7)) (list8) (quick-sortempty)) (list 11) (quick-sort (list 14))) = (append (append (append (quick-sortempty) (list7) (quick-sortempty)) (list 8) (quick-sortempty)) (list 11) (quick-sort (list 14))) = ...

Quicksort Evaluation Example = (append (append (appendempty (list7) empty) (list 8) empty) (list 11) (quick-sort (list 14))) = (append (append (list7) (list8) empty) (list 11) (quick-sort (list 14))) = (append (list78) (list11) (quick-sort (list14))) = ...

mergesort:The Idea Idea: split the list in the middle apply the function to the sub-lists recursively merge the two sorted lists into a new sorted list

Merging two ordered lists • Given two ordered lists ls1 and ls2 • How do we merge them into a new ordered list? ls-1 ls-2 1 2 4 7 3 5 6 8 Compareandcopythesmallerelement, thencontinue sorted-list 1 2 3 4 5 6 7 8

Merging two ordered lists (define (merge ls1 ls2) (cond [(empty? ls1) ls2] [(empty? ls2) ls1] [(<= (first ls1) (first ls2)) (cons (first ls1) (merge (rest ls1) ls2)) ] [else (cons (first ls2) (merge ls1 (rest ls2)))] ) )

mergesort:algorithm in Scheme (define (merge-sortalon) (local ((define (merge-stepleftright) (cond [(>= leftright) alon] [else (local( (definemid (floor (/ (+ leftright) 2))) (defineleft-list (merge-sort (extractalonleftmid))) (defineright-list (merge-sort (extractalon (+ mid 1) right)))) (mergeleft-listright-list) ) ] ))) (merge-step 1 (lengthalon))))

mergesort: algorithm in Scheme (define (extract alonleft right) (cond [(empty? alon) empty] [(> left right) empty] [(> left 1) (extract (rest alon) (- left 1) (- right 1))] [else (cons (first alon) (extract alon (+ left 1) right))]))

Guidelines for DesigningGenerative Recursive Procedures • Understand the nature of the data of the procedure • Describe the process in terms of data, creating a new structure or partitioning a list of numbers • Distinguish between those input data • which can be processed trivially, • and those which cannot • The generation of problems is the key to algorithm design • The solutions of the generated problems must be combined

A Return to the Six Stages of Structural Design • Data analysis and design: • Analyze and define data collections representing the problem • Contract, purpose, header • Specify what the function does • Explain in general terms how it works • Function examples: • Illustrate how the algorithm proceeds for some given input • Template: • Follow a general template • Definition: • Answer the questions posed by the template • Test • Test the completed function • Eliminate the bugs

General Template forGenerative Recursive Functions (define (generative-recursive-funproblem) (cond [(trivially-solvable? problem) (determine-solutionproblem)] [else (combine-solutions ... problem ... (generative-recursive-fun (generate-problem-1 problem)) ... (generative-recursive-fun (generate-problem-n problem)))]))

Procedure Definition • What is a trivially solvable problem, and the pertaining solution? • How do we generate new problems that are easier to solve than the initial problem? • Is there one new problem to generate, or are there more? • Is the solution for the given problem the same as for (one of) the new problems? • Or do we have to combine solutions to create a solution for the initial problem? • If so, do we require parts of the data constituting the initial problem?

Termination of Structurally Recursive Procedures • So far, each function has produced an output for a valid input  Evaluation of structurally recursive procedures has always terminated. • Important characteristic of the recipe for structurally recursive procedures: • Each step of natural recursion uses an immediate component of the input, not the input itself • Because data is constructed in a hierarchical manner, the input shrinks at every stage • The function sooner or later consumes an atomic piece of data and terminates.

Termination of Generative Recursive Procedures • This characteristic is not true for generative recursive functions. • The internal recursions do not consume an immediate component of the input, but some new piece of data, which is generated from the input. • This step may produce the initial input over and over again and thus prevent the evaluation from ever producing a result • We say that the program is trapped in an infinite loop.

Non-Terminating Procedures • What happens if we place the following three expressions at the bottom of the DrScheme Definitions window and click execute? • Does the second expression ever produce a value so that the third expression is evaluated and the canvas disappears? (startWIDTHHEIGHT) (move-until-out (make-ball102000)) (stop)

Non-TerminatingProcedures The slightest mistake in process definition may cause an infinite loop: ;; less-or-equal: (list-of-numbers) number-> (list-of-numbers) (define(less-or-equalalonthreshold) (cond [(empty?alon) empty] [else (if (<= (firstalon) threshold) (cons (firstalon) (less-or-equalalonthreshold)) (less-or-equal(restalon) threshold))])) Insteadof(restalon) Quicksortdoes not terminatewiththenewfunction (quick-sort (list 5)) = (append (quicksort (less-or-equal5 (list 5))) (list 5) (quicksort (greater-than5 (list 5)))) = (append (quicksort(list 5)) (list 5) (quicksort (greater-than5 (list 5))))

Termination Argument • The termination argument is an additional step in the design recipe of generative recursive procedures • Termination argument explains • why the process produces an output for every input • how the function implements this idea • when the process possibly does not terminate

Termination Argument for Quicksort At each step, quick-sortpartitions the list into two sublists using less-or-equaland greater-than. Each function produces a list that is smaller than the input (the second argument), even if the pivot element (the first argument) is an item on the list. Hence, each recursive application of quick-sort consumes a strictly shorter listthan the given one. Eventually, quick-sortreceives an empty list and returns empty.

New Termination Cases Termination argument may reveal additional termination cases. This knowledge can be added to the algorithm: So liefert (less-or-equal N (list N))und(greater-than N (list N)) immer empty ;; quick-sort : (listofnumber) -> (listofnumber) (define (quick-sortalon) (cond [(empty? alon) empty] [(empty? (restalon)) alon] [else (append (quick-sort(less-or-equal (restalon) (firstalon))) (list (firstalon)) (quick-sort(greater-thanalon(firstalon)))) ]))

Guidelines for Algorithm Design

trivially-solvable? empty? generate-problem rest Structural Recursion as aSpecial Case of Generative Recursion Template forgenerative recursion (define (generative-recursive-funproblem) (cond [(trivially-solvable? problem) (determine-solutionproblem)] [else (combine-solutions problem (generative-recursive-fun (generate-problemproblem)))])) Template forlistprocessing (define (generative-recursive-funproblem) (cond [(empty?problem) (determine-solutionproblem)] [else (combine-solutions problem (generative-recursive-fun (restproblem)))]))

Structural vs. Generative Recursion • Is there a difference between structural and generative recursion? • Structurally recursive functions seem to be just special cases of generative recursion • But this "it's all the same" attitude does not improve the understanding of the design process • Structurally and generative recursive functions are designed using different approaches and have different consequences.

Greatest Common Denominator (GCD) • Examples • 6 and 25 are both numbers with several denominators: • 6 is evenly divisible by 1, 2, 3, and 6; • 25 is evenly divisible by 1, 5, and 25. • The greatest common denominator of 25 and 6 is 1. • 18 and 24 have many common denominators : • 18 is evenly divisible by 1, 2, 3, 6, 9, and 18; • 24 is evenly divisible by 1, 2, 3, 4, 6, 8, 12, and 24. • The greatest common denominator is 6.

GCD Based on Structural Recursion Test for every number i = [min(n,m) .. 1] whether it divides both n and m evenly and return the first such number. ;; gcd-structural : N[>= 1] N[>= 1] -> N ;; structuralrecursionusingdatadefinitionof N[>= 1] (define (gcd-structural n m) (local ((define (first-divisor i) (cond [(= i 1) 1] [(and(= (remainder n i) 0) (= (remainder m i) 0)) i] [else (first-divisor (- i 1))] ) ) ) (first-divisor (min m n)))) Inefficientfor large numbers!

Analysis of Structural Recursion • gcd-structural simply tests every number whether it divides both n and m evenly and returns the first such number • For small natural numbers, this process works just fine • However, for (gcd-structural 101135853 450146405) = 177 the procedure will compare 101135853 – 177 = 101135676 numbers! • Even reasonably fast computers spend several minutes on this task. • Enter the definition of gcd-structural into the Definitions window and evaluate the following expression… in the Interactions window … and go get a coffee  (time (gcd-structural 101135853 450146405))

GCD Euclidean Algorithm • The Euclidean algorithm to determine the greatest common denominator (GCD) of two integers is one of the oldest algorithms known • Appeared inEuclid’s Elements around 300 BC • However, the algorithm probably was not discovered by Euclid and it may have been known up to 200 years earlier. Insight: For two natural numbersnandm, n > m, GCD(n, m) = GCD(m, remainder(n,m)) (gcd larger smaller) = (gcdsmaller (remainder larger smaller)) Example: GCD(18, 24) = GCD(18,remainder(24/18)) = GCD(18,6) = GCD(6,0) = 6

GCD Generative Algorithm ;; gcd-generative : N[>= 1] N[>=1] -> N (define (gcd-generative n m) (local ((define (clever-gcd larger smaller) (cond [(= smaller 0) larger] [else (clever-gcd smaller (remainder larger smaller))])) ) (clever-gcd (max m n) (min m n)))) clever-gcdis based on generative recursion: • The trivially solvable case is when smaller is 0 • The generative step calls clever-gcd with smaller and (remainder larger smaller) (gcd-generative101135853450146405)needsonly9 iterations!

Introduction to Computer Science I Topic 6: Generative Recursion