CITS1401 Problem Solving and Programming

1. CITS1401 Problem Solving and Programming Semester 1, 2014A/Prof Lyndon WhileSchool of Computer Science & Software EngineeringThe University of Western Australia Problem-solving Techniques

2. CITS1401 • CITS1401 covers • many important problem-solving techniques used widely in Computer Science and in programming • writing basic programs in Python, a modern high-level programming language • an introduction to software engineering • Problem-solving techniques have been covered mostly via lab work • in this lecture we will review the techniques covered CITS1401 problem-solving techniques

3. Techniques discussed Morphological analysis and testing Reduction and analogy Enumeration and search Abstraction Divide-and-conquer Backtracking CITS1401 problem-solving techniques 3

4. Morphological analysis and testing • Lab 2 • “morphology” is the study of patterns or forms • occurs widely in many branches of science • In CS/SE/IT, it occurs mostly in two contexts • classification of inputs for processing • classification of inputs for testing CITS1401 problem-solving techniques

5. Classification of inputs for processing • A term in a polynomial is defined by its coefficient and its exponent • what if we want to turn the term into a string? • In the “easy case”: (–6, 4) → “–6x^4” CITS1401 problem-solving techniques

6. Classification of inputs for processing • But what if the coefficient == 0? (0, 4) → “0” > 0? (6, 4) → “6x^4” == 1? (1, 4) → “x^4” == –1? (–1, 4) → “–x^4” • And what if the exponent == 0? (–6, 0) → “–6” == 1? (–6, 1) → “–6x” < 0? (–6, –4) → “–6/x^4” == –1? (–6, –1) → “–6/x” CITS1401 problem-solving techniques

7. Classification of inputs for testing • In optional preferential voting, the voter can rank any subset of the candidates, from one of them up to all of them • so any sequence of non-repeating integers increasing from 1 is a valid vote • If we write a function to parse OPV votes, what would be a good set of test data? • Assume an election with three candidates • a vote is represented by a string containing three characters CITS1401 problem-solving techniques

8. Testing example: classifying votes • Intentionally ranking all three candidates: • i.e. permutations of 123 • 6 of these • 123, 132, 213, 231, 312, 321 • Intentionally ranking only two candidates: • i.e. permutations of 12<sp> • 6 of these • 12<sp>, 1<sp>2, <sp>12, 21<sp>, 2<sp>1, <sp>21 CITS1401 problem-solving techniques

9. Testing example: classifying votes contd. • Intentionally ranking only one candidate: • i.e. a 1 with two spaces • 3 of these • 1<sp><sp>, <sp>1<sp>, <sp><sp>1 • Accidentally ranking only two candidates: • i.e. some non-space instead of the 3 • permutations of 124? • 6 of these • 124, 142, 214, 241, 412, 421 CITS1401 problem-solving techniques

10. Testing example: classifying votes contd. • Accidentally ranking only one candidate: • omitting the 2 • 6 permutations of 134? • duplicating the 2 • 3 permutations of 122 • Accidentally ranking no candidates: • omitting the 1 • 6 permutations of 234? • replicating the 1 • 4 possibilities: 11<sp>, 1<sp>1, <sp>11, 111 CITS1401 problem-solving techniques

11. Testing example: classifying votes contd. • All spaces: • <sp><sp><sp> • Over-length: • anything with more than three characters • Under-length: • anything with fewer than three characters • Already we have 43 test cases! • Every time we change the function, we should re-run all tests • clearly we need a testing program! CITS1401 problem-solving techniques

12. Reduction and analogy • Lab 3 • Reduction is solving a new problem by converting it into another problem for which we already have a solution • e.g. the problem of finding your way around an unknown city can be reduced to the problem of finding a map of the city • assuming you can read maps! • That problem can be reduced to the problem of finding a shop that sells maps • which can be reduced to the problem of reading the information at the airport… CITS1401 problem-solving techniques

13. Reduction example: building tables Assume that we have written a function buildEvenTablethat works for even n Now we need to write the function buildTablethat works for all n It would be hugemistake to duplicate the code Instead define buildTable by reducing the problem to buildEvenTable plus stripDummy and some other logic CITS1401 problem-solving techniques 13

14. Reduction example: running programs Imagine we have a Python interpreter that can run while-loops Then someone says “Let’s add for-loops to Python” We could change the interpreter but that might be a lot of work, especially if it was originally written by someone else Or we could use reduction replace each for-loop with an equivalent while-loop k = 0 while k < n: <statements> k += 1 for k in range(n): <statements> becomes CITS1401 problem-solving techniques 14

15. Then someone says “Let’s add list comprehensions to Python” Use reduction again: replace each list comprehension with an equivalent for-loop Note the hierarchical approach Reduction example: programs contd. zs = [] for k in range(n): if p(k): zs.append(f(x)) zs = [f(x) for k in range(n) if p(k)] becomes CITS1401 problem-solving techniques 15

16. Enumeration and search • Lab 4 • Very simple idea: generate all possible solutions to the problem, and then check each one to see if it’s correct/good • Used widely in • cryptography • artificial intelligence • game playing CITS1401 problem-solving techniques

17. Enumeration example: verbal arithmetic SEND + MORE = MONEY consistently replace each letter with a digit from 0, 1, …, 9 so that the arithmetic is correct By enumeration, we could create all 10 x 9 x … x 3 = 1,814,400 possible assignments of digits to letters, and check each one notionally very easy computationally very expensive CITS1401 problem-solving techniques 17

18. Enumeration issues • Often “all possible solutions” is way too many! • especially if there’s an infinite number of them… • Often advantageous to • rank potential solutions • likely to find a correct/good one sooner • use known correct/good solutions to develop new (improved) possibilities • e.g. hill-climbing algorithms or genetic algorithms • randomness helps sometimes! CITS1401 problem-solving techniques

19. Enumeration example: cryptography You are given a coded English message that you know was derived using a substitution cipher i.e. each letter in the original was consistently replaced by a different letter Using naïve enumeration gives 26 x 25 x … x 1 = 403,291,461,126,605,635,584,000,000 possibilities So use tricks like: ‘e’ probably occurs very often (& ‘a’, ‘r’, ‘t’, ‘n’, etc.) the sequence ‘jx’ probably never occurs (& ‘zq’, etc.) most occurrences of ‘q’ will be followed by a ‘u’ CITS1401 problem-solving techniques 19

20. Enumeration example: missionaries & cannibals On one side of a river are three missionaries, three cannibals, and a canoe that can carry one or two people any time on either side of the river, if the number of cannibals exceeds the number of missionaries, something unpleasant happens Can you come up with a sequence of canoe trips that gets everyone safely across the river? e.g. the first trip could be M crosses: no! MM cross: no! C crosses: ok, but the next trip must be just him coming back CC cross, or MC cross: maybe… CITS1401 problem-solving techniques 20

21. Enumeration example: missionaries & cannibals By enumeration, we could create all possible sequences but we need to check for loops and it’ll be a lot Instead just apply the rule “always maximise the number of people on the far bank” leads almost directly to a solution! these sorts of guidelines are called heuristics CITS1401 problem-solving techniques 21

22. Abstraction • Lab 5 • Abstraction means simplifying a problem as much as possible before solving it • Examples of this principle include • operate on models instead of the “real world” • ignore some details to focus on others • discretise space and/or time (and other dimensions) • prefer simple data reps. (e.g. integers vs. dates) • Abstraction can lead to more-general solutions CITS1401 problem-solving techniques

23. Abstraction examples • Graph problems • entities as nodes, connections as arcs • focus is on the topology • Dates as integers • faster, simpler, and more flexible • Database construction • store only relevant data • but of course “relevant” is context-dependent… CITS1401 problem-solving techniques

24. Abstraction • Einstein’s Constraint: “Everything should be made as simple as possible, but not simpler.” • omit all details that don’t contribute to a solution • allows you to focus on the “important bits” • but don’t omit any important bits! CITS1401 problem-solving techniques

25. Divide-and-conquer • Lab 6 • Divide-and-conquer means: • divide a problem instance into several smaller pieces • solve each of the pieces separately • combine their solutions to solve the original problem • Very widely-used technique, especially for processing data structures • Often leads to very efficient algorithms CITS1401 problem-solving techniques

26. Divide-and-conquer example: mergesort • Given a list of n numbers • [8, 0, 3, 6, 1, 7, 4, 2, 9, 5] • Split the list down the middle • [8, 0, 3, 6, 1] and [7, 4, 2, 9, 5] • Separately sort these two lists • [0, 1, 3, 6, 8] and [2, 4, 5, 7, 9] • Merge the two sorted lists • [0, 1, 2, 3, 4, 5, 6, 7, 8, 9] • need only (repeatedly) compare the heads CITS1401 problem-solving techniques

27. Divide-and-conquer example: quicksort • Given a list of n numbers • [8, 0, 3, 6, 1, 7, 4, 2, 9, 5] • Choose a pivot (say 5) and partition the list • [0, 3, 1, 4, 2] and [8, 6, 7, 9] • elements smaller than the pivot in the first list • Separately sort these two lists • [0, 1, 2, 3, 4] and [6, 7, 8, 9] • Append the two sorted lists and re-insert the pivot • [0, 1, 2, 3, 4, 5, 6, 7, 8, 9] CITS1401 problem-solving techniques

28. Divide-and-conquer issues • Dividing up the data equally gives the best performance • e.g. quicksort • the best pivot leaves two lists with n/2 elements • the worst pivot leaves one list with n–1, plus [ ] • Auxiliary operations should be cheap • e.g. merge, partition • Larger base cases may improve performance • Sometimes identical sub-problems arise CITS1401 problem-solving techniques

29. Backtracking • Lab 7 • Backtracking is a major enhancement to enumeration and search • Enumeration & search: • generate all possible complete solutions, then check each one for correctness • Backtracking: • build up partial solutions bit by bit, checking for correctness at each stage CITS1401 problem-solving techniques

30. Backtracking example: verbal arithmetic • SEND MORE + ----------- MONEY • consistently replace each letter with a digit from 0, 1, …, 9 so that the arithmetic is correct • e.g. D + E = Y • or D + E = Y + 10 • e.g. N + R = E • more generally, N + R [+ 1] = E [+ 10] CITS1401 problem-solving techniques

31. Verbal arithmetic with Enumeration & Search • D = 1, E = 2, Y = 3, N = 4, R = 5, S = 6, M = 7, Y = 8: NO • D = 1, E = 2, Y = 3, N = 4, R = 5, S = 6, M = 7, Y = 9: NO • D = 1, E = 2, Y = 3, N = 4, R = 5, S = 6, M = 7, Y = 0: NO • D = 1, E = 2, Y = 3, N = 4, R = 5, S = 6, M = 8, Y = 7: NO • D = 1, E = 2, Y = 3, N = 4, R = 5, S = 6, M = 8, Y = 9: NO • D = 1, E = 2, Y = 3, N = 4, R = 5, S = 6, M = 8, Y = 0: NO • D = 1, E = 2, Y = 3, N = 4, R = 5, S = 6, M = 9, Y = 7: NO • D = 1, E = 2, Y = 3, N = 4, R = 5, S = 6, M = 9, Y = 8: NO • D = 1, E = 2, Y = 3, N = 4, R = 5, S = 6, M = 9, Y = 0: NO • D = 1, E = 2, Y = 3, N = 4, R = 5, S = 6, M = 0, Y = 7: NO • D = 1, E = 2, Y = 3, N = 4, R = 5, S = 6, M = 0, Y = 8: NO • D = 1, E = 2, Y = 3, N = 4, R = 5, S = 6, M = 0, Y = 9: NO • D = 1, E = 2, Y = 3, N = 4, R = 5, S = 7, M = 6, Y = 8: NO • D = 1, E = 2, Y = 3, N = 4, R = 5, S = 7, M = 6, Y = 9: NO • etc. • 1,814,400 possible solutions to check CITS1401 problem-solving techniques

32. Backtracking example: verbal arithmetic • D = 1 • E = 2 • Y = 0: NO – 2,520 possibilities discarded • Y = 9, 8, 7, 6, 5, 4: NO – 15,120 more discarded • Y = 3 • N = 4 • R = 5: NO – 60 more discarded • etc. • E = 3 • etc. • D = 2 • etc. CITS1401 problem-solving techniques

33. Backtracking issues • Design a representation that allows building, checking, and discarding of partial solutions • Discard partial solutions as early as possible • Incorporate all available clues! CITS1401 problem-solving techniques