1 / 37

Analysis of Algorithms Chapter - 01 Introduction

Analysis of Algorithms Chapter - 01 Introduction. This Chapter Contains the following Topics: Syllabus & Marks Distribution Study of Algorithms Algorithm Study of Algorithms Pseudocode Conventions Recursive Algorithms Sorting Problem Growth of Functions Objective

zacharyv
Download Presentation

Analysis of Algorithms Chapter - 01 Introduction

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript


  1. Analysis of Algorithms Chapter - 01 Introduction

  2. This Chapter Contains the following Topics: • Syllabus & Marks Distribution • Study of Algorithms • Algorithm • Study of Algorithms • Pseudocode Conventions • Recursive Algorithms • Sorting Problem • Growth of Functions • Objective • Example of Function Growth • Complexity of Algorithm • Asymptotic Notations

  3. Syllabus & Marks Distribution

  4. Tentative Syllabus & Text Book • Introduction, Asymptotic Notations • Recurrences • Sorting Algorithms • Divide & Conquer • Binary search tree • Dynamic Programming • Greedy Graph Algorithms • Minimum spanning tree • Data Compression • Binomial heap • Fibonacci heap Text Books • Introduction to Algorithms, 2nd edition by Corman, Rivest,Leiserson, and Stein. • Computer Algorithms: Introduction to Design and Analysis By Sara Baase, 3rd Edition, Addison-Wesley Publishing Company

  5. Grading Note: This is only a guide, percentages and rules may be changed during the semester as needed. Percentages: The final grade will be composed of the following Four components: ⋆ 50% : Final exam. ⋆ 20% : First exam. ⋆ 20% : Second exam. ⋆ 10% : Assignments and Quizzes.

  6. Study of Algorithms

  7. Algorithm • The word ‘algorithm’ comes from the name of a Persian author, Abu Ja’far Mohammad Ibn Musa Al Khowarizmi (825 AD). • Definition: An algorithm is a finite set of instructions that, if followed, carries out a particular task. In addition, all algorithms must satisfy the following criteria: • Input: Zero or more quantities are externally supplied. • Output: At least one quantity is produced. • Definiteness: Each instruction is clear and unambiguous. • Finiteness: If we trace out the instructions of an algorithm, then for all cases, the algorithm terminates after a finite number of steps. • Effectiveness: Every instruction must be very basic so that it can be carried out, in principle, by a person using only pen and paper. It must be feasible.

  8. Study of Algorithms • The study of algorithms includes many important and active areas of research. • Given a problem, how do we find an efficient algorithm for its solution? • Once we have found an algorithm, how can we compare this algorithm with other algorithms that solve the same problem? • How should we judge the goodness of an algorithm? • About the study of algorithms, most researchers agree that there are four distinct areas of study one can identify. • How to devise algorithms? • How to validate algorithms? • How to test a program? • How to analyze algorithms?

  9. Study of Algorithms (Contd.) • How to devise algorithms? • Creating an algorithm is an art which may never be fully automated. • How to validate algorithms? • Once an algorithm is devised, it is necessary to show that it computes the correct answer for all possible legal inputs. • A program can be written and then be verified. • How to test a program? • Debugging is the process of executing programs on sample data sets to determine whether the faulty results occur and, if so, to correct them. • Profiling is the process of executing a correct program on data sets and measuring the time and space it takes to compute the results.

  10. Study of Algorithms (Contd.) • How to analyze algorithms? • Analysis of algorithms refers to the task of determining how much computing time and storage an algorithm requires. • It allows you to make quantitative judgments about the value of one algorithm over another. • It allows you to predict whether the software will meet any efficiency constraints that exist. • Questions such as how well does an algorithm perform in the best, worst and average cases. • Theoretical analysis: • All possible inputs. • Independent of hardware/ software implementation • Experimental Study: • Some typical inputs. • Depends on hardware/ software implementation. • A real program.

  11. Pseudocode Conventions • Comments begin with // and continue until the end of line. • Blocks are indicated with matching braces: {and}. • An Identifier begins with a letter: max. • Assignment of values to variables is done using assignment statement : Variable:=expression. • Logical operators: and, or and not are provided. • Relational operators: <, ≤, =, ≠, ≥ and > provided. • Elements of arrays are accessed using: [ and ]. • While loop: while (condition) do { (statement 1) ……………… (statement n) } • For loop: for variable:=value1 to value2 step step do { (statement 1) ……………… (statement n) }

  12. Pseudocode Conventions (Contd.) • Repeat-until loop: repeat { (statement 1) ………… (statement n) } until (condition) • Conditional statement: • if (condition) then (statement) • if (condition) then (statement 1) else (statement 2) • Case statement: case { : (condition 1): (statement 1) …………………. : (condition n): (statement n) : else: (statement n+1) } • Input and output are done using: read and write. • There is only one type of procedure: Algorithm. • An algorithm consists of a heading and a body. • The heading of an algorithm takes the form Algorithm Name ((parameter list))

  13. Recursive Algorithms • A recursive function is a function that is defined in terms of itself. • Similarly, an algorithm is said to recursive if the same algorithm is invoked in the body. • An algorithm that calls itself is direct recursive. • Algorithm Ais said to be indirect recursive if it calls another algorithm which in turn calls A. Algorithm TowersOfHanoi (n, x, y, z) // Move the top n disks from tower x to tower y. { if (n ≥ 1) then { TowersOfHanoi (n-1, x, z, y); write (“Move top disk from tower”, x, “to top of tower”, y); TowersOfHanoi (n-1, z, y, x); } } • Illustrate the above algorithm with an example.

  14. Sorting Problem • Input: A sequence of n numbers (a1, a2,…, an). • Output: A permutation of n numbers (reordering) (a'1, a'2,…, a'n) of the input sequence such that a'1≤ a'2 ≤… ≤ a'n. • Solve the problem using Insertion sort. • Insert an element to a sorted array such that the order of the resultant array be not changed. Algorithm InsertionSort (A, n) { for i:=2 ton do { key:=A[i]; // Insert A[i] into the sorted sequence A[1…i-1]. j:=i-1; while ( (j>0) and (A[j]>key) ) do { A[j+1]:=A[j]; j:=j-1; } A[j+1]:=key; } } • Illustrate the above algorithm with an example.

  15. Loop Invariant and Correctness • At the start of each iteration of the ‘for loop’, the sub-array A[1…i-1] consists of the elements originally in A[1…i-1] but in sorted order. • We use Loop invariants to help us understand why an algorithm is correct. We must show three things about a loop invariants: • Initialization: It is true prior to the first iteration of the loop. • Maintenance: if it is true before an iteration of the loop, it remains true before the next iteration. • Termination: When the loop terminates, the invariant gives us a useful property that helps show that the algorithm is correct. • Let us see how these properties hold for insertion sort: • Initialization: Before 1st loop iteration, when i=2, the subarray A[1…i-1] consists of single element A[1], which is already sorted. • Maintenance: The body of outer ‘for loop’ works by moving A[i-1], A[i-2], A[i-3], and so on by one position to the right until the proper position for A[i] is found. • Termination: When i=n+1, the subarray A[1…n] consists of the elements originally in A[1…n], but in sorted order.

  16. Analysis of Insertion Sort T(n) = c1n + c2(n-1) + c3(n-1) + c4Σti + c5 Σ(ti -1) + c6 Σ(ti -1) + c7(n-1) Best Case: ti=1, for i=2,3,…..n T(n) = c1n + c2(n-1) + c3(n-1) + c4(n-1) + c7(n-1) = (c1+c2+c3+c4+c7)n – (c2+c3+c4+c7) So, T(n)=an+b Worst Case: ti = i, for i=2,3,……,n. Σi= n(n+1)/2 – 1 Σ(i-1) = n(n-1)/2 T(n)= c1n + c2(n-1) + c3(n-1) + c4(n(n+1)/2 – 1) + c5 (n(n-1)/2) + c6 (n(n-1)/2) + c7(n-1) = (c4/2 + c5/2 + c6/2)n2 + (c1+c2+c3+c4/2-c5/2-c6/2+c7)n - (c2+c3+c4+c7) So, T(n)=an2+bn+c

  17. Growth of Functions

  18. Objective • Objective: A language to express that Algorithm A is better or worse or equivalent to Algorithm B. • Need to define “≤” between functions measuring the growth of functions. • Independence of the hardware/ software environment: Turing machine, RAM machine, classroom model, today computers, and future super-computers. • Ignore constants that can be affected by changing the environment.

  19. Examples of Function Growth • Maximum size of a problem that can be solved in one second, one minute, and one hour, given below: • Increase in the maximum size of a problem by using a computer that is 256 times faster than the previous one. • Each entry is given as a function of m, the previous maximum problem size.

  20. Complexity of Algorithm • How much of resources the algorithm requires? • Usually: time and space (memory). • Complexity: as a function of the input length n. • Usually an integer n > 0. • Usually a monotonic non-decreasing function T(n). • The limiting behavior of the complexity as size increases is called the asymptotic complexity of the algorithm. Worst Case and Average Case Complexity • T(n) is a worst case complexity: • If for all inputs of length n the complexity is T(n). • T(n) is an averagecase complexity: • If the average complexity over all length n inputs is T(n). • Averaging following some distribution of the inputs. • Usually the uniform distribution.

  21. Asymptotic Notations • Big-Oh:f(n) = O(g(n)) if f(n)asymptotically less than or equal tog(n). • Big-Omega:f(n) = ῼ(g(n)) if f(n)asymptotically greater than or equal tog(n). • Big-Theta:f(n) = ⊖(g(n)) if f(n)asymptotically equal tog(n). • Little-oh:f(n) = o(g(n)) if f(n)asymptotically strictly less thang(n). • Little-omega:f(n) = ω(g(n)) if f(n)asymptotically strictly greater thang(n).

  22. Big-Oh, Big-Omega, Big-Theta • Big-Oh:: f(n) = O(g(n)), if • There exists a real constant c > 0 and an integer constant n0 > 0 such that f(n) ≤ cg(n) for every integer n ≥ n0. • Big-Omega:: f(n) = ῼ(g(n)), if • There exists a real constant c > 0 and an integer constant n0 > 0 such that f(n) ≥ cg(n) for every integer n ≥ n0. • Big-Theta:: f(n) = ⊖(g(n)), if • Thereexist two real constants c′, c′′ > 0 and an integer constant n0 > 0 such that c′′g(n) ≤ f(n) ≤ c′g(n) for every integer n ≥ n0.

  23. Big-Oh

  24. Big-Omega

  25. Big-Theta

  26. Examples • 3n + 2 = O(n) As 3n + 2 ≤ 4n, for all n≥2. • 100n + 6 = O(n) As 100n + 6 ≤ 101n, for all n≥ 6. • 10n2 + 4n + 2 = O(n2) As 10n2 + 4n + 2 ≤ 11n2, for all n≥ 5. • 1000n2 + 100n - 6 = O(n2) As 1000n2 + 100n - 6 ≤ 1001n2, for all n≥100. • 6*2n + n2 =O(2n) As 6*2n + n2 ≤ 7*2n, for all n≥4. • 3n + 2 = O(n2) 3n + 2 ≤3n2, for all n≥2. • But 3n + 2 ≠ O(1) and 10n2 + 4n + 2 ≠ O(n). • 3n + 2 = Ω(n) As 3n + 2 ≥ 3n, for all n≥1. • 10n2 + 4n + 2 = Ω (n2) As 10n2 + 4n + 2 ≥ n2, for all n ≥ 1. • 6*2n + n2 = Ω(2n) As 6*2n + n2 ≥ 2n, for all n≥1. • 3n + 2 = Θ(n) 3n ≤ 3n + 2 ≤ 4n, for all n ≥2. • 10n2 + 4n + 2 = Θ(n2) • 6*2n + n2 = Θ(2n)

  27. Little-oh and Little-omega • f(n) = o(g(n)) if limn→∞f(n)/g(n) = 0: • For any constant c > 0 there exists an integer constant n0 > 0 such that f(n) < cg(n) for every integer n ≥ n0. • f(n) = ω(g(n)) if limn→∞f(n)/g(n) = ∞: • For any constant c > 0 there exists an integer constant n0 > 0 such that f(n) > cg(n) for every integer n ≥ n0.

  28. Hierarchy of Functions

  29. Simplifying the Analysis

  30. Order Notation

  31. Algorithm Analysis: Example • Alg.: MIN (a[1], …, a[n]) m ← a[1]; for i ← 2 to n if a[i] < m then m ← a[i]; • Running time: • the number of primitive operations (steps) executed before termination T(n) =1 [first step] + (n) [for loop] + (n-1) [if condition] + (n-1) [the assignment in then] = 3n - 1 • Order (rate) of growth: • The leading term of the formula • Expresses the asymptotic behavior of the algorithm

  32. Extra Examples • 2n2 = O(n3): • n2 = O(n2): n = O(n2): 2n2≤ cn3  2 ≤ cn  c = 1 and n0= 2 n2≤ cn2  c ≥ 1  c = 1 and n0= 1 n ≤ cn2  cn ≥ 1  c = 1 and n0= 1

  33. Examples • 100n + 5 ≠ (n2) For the above question, we had a solution proposed as follows to prove 100n + 5 = (n2) :- cn2< 100n + 5 Let no = 100 . That gives us, c x 10000 < 100 x 100 + 5 c < 10005 / 10000 So the contant c = 1.0005 But does this work when c = 1.0005 and n = 200 ( remember, it should satisfy for all n > no . Here n is 200 and no is 100) No, it doesn’t work. Hence you cannot prove that 100n + 5 = (n2)

  34. Examples • Prove that 100n + 5 = O(n2) • 100n + 5 ≤ 100n + n = 101n ≤ 101n2 for all n ≥ 5 n0 = 5 and c = 101is a solution • 100n + 5 ≤ 100n + 5n = 105n ≤ 105n2for all n ≥ 1 n0 = 1 and c = 105is also a solution Must findSOMEconstants c and n0 that satisfy the asymptotic notation relation

  35. Nested Dependent Loop for i =1 to n do for j = i to n do sum=sum+1

More Related