Sorting & Searching

1. Sorting & Searching Geletaw S (MSC, MCITP)

2. Objectives At the end of this session the students should be able to: • Design and implement the following simple sorting algorithms: • Bubble sort • Insertion sort • Selection sort • Compare the efficiency of the sorting algorithms in terms of Big-O complexity and space requirements to sort small number of elements. • Discuss the efficiency of the following simple searching algorithms: • Sequential or linear search • Binary search

3. Sorting: an operation that segregates items into groups according to specified criterion. A = { 3 1 6 2 1 3 4 5 9 0 } A = { 0 1 1 2 3 3 4 5 6 9 } What is Sorting?

4. Consider: Sorting Books in Library Sorting Individuals by Height Sorting Movies in Best-seller Sorting Numbers (Sequential) Why Sort and Examples

5. There are many, many different types of sorting algorithms, but the primary ones are: Types of Sorting Algorithms • Bubble Sort • Selection Sort • Insertion Sort • Merge Sort • Shell Sort • Heap Sort • Quick Sort • Radix Sort • Swap Sort

6. Bubble Sort • The simplest algorithm to implement • The slowest algorithm on very large inputs. Basic Idea: • Loop through array from i=0 to n and swap adjacent elements if they are out of order.

7. Pseudo code Set flag = false • Traverse the array and compare pairs of two elements 1.1 If E1  E2 - OK 1.2 If E1 > E2 then Switch(E1, E2) and set flag = true • If flag = true goto 1.

8. Implementation void bubbleSort (Array S, length n) { booleanisSorted = false; while(!isSorted) { isSorted = true; for(i = 0; i<n; i++) { if(S[i] > S[i+1]) { int aux = S[i]; S[i] = S[i+1]; S[i+1] = aux; isSorted = false; } } }

9. Bubble Sort • 1 23 2 56 9 8 10 100 • 1 2 23 56 9 8 10 100 • 1 2 23 9 56 8 10 100 • 1 2 23 9 8 56 10 100 • 1 2 23 9 8 10 56 100 ---- finish the first traversal ---- ---- start again ---- • 1 2 23 9 8 10 56 100 • 1 2 9 23 8 10 56 100 • 1 2 9 8 23 10 56 100 • 1 2 9 8 10 23 56 100 ---- finish the second traversal ---- ---- start again ---- ………………….

10. Analysis of Bubble Sort How many comparisons? • (n-1)+(n-2)+…+1= O(n2) How many swaps? • (n-1)+(n-2)+…+1= O(n2)

11. Selection Sort How does it work: • first find the smallest in the array and exchange it with the element in the first position, • then find the second smallest element and exchange it with the element in the second position, and • continue in this way until the entire array is sorted.

12. Con’t… • Selection sort is: • The simplest sorting techniques. • a good algorithm to sort a small number of elements • an incremental algorithm – induction method • Selection sort is Inefficient for large lists. Incremental algorithms process the input elements one-by-one and maintain the solution for the elements processed so far.

13. Selection Sort Algorithm Input: An array A[1..n] of n elements. Output: A[1..n] sorted in nondecreasing order. 1. for i  1 to n - 1 2. k i 3. for j i + 1 to n {Find the i thsmallest element.} 4. if A[j] < A[k] then k j 5. end for 6. if k i then interchange A[i] and A[k] 7. end for

14. Example • Original array:6 3 5 4 9 2 7 • 1st pass -> 2 3 5 4 9 6 7 (2 and 6 were swapped)2nd pass -> 2 3 4 5 9 6 7 (4 and 5 were swapped)3rd pass -> 2 3 4 5 6 9 7 (6 and 9 were swapped)4th pass -> 2 3 4 5 6 7 9 (7 and 9 were swapped)5th pass -> 2 3 4 5 6 7 9 (no swap)6th pass -> 2 3 4 5 6 7 9 (no swap)

15. Analysis of Algorithms • Algorithm analysis: quantify the performance of the algorithm, i.e., the amount of time and space taken in terms of n. • T(n) is the total number of accesses made from the beginning of selection_sort until the end. • selection_sort itself simply calls swap and find_min_index as i goes from 1 to n-1 = n-1 + n-2 + n-3 + … + 1 = n(n-1)/2 Or = ∑ (n - i) = n (n - 1) / 2 - O(n2)

16. Insertion Sort • Insertion sort keeps making the left side of the array sorted until the whole array is sorted.

17. Insertion Sort • while some elements unsorted: • Using linear search, find the location in the sorted portion where the 1st element of the unsorted portion should be inserted • Move all the elements after the insertion location up one position to make space for the new element

18. An insertion sort partitions the array into two regions

19. Example: sorting numbered cards 23 45 22 12 17 18 Given some numbered cards. Our aim is to put them into nondecreasing order.

20. Example: sorting numbered cards 23 45 22 12 17 18 3 1 6 5 4 2

21. Example: sorting numbered cards 23 45 22 12 17 18 3 1 6 5 4 2 3 1 6 5 4 2

22. 3 1 6 5 4 2 Example: sorting numbered cards 45 22 12 17 18 3 1 6 5 4 2 23

23. 3 1 6 5 4 2 Example: sorting numbered cards 45 22 12 18 3 1 6 5 4 2 17 23

24. 3 1 6 5 4 2 Example: sorting numbered cards 22 12 18 3 1 6 5 4 2 17 23 45

25. 3 1 6 5 4 2 Example: sorting numbered cards 22 12 3 1 6 5 4 2 17 23 45 18

26. 3 1 6 5 4 2 Example: sorting numbered cards 3 1 6 5 4 2 18 22 45 12 23 17

27. Algorithm: INSERTIONSORT Input: An array A[1..n] of n elements. Output: A[1..n] sorted in nondecreasing order. 1. for i  2 to n 2. x A[i] 3. j i - 1 4. while (j >0) and (A[j] > x) 5. A[j + 1] A[j] 6. j j - 1 7. end while 8. A[j + 1] x 9. end for

28. An insertion sort of an array of five integers

29. Con’t…. Algorithm Detail • A[i] is inserted in its proper position in the ith iteration in the sorted subarray A[1 .. i-1] • In the ith step, the elements from index i-1 down to 1 are scanned, each time comparing A[i] with the element at the correct position. • In each iteration an element is shifted one position up to a higher index. • The process of comparison and shifting continues until: • Either an element ≤ A[i] is found or • When all the sorted sequence so far is scanned. • Then A[i] is inserted in its proper position.

30. Analysis • Let a0, ..., an-1 be the sequence to be sorted. At the beginning and after each iteration of the algorithm the sequence consists of two parts: the first part a0, ..., ai-1 is already sorted, the second part ai, ..., an-1 is still unsorted (i  in  0, ..., n). • The worst case occurs when in every step the proper position for the element that is inserted is found at the beginning of the sorted part of the sequence.

31. Searching • Definition • The process of finding a particular element of an array • Types of Search • Sequential or linear search • Binary search • Why Search & Examples • Looking up a phone number • Accessing a Web site and • Checking the definition of a word in a dictionary

32. Linear Search (Sequential Search) Algorithm • Each element of an array is read one by one sequentially and it is compared with the search key. • Let A be an array of having n elements, A[0],A[1],A[2], ...... A[n-1]. “item” is the element to be searched. Then this algorithm will find the location “loc” of item in A. • Set loc = – 1, if the search is unsuccessful.

33. Con’t…. • Input an array A of n elements and “item” to be searched and initialize loc = – 1. • Initialisei = 0; and repeat through step 3 if (i < n) by incrementing i by one. • If (item = A[i]) loc= i GOTO step 4 4. If (loc >= 0) Display “item is found and searching is successful” 5. else Display “item is not found and searching is unsuccessful” 6. exit

34. Efficiency of Linear Search • The linear search algorithm runs in O(n) time

35. Example for implementation intLinear_Search(int list[], int key) { int index=0; int found=0; Do { if(key==list[index]) found=1; else index++; }while(found==0&&index<n); if(found==0) index=-1; return index; }

36. Binary Search algorithm • Binary search is an extremely efficient algorithm when it is compared to linear search. • Binary search technique searches “item” in minimum possible comparisons. Assumption • the given array is a sorted one, otherwise first we have to sort the array elements.

37. Con’t…. • Find the middle element of the array (i.e., n/2 is the middle element if the array or the sub-array contains n elements). • Compare the middle element with the data to be searched, and then there are following three cases. • If it is a desired element, then search is successful. • If it is less than desired data, then search only the first half of the array, i.e. the elements which come to the left side of the middle element. • If it is greater than the desired data, then search only the second half of the array, i.e., the elements which come to the right side of the middle element. • Repeat the same step until an element is found or exhaust the search area.

38. Efficiency The computational time for this algorithm is proportional to log2n, Therefore the time complexity is O(log n)

39. Example for Implementation intBinary_Search(int list[],int k) { int left=0; int right=n-1; int found=0; do{ mid=(left+right)/2; if(key==list[mid]) found=1; Else { if(key<list[mid]) right=mid-1; else left=mid+1; } }while(found==0&&left<right); if(found==0) index=-1; else index=mid; return index; }

40. Individual Assignment • Write a program for sorting (bubble, insertion, selection) and searching (linear, binary) algorithm? Hint: • Menu • Sorting • Bubble • Selection • Insertion • Searching • Linear • Binary • Criteria for Evaluation • Running time • Proper Indentation • Proper Declaration and Function usage