EE6358 Computer Vision – Regions Department of Electrical Engineering

1. EE6358 Computer Vision – RegionsDepartment of Electrical Engineering Presented by: Xin Yan (Cindy)

2. Outline • Regions and Edges • Region Segmentation • Region Representation • Split and Merge • Region Growing UTA

3. Regions and Edges • To analysis and understanding of an image • Partition the image to regions, by using the characteristics of the gray values of the pixels in the image • Two approaches to partition an image • Region based segmentation • Boundary estimation using edge detection UTA

4. Region Segmentation • Segmentation: • Group Pixels of an object together and marked to indicate that they belong to one region • How: • Value similarity • Measure difference between the gray values • Spatial proximity • Measure the Euclidean distance • Threshoding and Component labeling • Finding the pixels that lie on a region boundary UTA

5. Automatic Thresholding • Analyzes the gray value distribution in an image • Then uses the knowledge about the application to select the most appropriate threshold UTA

6. P-Tile Method • Uses knowledge about the area or size of the desired object to threshold an image • objects occupy p% of the image area UTA

7. Mode Method • The objects in an image have the same gray value, the background has a different gray value, and the image pixels are affected by zero-mean Gaussian noise UTA

8. Algorithm: Peakiness detection for Appropriate Threshold Selection • Find the two highest local maxima in the histogram. Suppose these are gi, gj • Find Lowest point gk in the histogram between gi, gj • Find Peakiness, defined as Min(H(gi),H(gj))/H(gk) • Use the combination (gi, gj ,gk )with highest peakiness to threshold the image. The value gk is a good threshold to separate objects corresponding to gi, gj UTA

9. Algorithm: Iterative Threshold Selection • Select an initial estimate of the Threshold, T. such as average intensity of the image • Partition the image into 2 groups: R1 and R2, using the threshold T • Calculate the mean gray values μ1 and μ2 of the partitions R1 and R2. • Select a new threshold: T= (μ1 + μ2)/2 • Repeat steps 2 to 4 until the mean value μ1 and μ2 do not change in successive iterations UTA

10. Local Thresholding • Methods to deal with uneven illumination or uneven distribution of gray values in the background • Analyze a histogram of the entire image • partition the image into m×m subimages and thresholding each subimage based on the histogram of the subimage • The final segmentation of the image is the union of the regions of its subimages. UTA

11. Local Thresholding UTA

12. Effect of illumination UTA

13. Variable Thresholding • Approximate the intensity values of the image by a simple function such as a plane or biquadratic. • The function fit is determined by the gray value of the background. • Histogramming and thresholding can be done relative to the base level determined by the fitted function. UTA

14. a: orig image b: 3-D plot c: histogram d: T = 85 e: T= 165 f: planar function g: difference plot h: the normalized image i: histogram of normalized image j: result T=110 UTA

15. Double Thresholding (1) Select two thresholds T1 and T2. (2) Partition the image into three regions : R1, containing all pixels with gray values below T1. R2 , containing all pixels with gray values between T1 and T2 , inclusive. R3 , containing all pixels with gray values above T2. where R1=core region, R2 = fringe (intermediate, transition) region, R3 =background. (3) Visit each pixel in region R2 . If the pixel has a neighbor in region R1, then reassign the pixel to region R1. (4) Repeat step 3 until no pixels are reassigned. (5) Reassign any pixels left in region R2 and R3 . UTA

16. (a). Orig image (b). Histogram (c). Core T1=70 (d). Result after growing the core using T2=155 (e).T= T2 (f). T1≤T≤T2 UTA

17. Region Representation • Three classes of region representations : • Array representations. • Hierarchical representations. • Symbolic representations. UTA

18. Array Representation • Use an array of the same size as the original image with entries that indicate the region to which a pixel belongs. • Use membership arrays (mask, bitmap) UTA

19. Hierarchical Representations • Allows image representation at multiple resolutions • Commonly used forms: • Pyramids • Quad trees UTA

20. Pyramids • Pyramid representation of an n × n image contains the image and k reduced versions of the image. n is the power of 2. • The other images are n/2 ×n/2, n/4 ×n/4… • The pixel at level i is obtained by combining information from several pixels in the image at level i + 1. • An implementation point is that the entire pyramid fits into a linear array of size: 2(22*level) UTA

21. Pyramids UTA

22. Quad Trees • An extension of pyramids for binary images. • Contains three types of nodes : white, black, and gray. • Is obtained by recursive splitting an image. • A region in an image is split into four subregions of identical size. UTA

23. Picture of Quad Tree UTA

24. Symbolic Representations • Some Commonly used symbolic characteristics: • Enclosing Rectangle • Centroid • Moments • Euler number • Others: • Mean, variance of intensity values UTA

25. Data Structure for Segmentation • When implementing region merging and splitting algorithms, information about the regions must be maintained in some data structure. UTA

26. Region Adjacency Graphs (RAG) • Represent regions and relationships among them in an image. • Nodes : regions, • arcs : a common boundary between regions. UTA

27. Algorithm: Region Adjacency Graph (1) Scan the membership array a and perform the following steps at each pixel index [i,j] (2) Let r1 = a[i,j] (3) Visit the neighbors [k,l] of the pixel at [i,j]. For each neighbor, perform the following: (4) Let r2 =a[k,l] , if r1 != r2 , add an arc between nodes r1 and r2 in the region adjacency graph UTA

28. Graph Representation of Region Adjacency Graph UTA

29. Picture Trees • Emphasizes the inclusion of a region within another region. • The emphasis is on nesting regions. • produced by recursively splitting an image into component parts. • Splitting stops when a region with constant characteristics has been reached. UTA

30. Picture representation of a picture tree. UTA

31. Super Grid • Boundaries should be located between the pixels of two adjacent regions. • However, often boundaries become actual pixels in image representations. • This is solved by a super grid UTA

32. Region Split and Merge • After segmentation the regions may need to be refined or reformed. • Split operation adds missing boundaries by splitting regions that contain part of different objects. • Merge operation eliminates false boundaries and spurious regions by merging adjacent regions that belong to the same object. UTA

33. Region Merging : • Determines the similarity between two regions. • Approaches to judge the similarity • on the gray value of regions • on the weakness of boundaries between the regions UTA

34. Algorithm: Region Merging (1) Form initial regions in the image using thresholding ( or a similar approach) followed by component labeling. (2) Prepare a region adjacency graph (RAG) for the image. (3) For each region in an image, perform the following steps: (a) Consider its adjacent region and test to see if they are similar. (b) For regions that are similar, merge them and modify the RAG. (4) Repeat step 3 until no regions are merged. UTA

35. Removing Weak Edges • Combine two regions if the boundary between them is weak. • remove weak edges between adjacent regions by considering not only the intensity characteristics, but also the length of the common boundary. UTA

36. Approach I : • Merging by removing weak boundaries, if the ratio of the weak boundary to the minimum region perimeter is above some Threshold T. UTA

37. Approach II: • Merging by removing weak boundaries, if the ratio of the weak boundary to the total common boundary is above some Threshold T. UTA

38. Region Splitting • If some property of a region is not constant • Regular decomposition Methods: divide the region into a fixed number of equal-sized regions. UTA

39. Algorithm: Region Splitting • Form initial region in the image • For each region in an image, recursively perform: • Compute the variance in the gray values for the region • If the variance is above a threshold, split the region along the appropriate boundary UTA

40. Algorithm: Split and Merge Region Segmentation • Start with the entire image as a single region • Split: Pick a region R. If H(R) is false, then split the region into four subregions. • H(R)=1 if the variance is small, 0 otherwise • Merge: Consider any two or more neighboring subregions, R1, R2,…, Rn, in the image. If H(R1UR2U…Rn) is true, merge the n regions into a single region • Repeat these steps until no further splits or merges. UTA

41. Region Growing • Group pixels or subregions into larger regions based on pre-defined data • Start with a set of ‘seed’ points • Check the neighboring pixels and add them to the region if they are similar to the seed UTA

42. How to Choose Seeds • Application dependent • for example: If targets need to be detected using infrared images, choose the brightest pixel(s) • If without a priori knowledge, compute the histogram and choose the gray-level values corresponding to the strongest peaks UTA

43. similarity criteria • based on any characteristic of the region in the image: • Average intensity • Variance • Color • Texture • Motion • Shape • Size UTA

44. Algorithm: Region growing using planar and Biquadratic Models • Partition the image into initial seed regions Ri(0) • Fit a planar model to each seed region. • If the chi-squared error is small enough, accept the seed region and its model; otherwise, reject the seed region • For each region, find all points that are compatible with the region by considering the neighbors of the region UTA

45. 5.If there were no compatible points, m = m+1. if m >M, then do not grow the region further; otherwise go to step 3 6. Form the new region Ri(k+1) = RikU Cik, Refit the model to Ri(k+1) and compute the Error 7. Compute the difference error UTA

46. 8. if ρ(k)<T1, go to step 3 9. Increase the order of the model: m = m+1. if m>M, then do not grow the region. 10. Refit the region at the new model f(x,y,a,m). If the error of fit decreases, accept the new model go to step 3, otherwise, do not grow region further UTA

47. References: • Shareholding: http://www.cse.unr.edu/~bebis/CS791E/Notes/Thresholding.pdf • Demo: http://www.olympusmicro.com/primer/java/digitalimaging/processing/automaticthresholding/ • Chapter 3: http://harmony.cs.pusan.ac.kr/board/main.cgi/cv04-03f.pdf?down_num=1079671972&board=Note_CV&command=down_load&d=&filename=cv04-03f.pdf • http://www.icaen.uiowa.edu/~dip/LECTURE/Understanding6.html • Paper: http://melisa.intranet.gr/documentation/publications/pcs_2003_andrade.pdf • Segmentation: http://www.icaen.uiowa.edu/~dip/LECTURE/Segmentation.html • Notes: http://www.cse.unr.edu/~bebis/CS791E/Notes/ • Chi-Square: http://www.georgetown.edu/faculty/ballc/webtools/web_chi_tut.html#requirements UTA