Efficient Image Region and Shape Detection by Perceptual Region Contour Grouping

1. Efficient Image Region and Shape Detection by Perceptual Region Contour Grouping Huiqiong Chen, Qigang Gao Faculty of Computer Science Dalhousie University

2. Outline • Introduction • Proposed Framework • Experimental Results • Conclusion

3. Introduction • Region Detection • Aim to find coherent regions from image. • Motivated by a variety of applications • Industrial inspecting systems • Vision based automation tasks • Region-based Image/Video retrieval • Challenges • Region associated with perceptual meaning • Detection accuracy • Reasonable time complexity

4. Introduction • Main solutions of Region Detection • Region based methods: • Identify the region interior by calculating pixel similarities. • Difficult to extract accurate boundary and shape from pixel collections. • Sensitive to noise; Over-segmentation • Boundary based methods: • Employ edge information to find region boundaries. • Pixel-level based (local information) • Time consuming

5. Introduction • Goal of our research • Extract meaningful regions from image and estimate their shapes without intensive computation. • Proposed method: • Taking advantage of inherent structure information carried by each Generic Edge Token (GET) feature, the perceptual structure of region shape can be obtained easily as well as region interior attributes.

6. Outline • Introduction • Proposed Framework • Experimental Results • Conclusion

7. Introduction of Generic Edge Token • GETs: Include Generic Segments (GS) and curve partition points (CPP). • GSs: Perceptually distinguishable segments classified by descriptive features. Categories of GSs The perceptual definitions of GSs

8. Introduction of Generic Edge Token • CPPs: Perceptually significant points where adjacent GSs meet and curve turning takes place. Curve partition example Basic Categories of CPPs

9. Key Ideas • Hypothesis • The inside of a homogeneous region does not have consistent boundaries. • Region representation • Meaningful regions can be represented perceptual GET-based closures in Perceptual Region/Closure hierarchy. • Perceptual closures extraction • An image is transformed into GET space on the fly represented by GET graph, which presents perceptual organization of GET associations. • Region detection can be achieved by perceptually grouping closures in GET graph.

10. Region Representation Perceptual region/closure concept hierarchy

11. Region Representation (Cont) • Regions can be represented by two types of perceptual closures (GET-based): • Basic contour Closure GET closure which can not be divided further into other closures. • Object Contour Closure GET closure which describes the outline of an object/individual component of object. Regions Detection Perceptual Closures Extraction

12. System Architecture Image GET feature Extraction GET features can be extracted by the edge tracker. GET feature Maps GET graph can be derived from GET feature map to code the perceptual structure of GET associations. GET graph Construction & Reduction Closure Definition GET Graph Perceptual closures are detected though perceptual cycle search in GET graph. Perceptual Closure Detection GET closures Regions can be constructed based on closures.

13. GET Feature Extraction Image regions GET-based closures Original image Extracted GET features

14. GET Graph Construction and Reduction • Transform image into GET graph • Convert GSs and CPPs to graph edges and vertices. • Remove noise edgesnot belonging to any cycle GET-based closures extraction graph cycle search Basic Contour Closure Basic cycle Object Contour Closure Outline cycle (a) GET features of an object in image (b) GET graph

15. Closure Detection Flow Chat GET Graph Select search initial edge for closure detection (get all closures with minimal efforts). Starting edge selection Starting edges track approximate adjacent edges along the GET graph and select proper edges to group the contour closures. (Cycle search strategy ) Closure detection by cycle search Cycles/Closures Classify result closures by its perceptual type. Result classification Classified closures

16. Determine Closure’s Initial edge • Starting edge selection • Closure searches only start from selected edges in stead of from all graph edges. • It reduces detection computation burden. • Edges not belonging to spanning trees are selected. • The correctness of selection is proven by Lemma 1. • Lemma 1: all perceptual closures can be obtained by tracking in graph starting from edges not belonging to spanning trees. That is, edges not belonging to spanning trees are considered as search starting edges.

17. Perceptual Cycle Search • Observation Basic cycle is a cycle without sub-cycle inside Outline cycle is cycle without any cycle outside. The outmost edge in one direction is the innermost edge in the other direction Basic cycle edge is Innermost edge in each step of tracking Outline cycle edge is the outmost edge in each step of tracking Cycle search forming perceptual closures

18. Perceptual Cycle Search (Cont) • Cycle Search strategy • Starting from closure edge ei with endpoints vm, vn select proper edges in each step of tracking so that form a path p = (vm)<ei, ..., ek, ei >(vn) in GET graph . • In each step of path selection, select the innermost edge in given direction (clockwise or anticlockwise) as next cycle edge among all adjacent edges. • Strategy validity proof • Lemma 2: A closure extracted using perceptual cycle search algorithm must be a perceptual closure.

19. Perceptual Closure Detection • Detect all perceptual closures in GET graph via Cycle Search algorithm Starting from each edge not included in any spanning tree, perform both anticlockwise and clockwise cycle searches by applying the cycle search strategy. • Detection method validity • Lemma 4: By using this method, (1) all extracted closures are perceptual closures; and (2) all perceptual closures are extracted from graph.

20. Closure Classification Classify closures into two perceptual types • Classification criterion: Total, summation of Included angles of the polygon simulating closure in cycle search direction. For a n-edge closure • if Total = 180*(n-2), the closure is basic contour closure; • if Total =180*(n+2), the closure is object contour closure. Simulate closure by polygon: (b) clockwise search from v7 of e7. (c) anticlockwise search from v10 of e11

21. Example Original image GET graph Object contour closure Basic contour closures

22. Outline • Introduction • Proposed Framework • Experimental Results • Conclusion

23. Result Samples • A set of test images is selected with different characteristics Original image Extracted GET features GET-basedregion contour Filled regions

24. Result Evaluation Table. Experiment result statistics. Average correctness= 91.13%.

25. Outline • Introduction • Proposed Framework • Experimental Results • Conclusion

26. Conclusion • Region detection is converted to GET graph search so that the computation is on GET feature level in stead of image pixel level. • It achieves high accuracy of detected regions without intensive computation. • It is suitable for segmenting regions with arbitrary shapes. The shape structure can be estimated by GET types. • Both object outline contours and their components can be detect with the association information.

27. Thank You for Your Attention Questions?

29. Region and Boundary integration Integrate boundary and region information to achieve a better detection result. • Use pre-extracted edge information in region-based processing. • Decision criterion control • Guidance of seed pixel selection • Fuse results of region and boundary based methods by combining their outputs. • Over-segmentation merge • Boundary refinement

30. GET Graph Reduction • Aim to reduce the computation cost of closure extraction in GET graph. • Remove noise edgesnot belonging to any cycle. (a) GET graph with noise edges (b) Reduced GET graph

31. Bridge Edge Gaps in Closure If two vertices are close within a small region (Normalized Distance dist(vi, vj) < T): Bridge edge gaps by virtual vertex -Virtual combination of two real vertices (a) GETs in image with edge gaps (b) GET graph with virtual vertices

32. Images used in Test

33. Result Samples

34. Comparative Results P. Bonnin, etc (1989) Y. Xiaohan, ect (1992) D. Sinclair (1999) F. Chan, ect (1996)

35. Future Research • Introduce other features such as color distribution and texture in processing to increase the robustness of detection. • Remove image noises and edge gaps in advance by additional knowledge or perceptual principal. • Encode region with a complete set of region attribute descriptions.