Abstract

1. Leaf Identifier for PC Professor: Wei-Yang Lin(林維暘) Team: Joey Su(蘇崇宇), Po-Yuan Su(蘇柏元) Abstract The problem with identifying a leaf manually is that it is time consuming given that the person has no prior training on how to identify a leaf. This means many people are turned off by the idea of having to sort through thousands of leaf pictures just to visually identify a leaf. Our goal is to simplify this process by letting users use a program, on a PC or mobile device, that takes a picture of a leaf and returns a percentage based match identifying the leaf. Procedure (Cont.) CSS and Comparing Leaves – The CSS image is a line plot of zero sigma vs. zero crossings. This unique circular curves on a CSS image illustrates an object’s inflection points and is not heavily biased towards objects that are rotated and/or out of alignment. • Problems • There are several problems that prevent a reliable image match: • Self-intersection – An image where parts of the leaf are overlapping • Rotated/misaligned – An image where the leaf is rotated and/or out of alignment when compared to the reference leaf. • Project Goals • To reliably match leaves that have self-intersections and/or are rotated • To create an Android application that will link with a PC based version of the program. • To build a database with a minimum of 5 different leaves with 5 training pictures each. As shown above, the curves are comparatively similar. This means directly comparing the curve characteristics between two CSS images will provide a reliable way to see if two objects are similar. The program uses a cross correlation algorithm defined by: Procedure Leaf Extraction – A leaf is first extracted by using a color filter, and then using the watershed algorithm to cut out anything unwanted. Where d = 1, 2, 3, …, n-1. This returns a [0, 1] number that shows how much correlation there is between the two series. Database – Our database is a tree based data structure. The root node, called the “Parent Directory”, contains a list of leaf types contained inside the database. The child node of the root node, called the “Type Node”, contains a lists of leaves that are of a type with generic test pictures to test for likeliness. The child node of the “Type Node”, called the “Species Node” contains a set of training images. Parent Directory Family_A Family_B Family_C The program compares each family’s test images, if the correlation is above 0.50 then the program will test the training images. If any correlation is below 0.50 then the program will skip it and move on to either the next leaf or family. Type A Leaf_A Leaf_B Species_A Experiment and Results Experiment – We did a brute force comparison of the correlation between a known leaf and all the leaves within our database. We take the average correlation of each family and see which family has the highest correlation. The leaf being compared will be picked from each family. To speed up the process we automatically reject a family if a leaf inside the family has a correlation below 0.50, and we automatically chose a family if a leaf has a correlation above 0.98. Curvature Scale Space (CSS) – The CSS image is a representation of the inflection points of a contour curve as it is smoothed. The curve evolution algorithm is: This is applied to the x and y values of a contour vector which is obtained by using OpenCV. The δ is increased for each evolution of the curve, and at each evolution the zero crossings points of the evolved curve are recorded. where Results – As the chart above shows each time the right family is chosen albeit at lower than expected correlation values. We were hoping to get average correlations of at least 0.90 when the correct family is being compared. From this small experiment we can conclude that the method is viable, yet reliability cannot be ensured until a large experiment with at least 100 different leaves is tested. δ = 0 δ = 1 δ = 11 δ = 22 δ = 33 δ = 44 δ = 55 δ = 66δ = 71 • Possible Future Expansions • The database can always be expanded. The tree structure provides an easy way to expand the database. • The data structure can be further improved to include information about the leaf and website links. We are not leaf experts so finding information about these leaves proved hard (Hence why this program is useful!) • We failed to implement an Android application. We ran into problems of the Android platform providing very limited memory. So we suggest making this application server based; where the Android device takes the picture, transmits it to a server, the server process the comparison, and transmits the results back to the Android device. Various stages of curve evolution as sigma increases. The curve stops evolving when there are no more zero crossings. References S. Abbasi, F. Mokhtarian. “Matching Shapes With Self-Intersections: Application to Leaf Classification”. IEEE Transactions on Image Processing Vol 13 No. 5 (2004): 653 – 661. Print. C. Q. Chang; F. Mai; Y. S. Hung. “Affine-Invariant Shape Matching and Recognition Under Partial Occlusion”. IEEE International Conference Image Processing (ICIP) : 26-29 Sept. 2010, Hong Kong. Pages: 4605 – 4608. Roy. “Resampling, Smoothing and Interest Points of Curves (via CSS) in OpeCV [w/ code]”. Ver. 1. More Than Technical, WordPress. Last Updated: 07 Dec. 2012. Web. Accessed: 11 Dec. 2013. P Bourke. “Cross Correlation”. V. 1. P. Bourke. August 1996. Web. 11 Dec. 2013. Itseez. Opencv. Ver. 2.4.6. Itseez. Last updated3 July 2013. Web, open source software library. Accessed :Aug. 2013