1 / 50

Computer Vision, 2007. ICCV 2007. IEEE 11th International Conference on

Learning Globally-Consistent Local Distance Functions for Shape-Based Image Retrieval and Classification. Computer Vision, 2007. ICCV 2007. IEEE 11th International Conference on Andrea Frome , EECS, UC Berkeley Yoram Singer, Google, Inc Fei Sha , EECS, UC Berkeley

phuong
Download Presentation

Computer Vision, 2007. ICCV 2007. IEEE 11th International Conference on

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript


  1. Learning Globally-Consistent Local Distance Functions for Shape-Based Image Retrieval and Classification Computer Vision, 2007. ICCV 2007. IEEE 11th International Conference on Andrea Frome , EECS, UC Berkeley Yoram Singer, Google, Inc FeiSha , EECS, UC Berkeley JitendraMalik, EECS, UC Berkeley

  2. Outline • Introduction • Training step • Testing step • Experiment & Result • Conclusion

  3. Outline • Introduction • Training step • Testing step • Experiment & Result • Conclusion

  4. What we do? • Goal • classify an image to a more appropriate category • Machine learning • Two steps • Training step • Testing step

  5. Outline • Introduction • Training step • Testing step • Experiment & Result • Conclusion

  6. Flow chart: training Generate features each image from dataset, ex: SIFT or geometric blur Compute distance dji, dki Input distances to SVM for training , evaluate W

  7. Flow chart: training Generate features each image from dataset, ex: SIFT or geometric blur Compute distance dji, dki Input distances to SVM for training , evaluate W

  8. Choosing features • Dataset: Caltech101 • Patch-based Features • SIFT • Old school • Geometric Blur • It’s a notion of blurring • The measure of similarity between image patches • The extension of Gaussian blur

  9. Geometric blur

  10. Flow chart: training Generate features each image from dataset, ex: SIFT or geometric blur Compute distance dji, dki Input distances to SVM for training , evaluate W

  11. Triplet • dji is the distance from image j to i • It’s not symmetric, ex: dji≠dij • dki > dji dji dki

  12. How to compute distance • L2 norm dji, 1 1 Image i 2 Image j 3 m features dji, 1 distance vectordji

  13. Example • Given 101 category, 15 images each category 101*15 Featurej distance vector distance vector 101*15 Image j vs training data

  14. Flow chart: training Generate features each image from dataset, ex: SIFT or geometric blur Compute distance dji, dki Input distances to SVM for training , evaluate W

  15. Machine learning: SVM • Support Vector Machine • Function: Classify prediction • Supervised learning • Training data are n dimension vector

  16. Example • Male investigate • Annual income • Free time • Have girlfriend?

  17. Ex: Training data

  18. space free vector income

  19. Mathematical expression(1/2)

  20. Mathematical expression(2/2)

  21. Support vector free Model income

  22. But the world is not so ideal.

  23. Real world data

  24. Hyper-dimension

  25. Error cut

  26. SVM standard mathematical expression Trade-off

  27. In this paper • Goal: to get the weight vector W 101*15 wj feature wj, 1 Image weight wj of W

  28. Visualization of the weights

  29. How to choose Triplets? • Reference Image • Good friend - In the same class • Bad friend - In the different class • Ex: 101category, 15 images per category • 14 good friends & 15*100(1500) bad friends • 15*101(1515) reference images • total of about 31.8 million triplets

  30. Mathematical expression(1/2) • Idealistic: • Scaling: • Different: The length of Weight i 0 0 triplet

  31. Mathematical expression(2/2) • Empirical loss: • Vector machine:

  32. Dual problem

  33. Dual variable • Iterate the dual variables:

  34. Early stopping • Satisfy KTT condition • In mathematics, a solution in nonlinear programming to be optimal. • Threshold • Dual variable updatefalls below a value

  35. Outline • Introduction • Training step • Testing step • Experiment & Result • Conclusion

  36. Flow chart: testing Query an image i Calculate Dxi, xis all training data, except itself. Output the most appropriate category

  37. Flow chart: testing Query an image i Calculate Dxi, xis all training data, except itself. Output the most appropriate category

  38. Query image? • Goal: classify the query image to an appropriate class • Using the remaining images in the dataset as the query image

  39. Flow chart: testing Query an image i Calculate Dxi, xis all training data, except itself. Output the most appropriate category

  40. Distance function(1/2) • Query image i Image i feature dxi, 1 distance vector distance vector 101*15 Image ivs all training data

  41. Distance function(2/2) 101*15 Dji Image I vs all the training data

  42. Flow chart: testing Query an image i Calculate Dxi, xis all training data, except itself. Output the most appropriate category

  43. How to choose the best image? • Modified 3-NN classifier • no two images agree on the class within the top 10 • Take the class of the top-ranked image of the 10

  44. Outline • Introduction • Training step • Testing step • Experiment & Result • Conclusion

  45. Experiment & Result • Caltech 101 • Feature • Geometric blur (shape feature) • HSV histograms (color feature) • 5, 10, 15, 20 training images per category

  46. Confusion matrix for 15

  47. Outline • Introduction • Training step • Testing step • Experiment & Result • Conclusion

  48. Conclusion • Learning Globally-Consistent Local Distance Functions for Shape-Based Image Retrieval and Classification

More Related