1 / 31

Efficient Object Localization & Image Classification Fusion Study

This research combines efficient object localization and image classification methods to enhance performance. The study explores the localization of objects within images and the classification of those objects. It presents a framework that efficiently localizes objects using a sliding window approach and combines various image representations, such as histograms, gradients, and features, for robust performance. By analyzing the results of experiments that combine image classification and localization, the study provides insights into the effectiveness of the fusion approach. The proposed combination model considers the detectability of objects by different modalities, classification, and detection, to improve the overall accuracy. Experimental results demonstrate the efficiency and accuracy of the fusion model in handling both detectable and undetectable objects. The study evaluates the performance using the PASCAL VOC dataset and compares it with state-of-the-art methods.

donaldlwilliams
Download Presentation

Efficient Object Localization & Image Classification Fusion Study

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. Combining efficient object localization and image classification H. Harzallah, F. Jurie and C. Schmid LEAR, INRIA Grenoble, LJK

  2. Tasks • Image classification: assigning labels to the image Car: present Cow: present Bike: not present Horse: not present …

  3. Cow Car Tasks • Image classification: assigning labels to the image Car: present Cow: present Bike: not present Horse: not present … • Object localization: define the location and the category Location Category

  4. Contributions • Object class localization method • Combining image classification and object localization Localization--Classification++ Localization++Classification--

  5. Overview • Related work and datasets • Efficient object localization • Experimental results • Combining image classification and localization • Experimental results • Conclusion

  6. Related work • Object localization • Sliding window [Dalal06] [Rowley95] • Implicit shape model [Leibe04] • SVM classifiers [Chum07] [Ferrari08] • Cascade of classifiers [Viola01] [Vedaldi09] • Context information • Combination of context sources [Divvala09] • Graphical model of events in images [Li07] • Local segmentation + global classification [Shotton08] [Heitz08]

  7. PASCAL VOC dataset • PASCAL VOC dataset 2007 and 2008 • Two tasks : classification and localization • Fixed train/test set-up for the 20 object classes • Standard evaluation measure • Area of overlap as detection matching criterion • Average precision for performance evaluation

  8. Overview • Related work and datasets • Efficient object localization • Experimental results • Combining image classification and localization • Experimental results • Conclusion

  9. Efficient object localization Sliding window based approach Image representation Combination of features Extensive parameters evaluation Robust image representation Efficient search strategy

  10. Image representation Histogram Histogram Histogram Histogram Histogram Histogram • Combination of 2 image representations • Histogram Oriented Gradient • Gradient based features • Integral Histograms • Bag of Features • SIFT features extracted densely + k-means clustering • Pyramidal representation of the sliding windows • One histogram per tile

  11. Efficient search strategy • Reduce search complexity • Sliding windows: huge number of candidate windows • Cascades: pros/cons • Two stage cascade: • Filtering classifier with a linear SVM • Low computational cost • Evaluation: capacity of rejecting negative windows • Scoring classifier with a non-linear SVM • Χ2 kernel with a channel combination [Zhang07] • Significant increase of performance

  12. Efficiency of the 2 stage localization • Performance w. resp. to nbr of windows selected by the linear SVM (mAP on Pascal 2007) • Sliding windows: 100k candidate windows • A small number of windows are enough after filtering

  13. Localization performance: aeroplane

  14. Localization performance: car

  15. Localization performance • Mean Average Precision on all 20 classes • PASCAL 2007 dataset

  16. Localization examples: correct localizations Bicycle Car Horse Sofa

  17. Localization examples: false positives Bicycle Car Horse Sofa

  18. Localization examples: missed objects Bicycle Car Horse Sofa

  19. Overview • Related work and datasets • Efficient object localization • Experimental results • Combining image classification and localization • Experimental results • Conclusion

  20. Image classification & localization use a different information Combination: key points • For many TP only one has a high score • Truncated objects: hard for the detector • Small objects: ok for the detector but not for the classifier using global information

  21. Combination model • Input: classification ( Si ) and localization ( Sw ) scores • Output: probability that object is present • Suppose that classification and localization outputs are independent:

  22. Combination model • For each modality (classification/detection): notion of detectability P(Di) for classifier and P(Dw) for detector • Encodes the ability to detect presence of the objects • Assuming that the classifier/detector outputs conditional probabilities: P(O|Di,Si) and P(O|Dw,Sw)

  23. Combination model • P (O |Si) = P(Di) × P(O|Si, Di) + P(¬Di) × P(O|Si,¬Di)‏ • P (O |Sw) = P(Dw) × P(O|Sw, Dw) + P(¬Dw) × P(O|Si,¬Dw)‏ • Final probability: • Handle both cases: • Object detectable by two modalities • Object detectable by only one modality

  24. Combination model • P(O|¬Di,Si) and P(O|¬Di,Si) : constant value • Sw = classification by localization: highest localization score • Priors P(Di) and P(Dw) class dependant

  25. Combination experimental setup • Image classifier : INRIA_flat classifier • SVM classifier Χ2 kernel using multiple feature channels [Zhang07] • Excellent results in PASCAL 2008 challenge • Detector : as described previously • Experimental validation on PASCAL VOC 2007 • Comparison to the state of the art on PASCAL VOC 2008

  26. Experimental results : gain obtained Classification Localization

  27. Experimental results Car localization • Correct but low score localization • High classification score • score increased after combination

  28. Experimental results Car classification • High classification score • No localization • score decreased after combination

  29. Comparison to the state of the art • Based on blind evaluation on PASCAL VOC 2008 • Classification • Best on 12 classes out of 20 • Localization • Best on 11 classes out of 20

  30. Conclusion • Efficient localization method • Successful combination of classification and localization • State of the art performance on both tasks

  31. Thank you

More Related