1 / 20

H euristic Pre-Clustering Relevance Feedback in Attention -Based Image Retrieval

H euristic Pre-Clustering Relevance Feedback in Attention -Based Image Retrieval. Wan-Ting Su , Wen-Sheng Chu and Jenn-Jier James Lien Speaker: Wen-Sheng Chu Robotics Lab. CSIE NCKU. Query Image. Positive Feedback. Negative Feedback. Heuristic Pre-Clustering View.

stacia
Download Presentation

H euristic Pre-Clustering Relevance Feedback in Attention -Based Image Retrieval

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. Heuristic Pre-Clustering Relevance Feedback in Attention-Based Image Retrieval Wan-Ting Su, Wen-Sheng Chu and Jenn-Jier James Lien Speaker: Wen-Sheng Chu Robotics Lab. CSIE NCKU

  2. Query Image Positive Feedback Negative Feedback Heuristic Pre-Clustering View User can change the positive group number on his/her own User can revise the clustering results manually Result View System Interface System Interface Robotics Lab, CSIE NCKU

  3. System Overview Offline Module : Attention-Based Image Retrieval Feature Extraction from Attended View Wavelet Transformation Attended View Extraction Image Database Low-Low Subband Query Image User Feedback? Ranking by Euclidean Distance Best Matches No END Yes Ranking by GBDA Learning PCA HeuristicPre-clustering UserRe-clustering VQ Online Module : Heuristic Pre-Clustering Relevance Feedback Robotics Lab, CSIE NCKU

  4. attention center Gaussian distance contrast value of pixel p at image location (i, j) neighborhood of pixel (i, j) Wavelet andAttended View Extraction • To reduce the computational cost • Contrast extraction is applied to the wavelet coefficient in the LL-subband. Got saliency map! Robotics Lab, CSIE NCKU

  5. System Overview Offline Module : Attention-Based Image Retrieval Feature Extraction from Attended View Wavelet Transformation Attended View Extraction Image Database Low-Low Subband Query Image User Feedback? Ranking by Euclidean Distance Best Matches No END Yes Ranking by GBDA Learning PCA HeuristicPre-clustering UserRe-clustering VQ Online Module : Heuristic Pre-Clustering Relevance Feedback Robotics Lab, CSIE NCKU

  6. Features Dimension Color mean, standard deviation and skew in HSV space 9 Standard deviation of the wavelet coefficients in 10 pyramid de-correlated sub-bands 10 13 statistical elements extracted from the edge map such as max fill time, max fork count, etc. 13 Visual Features Extraction • Table1. 32 low-level visual features Robotics Lab, CSIE NCKU

  7. System Overview Offline Module : Attention-Based Image Retrieval Feature Extraction from Attended View Wavelet Transformation Attended View Extraction Image Database Low-Low Subband Got features! Query Image User Feedback? Ranking by Euclidean Distance Best Matches No END Yes Ranking by GBDA Learning PCA HeuristicPre-clustering UserRe-clustering VQ Online Module : Heuristic Pre-Clustering Relevance Feedback Robotics Lab, CSIE NCKU

  8. Pre-Clustering • Principal Component Analysis (PCA) + • Vector Quantization algorithm (VQ) Robotics Lab, CSIE NCKU

  9. User Re-clustering Result User Re-clustering User Re-clustering System Pre-clustering Result Robotics Lab, CSIE NCKU

  10. System Overview Offline Module : Attention-Based Image Retrieval Feature Extraction from Attended View Wavelet Transformation Attended View Extraction Image Database Low-Low Subband Query Image User Feedback? Ranking by Euclidean Distance Best Matches No END Yes Ranking by GBDA Learning PCA HeuristicPre-clustering UserRe-clustering VQ Online Module : Heuristic Pre-Clustering Relevance Feedback Robotics Lab, CSIE NCKU

  11. Negative Samples Positive Samples Bouquets of Flowers Single Flower Re-weighting Scheme • Group-Based Discriminant Analysis (GBDA) • Multiple positive and multiple negative classes • Clustering each positive class • Scattering the negative example away from each positive class Robotics Lab, CSIE NCKU

  12. mi : the mean of the ith positive classCi c: the number of positive groups D : a set of negative examples GBDA Sw : the sum of the within-class scatter matrix of the positive groups SPN is the sum of between-class scatter matrices of positive-to-negative Robotics Lab, CSIE NCKU

  13. Experiment Result (1) • COREL image database • QS2: 1000 images from 10 selected categories • Each of 10 categories contains 100 images and is used as queries. Table 1. Image Categories in Query Set 2 Robotics Lab, CSIE NCKU

  14. 60.00% Attention-Based System Global 55.00% 50.00% 45.00% Precision 40.00% 35.00% 30.00% 25.00% 20.00% 10 20 30 40 50 60 70 80 90 100 Scope Experiment Result (2) Robotics Lab, CSIE NCKU

  15. 80.00% Attention-Based System Global 70.00% 60.00% 50.00% Precision 40.00% 30.00% 20.00% 10.00% 0.00% 1 2 3 4 5 6 7 8 9 10 Category ID Experiment Result (3) Robotics Lab, CSIE NCKU

  16. Query Image Experimental Results (4) First-time retrieval results Precision = 5/10 Precision = 7/20 Robotics Lab, CSIE NCKU

  17. Experimental Results (5) First-time feedback results Precision = 8/10 Precision = 17/20 Robotics Lab, CSIE NCKU

  18. Experimental Results (6) Second-time feedback results Precision = 10/10 Precision = 20/20 Robotics Lab, CSIE NCKU

  19. Conclusion • The major work in this study is integrating attention-based image retrieval with the relevance feedback algorithm using multiple positive and negative groups. • The system guides the user in clustering positive feedbacks by providing heuristic pre-clustering results. Then the user can revise the clusters manually. Robotics Lab, CSIE NCKU

  20. Experiment Result - Video Demo Robotics Lab, CSIE NCKU

More Related