1 / 12

A Novel Method for Generation of Motion Saliency

A Novel Method for Generation of Motion Saliency. Yang Xia, Ruimin Hu, Zhenkun Huang, and Yin Su. ICIP 2010. Outline. Introduction Itti’s model Proposed Visual Saliency Generation of motion feature map Enhancement of motion sub-saliency map Experiment Results Conclusion.

felton
Download Presentation

A Novel Method for Generation of Motion Saliency

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. A Novel Method for Generation of Motion Saliency Yang Xia, Ruimin Hu, Zhenkun Huang, and Yin Su ICIP 2010

  2. Outline • Introduction • Itti’s model • Proposed Visual Saliency • Generation of motion feature map • Enhancement of motion sub-saliency map • Experiment Results • Conclusion

  3. Introduction • Visual saliency • Bottom up saliency • Top down saliency • Applications • Image segmentation, motion detection, image/video compression…… • Motion saliency • Motion object is more salient to human vision system(HVS) than spatial contrast in video.

  4. Itti’s Model • For Image • Spatial features • Color, Intensity, and Orientation • Feature maps • Combining the normalized activation maps • Visual Saliency Model For Video • Temporal features • Flicker and Motion

  5. Itti’s Model • Motion feature map • Computed by spatially-shifted differences between Gabor pyramids from the current frame n and previous frame n-1 • Motion feature • The minimum captured object velocity at scale • : motion feature map for scale and orientation • : Gabor pyramid of original frame n • : the shifted Gabor pyramid of original frame n dx, dy: horizontal and vertical shift distance f : the frame rate

  6. Itti’s Model • Drawback • Inaccurate when the objects move slowly • when the velocity is smaller than , none of the pyramidal scales can capture the movement • group into the background • Only the edge of object is labeled salient • Using spatially-shifted differences Multi reference frame Enhance motion saliency map by spatial saliency information

  7. Proposed Visual Saliency • Generation of motion feature map • Multi reference frames to enhance the ability to capture object movement • Motion feature map • Processed by graph theory to form the activation map[1] (Reference frame) p n add two velocity profiles about and [1] J. Harel, C. Koch, and P. Perona, “graph-based visual saliency,” in Advances in Neural Information Processing Systems 19, Cambridge, MA: MIT Press, 2007

  8. Proposed Visual Saliency • Enhancement of motion sub-saliency map • Spatial sub-saliency map • Find the point which belongs to salient object • Check un-salient point is near a salient point which has large saliency value both in motion and spatial sub-saliency maps, and its spatial saliency value is close to that of the salient point. • Update the motion saliency • Generate the whole saliency map

  9. Enhancement of Motion Sub-saliency Map Top 25% of locations which have larger saliency values in SMS Top 5% of locations which have larger saliency values in SMM If and the difference of the spatial saliency values between and New saliency location set Motion saliency points Whole saliency map +

  10. Experimental Results • Dataset in CAVIAR—ThreePastShop1cor • ROC(Relative Operating Characteristic) score between estimated saliency maps(ESMs) and ground-truth saliency maps(GSMs) Anchor1: Itti’s model Anchor2: Itti’s model using activation operator based on graph theory SMRF: saliency model with the multi-reference frames SMRF+STE: plus spatio-temporal enhancement

  11. Experimental Results anchor1 anchor2 motion channel SMRF SMRF+STE anchor1 five channel anchor2 SMRF+STE

  12. Conclusion • First analyze the drawback of Itti’s motion saliency model. • Propose a novel motion saliency modelin which motion saliency map is obtained through the multi reference frames, and enhanced by spatial saliency information.

More Related