1 / 43

“OBJECTIVE AND SUBJECTIVE IDENTIFICATION OF INTERESTING AREAS IN VIDEO SEQUENCES”

“OBJECTIVE AND SUBJECTIVE IDENTIFICATION OF INTERESTING AREAS IN VIDEO SEQUENCES”. Danko Toma š i ć DIPLOMA PROJECT Erasmus Student, University of Trieste, Italy Responsible Assistant: Elisa Drelie Gelasca Professor: Touradj Ebrahimi. Outline. Introduction Context Application

adelie
Download Presentation

“OBJECTIVE AND SUBJECTIVE IDENTIFICATION OF INTERESTING AREAS IN VIDEO SEQUENCES”

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. “OBJECTIVE AND SUBJECTIVE IDENTIFICATION OF INTERESTING AREAS IN VIDEO SEQUENCES” Danko Tomašić DIPLOMA PROJECT Erasmus Student, University of Trieste, Italy Responsible Assistant: Elisa Drelie Gelasca Professor: Touradj Ebrahimi

  2. Outline • Introduction • Context • Application • State of the Art • Problem Statement • Subjective Experiments for Segmentation Quality Evaluation • Generation of Synthetic Segmentation Errors • Experimental Method • Data Analysis • Work in progress • Conclusions and Future Work

  3. Motivation • REAL WORLD SCENES • presents a large quantity of information to the Human... • high visual acuity – fovea • eye movements – not random ! VISUAL ATTENTION SELECTIVITY Automatic method for .. REGIONS OF INTEREST (ROI) In video sequences

  4. APPLICATIONS • COMPRESSION - preserve the quality in ROIs • SEGMENTATION QUALITY EVALUATION - overall and individual object • IMAGE AND VIDEO QUALITY ANALYSIS • Perceptual PSNR.. • IMAGE AND VIDEO DATABASES - extract ROIs in each scene ‘a priori’ • DIGITAL WATERMARKING - perceptual marking • MACHINE VISION - real-time robot navigation

  5. FACTORS WHICH INFLUENCE VISUAL ATTENTION • high-level or top-down - presence of people - background / foreground • low-level or bottom-up - motion - position - size - color and brightness - contrast - shape - orientation slower volition controlled Relevance.. task dependent rapid saliency driven task independent

  6. SALIENCE AND RELEVANCE • Salience - bottom-up task-independent factors - connected with observal-external objects or properties - unexpectedness or unusualness of an object • Relevance - top-down volition-controlled and task-dependent factors - observal-internal factors (goals and motivations) - connected to the specific situation = Conspicuity

  7. PREVIOUS ATTENTION MODELS • Correia and Perreira - estimation of video object’s relevance - both individually and in a given context • Osberger - perceptual vision model for image quality assessment and compression applications - early vision model and higher level attention processes - Importance Maps

  8. PREVIOUS MODELS • Osberger and Rohaly - automatic detection of ROIs in video sequences - improvement of previous model - adding temporal and movement features • Pardo - extraction of semantic objects from still images - perceptual metric and Importance Maps - improvement of Osberger’s models

  9. COLOR – STATE OF THE ART • Osberger - redattracts attention more than other colors - red induces higher amount of masking • Correia and Perreira - bright and colored objects are more noticed - red seems to be preferred PROBLEM – NO OTHER QUANTIFIED SOLUTIONS !!!

  10. COLOR – SIGNALLING AND MARKETING • American Standards Association and French Standard - red - dinamic, warm - fireproof protection - red - salient ‘par excellence’ - orange and yellow - salient colors – danger - green - calming color - materials of first aid - blue - cold and calming - caution “psychologically active” PROBLEM–NO OTHER QUANTIFIED SOLUTIONS !!!

  11. Subjective Experiment on Color.. • COLOR ColorTest I Color Test II Color Test III

  12. COLOR – COLOR TEST I First cycle Tested colors in CIELab color space Second cycle

  13. COLOR TEST I - RESULTS

  14. COLOR – COLOR TEST II • Colored objects with semantical meaning • Annoyance and Salience of a color interconnected ? • Adding gaussian noise (sigma =...)to channel L in HSL color space • Matlab - RANDN (M,N) - normal distribution • mean = 0 variance = 0.005 • Rating the annoyance level

  15. COLOR – COLOR TEST II Tested colors in CIELab color space Test image

  16. COLOR TEST II - RESULTS

  17. COLOR TEST II - RESULTS Cyan Green Pink Magenta Red = 241 Light blue Blue Orange Yellow = 241

  18. COLOR – COLOR TEST III • Colored objects with semantical meaning • Annoyance and Salience of a color interconnected ? • Blurred images - 3 levels of blurriness • Rating the annoyance level

  19. COLOR – COLOR TEST III Original image Test image

  20. COLOR TEST III - RESULTS COLOR TEST I vs. COLOR TEST III Second cycle highest level of blurriness 5 CASES Most important Most annoying 3 CASES 1st or 2nd most 2nd or 1st most important annoying 4 CASESRED 3rd most annoying !!! 4 CASESCYAN 1st most annoying !!!

  21. COLOR TEST III - RESULTS COLOR TEST I vs. COLOR TEST III • Light blue • Violet • Dark Green • Maroon • Red • Yellow • Green • Blue Similar importance Similar annoyance Similar importance Similar annoyance

  22. COLOR TEST III - RESULTS Average level lines 2nd level of blurriness 3rd level of blurriness

  23. COLOR TEST III - RESULTS HSL color space CIELab color space

  24. OBJECTIVE IDENTIFICATION OF INTERESTING AREAS Proposed method

  25. SEGMENTATION • WATERSHED SEGMENTATION Sequence #1 - Akiyo Frame #15 ORIGINAL FRAME #15 WATERSHED SEGMENTATION RESULT OF MERGE

  26. FACTORS WHICH INFLUENCE ATTENTION • COLOR

  27. FACTORS WHICH INFLUENCE ATTENTION • CONTRAST Ri,j – considered region Rj – neighboring region L, a, b – CIELab coordinates Bi,j – border pixels in Ri,j with Rj kborder = 10

  28. FACTORS WHICH INFLUENCE ATTENTION • SIZE

  29. FACTORS WHICH INFLUENCE ATTENTION • POSITION

  30. FACTORS WHICH INFLUENCE ATTENTION • SKIN HSV color space

  31. FACTORS WHICH INFLUENCE ATTENTION • MOTION - OPTICAL FLOW Original video Movement mask

  32. FACTORS WHICH INFLUENCE ATTENTION • FINAL COMBINATION OF FEATURES

  33. SUBJECTIVE EXPERIMENTS • SEGMENTATION QUALITY Sequence #1 - Akiyo Frame #15 DEFECT IN THE MOST IMPORTANT REGION DEFECT IN THE LEAST IMPORTANT REGION

  34. SUBJECTIVE EXPERIMENTS • SEGMENTATION QUALITY Sequence #2 - Highway Frame #28 DEFECT IN THE MOST IMPORTANT REGION DEFECT IN THE LEAST IMPORTANT REGION

  35. SUBJECTIVE EXPERIMENTS • SEGMENTATION QUALITY Sequence #3 - VideoEPFL Frame #39 DEFECT IN THE MOST IMPORTANT REGION DEFECT IN THE LEAST IMPORTANT REGION

  36. SUBJECTIVE EXPERIMENTS • RESULTS Sequence #1 - Akiyo

  37. SUBJECTIVE EXPERIMENTS • RESULTS Sequence #2 - Highway

  38. SUBJECTIVE EXPERIMENTS • RESULTS Sequence #3 - VideoEPFL

  39. VISUAL PERCEPTION • extracting information from the light • Acquiring knowledge - HVS = video camera • Objects and events in the environment • Light emitted or reflected by objects

  40. VISUAL ATTENTION • processing different information within the visual field • Overt eye movements - determine available optic information • Covert selective attention - determines what gets full processing CAPACITY SELECTIVITY amount of available what gets processed perceptual resources and what does not

  41. COLOR TEST I - RESULTS

  42. COLOR TEST II - RESULTS Cyan Green Pink Magenta Red >>10 Light blue Blue Orange Yellow >>10

  43. FACTORS WHICH INFLUENCE ATTENTION • COLOR ColorExperiment 1 Color Experiment 2 Color Experiment 3

More Related