1 / 24

Structured Video Computing

Structured Video Computing. Y. Tonomura, A. Akutsu, Y. Taniguchi, G. Suzuki NTT Human Interface Laboratories IEEE Multimedia, Sept. 1994 Presented by Jason Hester CS598kn 23 Feb 2005. Outline. Introduction and Motivation Video Production Video Computing Segmentation Shot analysis

alice-albert
Download Presentation

Structured Video Computing

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. Structured Video Computing Y. Tonomura, A. Akutsu, Y. Taniguchi, G. Suzuki NTT Human Interface Laboratories IEEE Multimedia, Sept. 1994 Presented by Jason Hester CS598kn 23 Feb 2005

  2. Outline • Introduction and Motivation • Video Production • Video Computing • Segmentation • Shot analysis • Interactive Functions / New Interfaces • Related / Future Work

  3. Motivation • Current video applications – access and replay movies interactively. • Much greater potential exists • Compression, powerful CPUs, high-capacity storage • Provide applications of more structured use of video

  4. Structured Video “Because video is a structured medium in which actions and events in time and space tell stories or convey particular visual information, a video program must be viewed as a document – not a non-structured sequence of frames.”

  5. Proposal • Analyze a video stream • Segment the stream into shots • Index shots using extracted features • Camera work characteristics • Color representations • Browsing methods and user interfaces

  6. Desired Video Interaction • Focus on fast visual browsing. • Ability to grasp idea of lengthy video in short time. • Not simply fast forward. • Challenge: find and manage essential visual cues, then present them visually in an effective way

  7. Viewer-Video Interaction:Conceptual Model a) Viewer Interaction b) Video Computing c) Video Production & Editing

  8. Video Production • Key Concepts: • Take: continuous video • Cut: separates takes • Camera characteristics • Pan, tilt, zoom, etc. • Shot: edited takes • Resulting video contains embedded info: cut points, camera characteristics

  9. Video Computing • Main Function: Make the implied video structure explicit.

  10. Video Segmentation: Problems • Traditional Cut Detection – detect differences between frames using inter-frame comparisons (intensity, RGB, motion vectors). • Misdetection due to rapid object motion, slow motion, animation, strobes, fading, wiping, dissolving, etc. • Result: Low successful detection rate.

  11. Video Segmentation V = image difference

  12. Video Segmentation: Solution • 92-98% success rate over 4.5 hours of video (news, movies, documentaries) • 90% success when 1/3 of all cuts were via special affects

  13. Shot Analysis • Shot is simply sequence of frames capturing a scene’s spatial and temporal context. • Extract this information: • Camera work yields spatial situation • Color info yields object information

  14. Camera Work Information Extraction • Camera movement causes global change in objects. • Resulting point traces = motion vectors • Motion vectors yield camera work parameters • Computationally complex, not robust

  15. Camera Work Information Extraction • Proposal based on video x-ray imaging. • Easy calculations, robust

  16. Camera Work Information Extraction • Parallel to time = fixed camera • Slant = camera pan • Degree of slant = speed of pan • Line spread = zoom • No information present for track and dolly

  17. Representative Color Extraction • Utilize major colors in a shot • Each pixel: translate from RGB to HVC (hue, value, chroma). • HVC indicative of human perception • Select most frequent colors in shot • Result reflects: • Background in a long shot • Objects in middle or close shot • Can be used to locate specific shot

  18. Browsing • Used to see sequential context or to overview • How we grasp video content • Extract representative images (1 per shot)

  19. Browsing (cont.) • Extraction rule: sample at offset from cut points (or first suitable point) • Select browsing time – system adjusts • Flash=temporal display of stills (picture-story) • Rush=sequence of moving images • Stroboscopic= spatial display of stills

  20. New Video Interfaces • VideoScope • VideoSpaceIcon • ViewSpaceMonitor • PaperVideo

  21. PaperVideo • Photo albums and video indexing. • Shows potential simplicity of structured video apps.

  22. VideoScope • Possible use as video engineering tool. • Shows potential complexity of structured video apps.

  23. Related Work • Importance of visual interface • Must activate user’s visual sense • Must stimulate user’s ability to manipulate video • What can be done in video production stage?

  24. Notable Reference Cut Detection K. Otsuji, Y. Tonomura, “Projection Detecting Filter for Video Cut Detection,” Proc. ACM Multimedia 93, ACM Press, New York, 1993.

More Related