1 / 18

ITEA CANDELA Content Analysis and Networked DELivery Architectures

!. 2023. å. ITEA CANDELA Content Analysis and Networked DELivery Architectures. Home Multimedia Demonstrations. International Review Helsinki, Finland June 16, 2005. MPEG2/4. MPEG7. Outline.

giacomo-franklin
Download Presentation

ITEA CANDELA Content Analysis and Networked DELivery Architectures

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. ! 2023 å ITEA CANDELAContent Analysis and Networked DELivery Architectures Home Multimedia Demonstrations International ReviewHelsinki, FinlandJune 16, 2005

  2. MPEG2/4 MPEG7 Outline • Introduction & PositioningVideo / Multimedia Content Analysis (VCA / MCA) and Networks for Home Multimedia • WP1: VCA / MCA for Home Multimedia • Service Unit AV segmentation • Service Unit Face detection • Service Unit Sport sequence analysis • WP2: System Architecture • Distributed content analysis (encapsulated components – Service Units), system architecture and system solutions for prototyping in-home networks • WP2: AV content and meta database • Database solutions for storing and retrieving features • Integration with other CANDELA application domains • Home Multimedia meets Mobile Multimedia

  3. CANDELA introduction & positioningFocus of the Home Multimedia application demonstrator Demonstrators WP1 WP2 WP3 Medical systems Philips Research, TU/e Surveillance Applications Home Multimedia Networkedinfrastructure Interoperability of media and devices

  4. Cable EPG WWW Content management play record Aim of CANDELA Home Multimedia Content Management in Consumer Electronics (CE) Networks The Future of Consumer Electronics In-Home Networks • storage capacity of >1 Tbyte  consumer desire for ease-of-use • Distributed processing GigaFlops • Problem: retrieving and managing AV content • Required: • Metadata (content descriptors) & their generation (VCA / MCA) • System solutions for distributed content analysis (DCA) including media database solutions and metadata standards for interoperability

  5. Content & descriptors AV Storage Content Management HDD MPEG7 Cable WWW Database VCA for ‘Smart’ content management • Real-time Content Analysis dissemination • Demo at ICME 2005 (Amsterdam, The Netherlands) • Collaboration projects CANDELA / MultimediaN User Interface

  6. WP1: VCA / MCA for home multimedia Service Unit (SU) AV Segmentation & Face Detection Content from e.g. MICROGENESIS / Bosch • AV Segmentation • Auto AV chaptering for CE devices • Annotation tool for Home Videos • SU Cut Detection (CD) + SU Audio Segmentation (AS) ... results in SU AV Segmentation (AVS) SU CD SU AS SU FD SU ScBD Media Database MPEG-7 conversion • Face Detection • Still & video content management • Surveillance & Automotive & Basis for Face Recognition: SU Face Detection (FD) • Face Detector on a commercial Philips products • Benchmarked of various Viola-Jones-based Face Detectors User e.g. SOLID / MICROGENESIS CANDELA: content from Co. Microgenesis / SOLID  MPEG-7 output to SOLID / BOSCH / Microgenesis

  7. WP1: VCA / MCA for home multimedia Service Unit Automatic Sports AV Analysis(Video Coding and Architectures Group / TU Eindhoven) 1. Detect white pixel 2. Detect line 3. Camera model 1. Get local area 2. White pixel rate 3. Threshold difference Court detection and camera calibration 1. Real world speed 2. Real world position 3. Speed change Video input Playing scene detection Player detection and tracking Output Event analysis 1. Build background 2. Produce binary map 3. Extract players 4. Track players

  8. Demo

  9. MPEG2/4 MPEG7 Outline • Introduction & PositioningAudioVisual (AV) Content Analysis (CA) and Network for Home Multimedia • WP1: AV CA for Home Multimedia • Service Unit AV content segmentation • Service Unit Face detection • Service Unit Sport sequence analysis • WP2: System Architecture • Distributed content analysis (encapsulated components – Service Units), system architecture and system solutions for prototyping in-home networks • WP2: AV content and meta database • Database solutions for storing and retrieving features • Integration with other CANDELA application domains • Home Multimedia meets Mobile Multimedia

  10. WP2: networked delivery • System architecture for the in-home system • database, terminals, processing components • interconnect • protocols • The connected devices form the physical platform • content sharing • shared processing • shared resources • Software architecture • service units encapsulate functionality • this functionality is exposed on the network • applications as compositions of service units (e.g. VCA applications)

  11. Example: commercial block detector Input PC2 video Meta YUV SU video • Required metadata: • commercial block • detection metadata AVDecoder SU MetaSync SU audio AFX SU Low-level features audio video metadata Network PC5 PC4 PC3 GUI Commercial Block Detector SU Cut Detector SU Silence Detector SU metadata PC6 metadata metadata AV Segmentor SU MDB High-level features Mid-level features

  12. Focus: reliability • Applications rely on the cooperation of several devices • devices may disappear • more points of failures • integration of 3rd party VCA/MCA components • Considered errors • Service Units • Logical (catchable by a Service Unit itself) • Physical (failstop of process running the service) • Host (device) breakdown • Communication (network) failure

  13. Example Error detection and recovery • Unplug a network cable from the system • The health keeperdetects the malfunction • Recovery: the missing component is started on another nodeand reconnected after a timeout has exceeded PC1 PC2 SU2 SU1 HM2 HM1 PC4 PC3 Health Keeper SU1_new

  14. MPEG2/4 MPEG7 Outline • Introduction & PositioningAudioVisual (AV) Content Analysis (CA) and Network for Home Multimedia • WP1: AV CA for Home Multimedia • Service Unit AV content segmentation • Service Unit Face detection • Service Unit Sport sequence analysis • WP2: System Architecture • Distributed content analysis, system architecture and system solutions for prototyping in-home networks • WP2: AV content and meta data database • Database solutions for storing and retrieving features • Integration with other CANDELA application domains • Home Multimedia meets Mobile Multimedia

  15. WP2: AV content and meta database • Allows persistent storage of features • Allows nonlinear features browsing and searching • Allows non real-time features processing • Enables features distribution using metadata standards • Enables new applications • Serves as a reference to evaluate new algorithms

  16. Asset Mng Asset Mng Asset Mng AV Features EPG Mng AV Features EPG Mng EPG Mng AV Features PersonMng PersonMng PersonMng APIs APIs APIs objects/ schemas objects/ schemas objects/ schemas wrappers wrappers ODBC SQL libdbi wrappers ODBC SQL libdbi ODBC SQL libdbi DBMSes DBMSes B-tree Solid XML MySQL SQLite B-tree Solid XML MySQL SQLite DBMSes B-tree Solid XML MySQL SQLite databases snapshots local DB databases snapshots local DB databases snapshots local DB Media Database (MDB) Framework • Persistent storage and management of: • digital asset and asset collections • AV features • info about people and devices • Data representation: • multiple formats (DIDL, MPEG7, ID3) • dynamic views (annotations) • views on remote datasets (snapshots) • Adaptable : • support for ‘unknown’ metadata • extensible (API, model) • Interfaces: • relational • object-oriented Mobile Stationary

  17. Integration with other CANDELA demonstratorsHome Multimedia meets Mobile Multimedia A/V Signal (Solid, Microgenesis) Mobile multimedia demo DB Analysis e.g. face detection Content Metadata MPEG7 Home multimedia demo

  18. Summarizing: results • VCA algorithms for generating metadata • video annotation with e.g. face recognition, shot detection, tennis scene analysis • Different semantic levels: low, mid and high-level features • mid level: semantic AV segmentation, face detection • mid & high level: tennis player position & behavior • Data management system for storing and retrieving AV features • adaptable for heterogeneous devices • interoperable with multiple data formats • System architecture • flexible and reliable composition of components into applications • distributed processing

More Related