1 / 64

Introduction and Architectures

INF 5070 – Media Storage and Distribution Systems:. Introduction and Architectures. 25/8 - 2003. Overview. Intro about the course multimedia applications and challenges Architectures Media (Video) on Demand Machine internals Video server structures Examples. INF5070: The Course.

carolsimon
Download Presentation

Introduction and Architectures

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript


  1. INF 5070 – Media Storage and Distribution Systems: Introduction and Architectures 25/8 - 2003

  2. Overview • Intro • about the course • multimedia applications and challenges • Architectures • Media (Video) on Demand • Machine internals • Video server structures • Examples

  3. INF5070:The Course

  4. Lecturers in INF5070 • Carsten Griwodz • email: griff @ ifi • room: 3203 • Pål Halvorsen • email: paalh @ ifi • room: 3309

  5. Network Network Network Network Content of INF5070 – I architectures file systems media data distribution protocols resource scheduling topologies

  6. Content of INF5070 – II • System architectures(server and system designs) • Media data(wire and file formats, codecs) • Media characteristics and user behavior (processing multimedia data, user interactivity) • Resource management(CPU and memory management)

  7. Content of INF5070 – III • Protocols with and without Quality of Service (QoS)(specific and generic QoS approaches) • Storage systems (management of multimedia files, retrieval) • Distribution(use of caches and proxy servers) • Adaptation, mobility and Peer-to-Peer(various clients, different amount of resources)

  8. Content of INF5070 – IV • Student assignment (will be presented more in-depth later): • (IXP programming) • read related article • assignment must be reported and presented to the class at the end of the course

  9. Goals of INF5070 • Media servers and distribution system • media characteristics • architectures • system support • protocols • distribution mechanisms • … • Be able to evaluate any combination of these mechanisms

  10. Pensum INF5070 • Approved presentation of student assignment • Oral exam (1-3/12): • all transparencies from lectures • content of own student assignment

  11. Applications and Challenges

  12. Applications • Multimedia enriches the user interface giving new applications • Broadcast – server is only VCR substitute (e.g., MTV Europe) • Media-on-Demand (MoD) • Video-on-Demand (VoD) (e.g., Bell Atlantic  ~1000 users, 700 choices) • News-on-Demand (NoD) (e.g., CNN, BBC  web today, BAD quality) • Learning-on-Demand (LoD) (e.g., the OMODIS project) • Virtual worlds (typically live, not on-Demand) • video conferencing (e.g., USIT’s electronic classroom) • games (e.g., Quake, …)

  13. Requirements • Application • QoS – time sensitivity • resource capabilities – support interactive streaming of multimedia content • Business • scalability • reliability • Architectural • topology • cost vs. performance

  14. Technical Challenges • User end system: • real-time processing of streams (1000 MIPS for an MPEG-II decoder) • request/response delay (< 150 ms for videophones) • high data rates, e.g., MPEG-II DVD quality: • average video rate of 3.5 Mbps • max. total data rate of 10.08 Mbps • max. user rate of 11.08 Mbps (all included like control signals) • Storage • real-time retrieval of contiguous media streams, e.g.: • 4000 movies * 90 minutes * 15 Mbps (HDTV) = 40.5 TB • 2000 CDs * 74 minutes * 1.4 Mbps = 1.4 TB • Network • real-time transport of contiguous media data

  15. Media-on-Demand (MoD) Systems • Classification parameters • structure of movies • interaction • presentation form • Common directions • analog  digital media • distribution  interaction • media broadcasting  media multicasting  personalized media • linear movies  branched movies  variable movies • Evolution • broadcast - traditional, no user control • pay-per-view - limited interactivity • quasi MoD - distinction into user groups, limited timely control • near MoD - same media distributed in intervals • true MoD - full user control, VCR capabilities, bidirectional

  16. Television (Broadcast) time channels sender receiver(s) • analog or digital • traditionally, one program per channel • analog use frequency division multiplexing only • digital may additionally use time division multiplexing inside one frequency (several programs per channel) • linear movies

  17. Near Video-on-Demand (NVoD) time channels sender receiver(s) • analog or digital broadcasting • one program over multiple channels • time-slotted emission of the program • linear movies

  18. (True) Video-on-Demand (VoD) time movies sender receiver(s) • digital uni- or multicasting • control channels • linear movies

  19. Comparison: NVoD vs. TVoD

  20. “Interactive Vision” time movie sender receiver(s) • digital uni- or multicasting • control channels • fixed non-linear movies

  21. “Cyber Vision” time sender receiver(s) • digital uni- or multicasting • control channels • variable non-linear “movies”, e.g., • - games, virtual reality, …

  22. Application Classification Overview Cyber Vision variable Interactive Vision movie structure branched HDTV VoD digital TV NVoD linear presentationform unidirectional VIDEO interaction analog bidirectional

  23. Media (Video) on Demand

  24. Challenges • VoD in LANs is solved: OVERPROVISIONING works • established in studio business • established in hotel/hospital/plane/… business • VoD in WANs • goals: • network-based distribution of media content to consumers • bring control to users • assumptions: • overprovisioning of resources will NOT work • no central control of delivery system • programs: • need for interoperability – not from a single source • need for co-operative distribution systems • amount of data: • estimated 65000 movies made in 1995  260 TB MPEG-2 data • additionally, data from TV-series, sport clips, news, … • historically: • much attention as “interactive TV (ITV)” some years ago • many (not successful) field trials • now: interest turned to Internet-based systems

  25. ITV Network Architechture Approaches • WAN backbones • SONET • ATM • Local distribution network • ADSL (asymmetric digital subscriber line) • FTTC (fiber to the curb) • FTTH (fiber to the home) • HFC (hybrid fiber coax) • Internet WAN • Diffserv over MPLS (multi-protocol layer switching) • point-to-point Gbps ethernet • Internet local • IP over the old distribution networks ATM / SONETbackbone network wireless Internet based systems ATM telephone cable

  26. Concerns: Internet-Based VoD Systems • Can technical problems be mastered? • broadband communication to every home • user-friendly end systems • server technology • Market success? • what prices will consumers accept? • high equipment (HW & SW) costs • data costs • will it be competitive? • to existing TV programs • to video rentals • what is the consequence of ITV field trials in USA and Europe? • no big success • only a few private consumers willing to participate • trials were cancelled or switched from TV-based to PC-based platforms

  27. Driving Forces • Hardware/software (IT) companies • computer (e.g., IBM, HP, Sun, Microsoft, …) • consumer electronics (e.g., Sony, Philips, …) • Network companies • telephone (e.g., Telenor, Telia, BT, AT&T, …) • cable TV (e.g., UPC, Time-Warner Cable, …) • Content companies • media • movies (e.g., Time-Warner, Disney, Paramount, Leo Kirch, …) • TV programs (e.g., NRK, TV2, TV3, RTL, …) • hyper media information bases (e.g., Springer, Bertelsmann, …) • home-shopping (e.g., Quelle, OTTO, …) • video games (e.g., Nintendo, Sega, Microsoft, …)

  28. VoD Deployment Status – I • Digital Video Broadcast (DVB) • no VoD • cable, antenna, or satellite broadcast • some NVoD scheduling approaches (e.g., TV 1000) • Digital Audio-Visual Council (DAVIC) • defines interfaces only • no standardization of algorithms for interoperation • Internet Engineering Task Force (IETF) • currently no large-scale, wide-area video distribution • sporadic use of cooperative web caching • starting AV caching considerations • defines protocols and inspires interoperability testing Broadcast world suited for Near VoD Broadcast world suited for True VoDInternet world suited for True VoD MPEG 4 suited for Interactive Vision

  29. VoD Deployment Status – II

  30. VoD System Architecture local distribution network backbone network local distribution network local distribution network

  31. master servers regional servers local servers end-systems VoD Storage Hierarchy completeness of available content • Popularity of movies:not all are equally popular – most request directed to only a few (Zipf distribution) • Use hierarchies: • Straight forward hierarchy: • popular videos replicated and kept close to clients • locality vs. communication vs. server costs

  32. VoD Components • Servers • Networks • backbone • local networks • Intermediate nodes • routers • proxy cache servers • replica servers • End-systems • PCs • TV sets with set-top boxes

  33. Traditional Server Machine Internals

  34. application file system communication system General Operating System Structure and Data Path user space kernel space

  35. Example: Intel Hub Architecture (850 Chipset) – I Intel D850MD Motherboard: RDRAM connectors CPU socket RDRAM interface system bus hub interface PCI bus Memory Controller Hub I/O Controller Hub PCI connectors

  36. application file system communication system disk network card system bus (64-bit, 400/533 MHz) I/O controller hub memory controller hub file system communication system RAM interface (two 64-bit, 200 MHz) application hub interface (four 8-bit, 66 MHz) network card PCI bus (32-bit, 33 MHz) disk Example: Intel Hub Architecture (850 Chipset) – II Note:these transfers only show data movement between sub-systems. Additionally, data touching operations within a sub-system will require that data is moved from memory and to the CPU, e.g.: - checksum calculation - encryption - data encoding - forward error correction Pentium 4 Processor registers cache(s) RDRAM RDRAM RDRAM RDRAM PCI slots PCI slots PCI slots

  37. application CPU CPU file system communication system L1 L1 disk network card (four 64-bit, 400 MHz) (eight 32-bit, 400 MHz) (four 64-bit, 400 MHz) RAM RAM RAM GX bus (two 32-bit, 600 MHz) PCI busses (32/64-bit, 33/66 MHz) RIO bus (two 8-bit, 500 MHz) Example: IBM POWER 4 POWER 4 chip core interface switch L2 fabric controller file system L3 memory controller GXcontroller L3 controller communication system application network card PCI slots remote I/O(RIO) bridge PCI host bridge PCI-PCI bridge disk PCI slots PCI host bridge PCI-PCI bridge Note:Again, data touching operations add movement operations

  38. Memory Hierarchies • We can’t access the disk each time we need data • Typical computer systems therefore have several different components where data may be stored • different capacities • different speeds • less capacity gives faster access and higher cost per byte • Lower levels have a copy of data in higher levels • A typical memory hierarchy: tertiary storage (tapes) price speed secondary storage (disks) main memory capacity cache(s)

  39. offline tape 1015 1013 magnetic disks 1011 online tape 109 main memory 107 105 cache 103 10-9 10-6 10-3 10 Storage Costs: Access Time vs Capacity typical capacity (bytes) access time (sec) from Gray & Reuter

  40. 104 cache main memory online tape 102 magnetic disks 100 10-2 offline tape 103 10-9 10-6 10-3 10 Storage Costs: Access Time vs Price dollars/Mbytes access time (sec) from Gray & Reuter

  41. Internal Server Design • Stream retrieval from disk and push to network • buffer requirements • bus transfers • CPU usage • concurrent streams (can be merged??) • storage (disk) system: • scheduling – ensure that data is available in time • block placement – contiguous, interleaving, striping • … • Stable operations: • redundant HW • multiple nodes • Much more, e.g., caching/prefetching, admission control, …

  42. Video Server Structure

  43. Internal content directory: External content directory: HP, DEC, Novell, … switch network attachment switch switch content directory/memory management switch switch file system switch switch storage management IBM TigerShark switch switch controller switch switch storage device Video Server:Server Components & Switches [Tetzlaff & Flynn 94]

  44. network attachment memory management file system storage management controller storage device Video Server Components incomingresolve request incomingdata request delivered resolution delivered data network attachment content directory control/application server data server

  45. data control Video Server:Simple General Server Architecture [Sitaram & Dan 00] • Storage subsystem: • stores data • different devices • Processor subsystem: • executing part • management and operations • Network subsystem: • transmit MM data storagesubsystem processor subsystem network subsystem control server data server clients application server • Data server: • data delivery • “specialized file system” • buffer manager • data importer/exporter • Application server: • user interface • billing • content database • user database • service gateways • Control server: • administrator • admission control • optimization

  46. data data Network Network content content Video Server:Directory Access & Data Retrieval • Two-step retrieval: • “problem”:resource management • Request redirection: • “problem”:client gets data from another machine

  47. server server client client Video Server:Directory Access & Data Retrieval • Pull model: • client sends several requests • deliver only small part of data • fine-grained client control • favors high interactivity • suited for editing, searching, etc. • Push model • client sends one request • streaming delivery • favors capacity planning • suited for retrieval, download, playback, etc.

  48. Single server easy to implement scales poorly Partitioned server users divided into groups content : assumes equal groups location : store all data on all servers load imbalance Network Network Network Video Server:Server Topology – I

  49. Externally switched servers use network to make server pool manages load imbalance(control server directs requests) still data replication problems (control server doesn’t need to be a physical box - distributed process) Fully switched server server pool storage device pool additional hardware costs e.g., Oracle, Intel, IBM data Network data data control data I/O switch Network data control Video Server:Server Topology – II

  50. Video Distribution Server:Typical In the Internet Today • Push systems(pull in video editing/database systems) • Traditional (specialized) file systems – not databases – for data storage • No in-band control (control and data information in separate streams) • External directory services for data location(RTSP/control server + data pump) • Request redirection for access control • Single stand-alone servers  (fully) switched servers

More Related