1 / 30

Streaming Video Over the Internet

Streaming Video Over the Internet. Andreas Panteli 03/05/2012 ECE 654 Advanced Computer Networks. Introduction. Video principles When an image appears on the retina it is retained for some millisecond before decaying

varden
Download Presentation

Streaming Video Over the Internet

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript


  1. Streaming Video Over the Internet Andreas Panteli 03/05/2012 ECE 654 Advanced Computer Networks Streaming Video Over The Internet

  2. Introduction • Video principles • When an image appears on the retina it is retained for some millisecond before decaying • If a sequence of images is drawn line by line at 50 images/sec the eye does not notice that it is looking at discrete images • Streaming Video • Video content need not be downloaded in full • It is being played out while parts of the content are being received and decoded • Real-time multimedia have timing constraints • Audio and video data must be played out continuously • Bandwidth, delay and loss requirements • No QoS guarantees to streaming video, from the current best-effort Internet! Streaming Video Over The Internet

  3. An Architecture for Video Streaming Streaming Server Client/Receiver Media Synchronization Video Decoder Audio Decoder Storage Device Video Compression Application-layer QoS Control Application-layer QoS Control Compressed Video Raw Video Audio Compression Compressed Audio Raw Audio Transport Protocol Transport Protocol Internet (Continuous media distribution services) Streaming Video Over The Internet

  4. Key Areas • Video Compression • Application-Layer QoS Control • Continuous Media Distribution Services • Media Synchronization Mechanisms • Protocols for Streaming Media Streaming Video Over The Internet

  5. Key Areas • Video Compression • JPEG standard • MPEG standard • Application-Layer QoS Control • Continuous Media Distribution Services • Media Synchronization Mechanisms • Protocols for Streaming Media Streaming Video Over The Internet

  6. Digital Video • Digital video is represented as a sequent of frames, each consisting of a rectangular grid of picture elements (pixels) • For color video, 8 bits for each of the RGB colors are being used • 24 bits/pixel => 16 million colors • To produce smooth motion, digital video, we must display at least 25 frames/sec, but to avoid flickering we need to repaint each frame on the screen twice • Smoothness and flickering affects the bandwidth requirements • Consider the lower video configuration for computer monitor 1024x768 with 24 bits/pixel and 25 frames/sec • Need to be fed at 472 Mbps • We need video compression! Streaming Video Over The Internet

  7. Video Compression • Two algorithms are needed • Encoding and Decoding • Asymmetries of the algorithms • Acceptable for the encoding algorithm to be slow and require expensive HW, if the decoding algorithm is fast • Only for video on demand • For real-time multimedia like video conferencing, slow encoding is unacceptable • Acceptable to have a slightly different video signal than the original, after encoding and decoding (Lossysystem) • Lossy systems are important • Accepting a small amount of information loss can give a huge payoff in terms of the compression ratio possible Streaming Video Over The Internet

  8. The JPEG Standard • Compression algorithm for still images • Could be applied to each image in succession to achieve video compression • Compute the luminance Y and the two chrominances I and Q • Y = 0.30R + 0.59G + 0.11B • I = 0.60R – 0.28G - 0.32B • Q = 0.21R – 0.52G + 0.31B • Compress the two chrominances twice than the luminance • Lossy reduction, but the eye barely notices it since it responds to luminance more than to chrominance • JPEG often produces 20:1 compression or better • Decoding a JPEG image requires running the algorithm backwards Streaming Video Over The Internet

  9. The MPEG Standard (1/2) • Main algorithms for video compression and international standards since 1993 • Can compress both audio and video • Three parts in MPEG-1 • Audio • Video • System (integrates the other two) Streaming Video Over The Internet

  10. The MPEG Standard (2/2) • Two kinds of redundancies in videos • Spatial redundancy • Coding each frame separately with JPEG • Temporal redundancy • Consecutive frames are often almost identical • MPEG-1 output consists of four kinds of frames • I (Intracoded) frames: Self-contained JPEG-encoded still pictures • P (Predictive) frames: Block-by-Block difference with the last frame • B (Bidirectional) frames: Differences between the last and the next frame • D (DC-coded) frames: Block averages used for fast-forward • MPEG-1 can achieve a compression factor of 40 • Decoding has about the same complexity as encoding Streaming Video Over The Internet

  11. Key Areas • Video Compression • Application-Layer QoS Control • Congestion control • Error control • Continuous Media Distribution Services • Media Synchronization Mechanisms • Protocols for Streaming Media Streaming Video Over The Internet

  12. Application-Layer QoS Control • Avoids congestion and maximizes video quality in the presence to packet loss Congestion Control Error Control FEC Delay-Constrained Retransmission Error-Resilient Encoding Error-Concealment • Rate Control • Source-Based Rate Control • Receiver-Based Rate Control • Hybrid-Rate Control • Rate Shaping • Frame-dropping filter • Layer-dropping filter • Frequency filter • Re quantization filter Streaming Video Over The Internet

  13. Rate Control (1/2) • Technique to determine the sending rate of video traffic based on the estimated available bandwidth of the network • Source-based rate control • The sender is responsible for adapting the video transmission rate based on feedback from the receivers • Probe-based approach • Sending rate adjustment to maintain the packet loss ratio below a certain threshold • Model-based approach Streaming Video Over The Internet

  14. Rate Control (2/2) • Receiver-based rate control • The receivers regulate the receiving rate of video streams by adding/dropping channels • Probe-based approach • When no congestion is detected, join a channel • => increase of it’s receiving rate • When congestion is detected, drop a channel • => reduction of it’s receiving rate • Model-based approach • Explicit estimation for the available network bandwidth • Also based on Equation (1) • Hybrid rate Control • Combination of the above mentioned techniques Streaming Video Over The Internet

  15. Rate Shaping • Matches the rate of a pre-compressed video bit stream to the target rate constraint • A rate shaper (or filter) is required for source based rate control • Types of filters • Codec filter • Performs transcoding between different compression schemes • Frame-dropping filter • I, P ,B , D frames in MPEG • Frequency filter • Operates in the frequency domain (DCT coefficients) • Re-quantization filter • Re-quantizes the DCT coefficients with a larger quantization step Streaming Video Over The Internet

  16. Error Control (1/2) • Forward Error Correction (FEC) • Add redundant information so that original message can be reconstructed in the presence of packet loss • Channel coding • Stream is chopped into segments • Each segment is packetized into k packets • A block code generates an n-packet block, where n>k • User only needs to receive any k packets out of n of the block • Source coding • N-th group of blocks contains redundant information of (n-1)th group of blocks • Joint source/channel coding • Optimal rate allocation between Channel and Source coding Streaming Video Over The Internet

  17. Error Control (2/2) • Delay-constraint Retransmission When the receiver detects the loss of packet N if (Tc + RTT + Ds < Td(N)) send the request for packet N to the sender Where Tc current time RTT estimated round-trip time Dsa slack term (e.g receiver’s decoding delay) Td(N) time when packet N is scheduled for display • Error-resilient Encoding • Enhances robustness of compressed video to packet loss • A video is compressed into multiple streams (descriptions) • Each description provides acceptable visual quality • Combined descriptions provide a better visual quality • Error Concealment • Performed by the receiver when packet loss has already occurred • Spatial interpolation • Missing pixel values are reconstructed using neighboring pixels • Temporal interpolation • Loss data is reconstructed using data from the previous frame Streaming Video Over The Internet

  18. Key Areas • Video Compression • Application-Layer QoS Control • Continuous Media Distribution Services • Network Filtering • Application-level Multicast • Content Replication • Media Synchronization Mechanisms • Protocols for Streaming Media Streaming Video Over The Internet

  19. Network Filtering • Adequate support from the network is critical in order to provide quality multimedia presentations • Routers have no knowledge of the format of the media streams • Randomly discard packets • Filters receive the client’s requests and adapt the stream sent by the server accordingly • Typically, frame-dropping filters are used • Drop packets in a way that gracefully degrades the stream’s quality • Increase bandwidth efficiency by discarding late frames R: Router Streaming Video Over The Internet

  20. Application-level Multicast • Internet’s original design fails to effectively support streaming-media multicast • IP multicast has barriers • Scalability, network management, support for higher layer functionality (e.g. error, flow and congestion control) • Application-level multicast to the rescue • Builds a multicast service on top of the Internet • Media bridge • Routing in the application layer • Each Media Bridge is interconnected with one or more neighboring Media-Bridges through explicit configuration • Collectively employ a distributed application-level multicast routing algorthm Unicast Multicast Streaming Video Over The Internet

  21. Content Replication • Advantages • Increased network’s bandwidth efficiency • Reduced load on streaming servers • Reduced latency for clients • Increased availability Mirroring Caching Exploits temporal locality Cache retrieves data from server Clients retrieve data from cache if available Cache sharing and cache hierarchies • Scatter copies of the original multimedia files in different locations around the Internet • Clients retrieve data from nearest duplicate server Streaming Video Over The Internet

  22. Key Areas • Video Compression • Application-Layer QoS Control • Continuous Media Distribution Services • Media Synchronization Mechanisms • Intra-stream Synchronization • Inter-stream Synchronization • Inter-object Synchronization • Protocols for Streaming Media Streaming Video Over The Internet

  23. Media Synchronization • Multimedia applications • Various integrated media streams that must be presented in a synchronized fashion • Media streams may lose synchronization after moving from the server to the client • Three layers of synchronization • Intra-stream synchronization • Maintain continuity of logical data units • Inter-stream synchronization • Maintain temporal relationships among different continuous media • e.g. movements of the lips of a speaker do not correspond to the presented audio • Inter-object synchronization • Relation with time-independent data, such as text, still image and streams • e.g. during a presentation, audio is commenting one slide while another slide is presented Streaming Video Over The Internet

  24. Media Synchronization Mechanisms • Delay in the network is unpredictable due to the best-effort nature of the Internet • Axes-based specifications, or time-stamping • At the source, a stream is time-stamped to keep temporal information within the stream and with respect to other streams • Mechanisms implemented on the end systems • Preventive mechanisms • Minimize latencies and jitters • i.e. disk-reading scheduling algorithms, network transport protocols, operating systems and synchronization schedulers • Corrective mechanisms • Recover synchronization in the presence of synchronization errors • i.e. Stream synchronization protocol (SSP) • Concept of ”intentional delay” Streaming Video Over The Internet

  25. Key Areas • Video Compression • Application-Layer QoS Control • Continuous Media Distribution Services • Media Synchronization Mechanisms • Protocols for Streaming Media • Transport Protocols • Session Control Protocols Streaming Video Over The Internet

  26. Protocols for Streaming Video • Data Plane • Compressed video/audio is packetized at the real-time transport protocol (RTP) layer • RTP packetized streams are then passed to the UDP/TCP layer and IP layer • IP packets are transported over the Internet • Control Plane • Real-time control protocol (RTCP) and real-time streaming protocol (RTSP) packets are multiplexed at the UDP/TCP layer • IP packets are transported over the Internet Streaming Video Over The Internet

  27. Transport Protocols • UDP and TCP provide basic transport functions • UDP and TCP • Multiplexing, Error control (e.g. checksum) • TCP only: • Retransmission, Congestion control, Flow control • UDP is typically employed as the transport protocol for video streams • No packet delivery guarrantees • Receiver needs to rely in upper layer to detect packet loss • RTP and RTCP run on top of UDP/TCP • RTP : does not guarantee QoS or reliable delivery • Time stamping • Sequence numbering • Payload type identification • Source identification • RTCP : designed to work in conjunction with RTP • QoS feedback • Participant identification • Control packets scaling • Inter-media synchronization • Minimal session control information Streaming Video Over The Internet

  28. Session Control Protocols • RTSP (Real-Time Streaming Protocol) • Support VCR-like control operations • (stop, pause/resume, fast forward, fast backward) • Choosing delivery channel (UDT, multicast UDT, TCP) • Establish and control streams of continuous audio and video media between the media servers and the clients • Media retrieval: ask server to setup a session to send the requested media data • Adding media to an existing session: server or client can notify each other about any additional media becoming available to the established session • SIP (Session Initiation Protocol) • Similar to RTSP • Support user mobility by proxying and redirecting request Streaming Video Over The Internet

  29. References • A. S. Tanenbaum, Computer Networks, 4th ed. New Jersey: Prentice Hall, 2002. • DapengWu, Hou, Y.T., WenwuZhu, Ya-Qin Zhang, Peha, J.M., "Streaming video over the Internet: approaches and directions“,Circuits and Systems for Video Technology, IEEE Transactions on , vol.11, no.3, pp.282-300, Mar 2001 • DapengWu, YiweiThomsHou,Ya-Qin Zhang, "Transporting real-time video over the Internet: challenges and approaches," Proceedings of the IEEE , vol.88, no.12, pp.1855-1877, Dec 2000 • DapengWu and Y. Thomas Hou and Jason Yao and Y. Thomas and Hou Jason Yao and H. Jonathan Chao, “Real-time Video over the Internet: A Big Picture”, IEEE NetWorld+Interop, 2000 Streaming Video Over The Internet

  30. Questions? Thank you! Streaming Video Over The Internet

More Related