CS 414 – Multimedia Systems Design Lecture 29 – Media Server (Part 3)

1. CS 414 – Multimedia Systems DesignLecture 29 – Media Server (Part 3) Klara Nahrstedt Spring 2010 CS 414 - Spring 2010

2. Administrative MP3 – deadline Monday, April 12, 5-7pm demonstrations in 216 Please, sign up for demonstration on Monday, April 12 in class. Note: The MP3 demonstration must happen among four computers!!! CS 414 - Spring 2010

3. Some Interesting Facts • DBMS2.com Source (May 2009) • Facebook had 400 terabytes of disks managed by Hadoop/Hive with an approx. 6:1 compression ratio • Facebook’s Hadoop/Hive system ingests 15 terabytes of new data per day • Facebook had 610 Hadoop nodes (in May 2009) running in a single cluster and was heading for 1000 • Yahoo had 2000 nodes (in May 2009) and was heading for 4000 CS 414 - Spring 2010

4. Outline • Disk Scheduling • SCAN-EDF • Group Sweeping • Mixed Scheduling • Admission Control • File System Metadata/Indexing • Block Size Issues CS 414 - Spring 2010

5. Disk Scheduling Policies • Goal of Scheduling in Traditional Disk Management • Reduce cost of seek time • Achieve high throughput • Provide fair disk access • Goal of Scheduling in Multimedia Disk Management • Meet deadline of all time-critical tasks • Keep necessary buffer requirements low • Serve many streams concurrently • Find balance between time constraints and efficiency CS 414 - Spring 2010

6. EDF (Earliest Deadline First) Disk Scheduling • Each disk block request is tagged with deadline • Policy: • Schedule disk block request with earliest deadline • Excessive seek time – high overhead • Pure EDF must be adapted or combined with file system strategies CS 414 - Spring 2010

7. EDF Example Note: Consider that block number Implicitly encapsulates the disk track number CS 414 - Spring 2010

8. SCAN-EDF Scheduling Algorithm • Combination of SCAN and EDF algorithms • Each disk block request tagged with augmented deadline • Add to each deadline perturbation • Policy: • SCAN-EDF chooses the earliest deadline • If requests with same deadline, then choose request according to scan direction CS 414 - Spring 2010

9. Implementation of SCAN-EDF • Notation: • Dibe deadline of disk block request ‘i’ • Nibe track (block) position on disk • Nmax be maximum number of disk tracks • Deadline Modification: • Di + f(Ni) • f(Ni) converts track number of ‘i’ into a small perturbation of deadline • Perturbation small enough so that • Di + f(Ni) ≤ Dj + f(Nj) for Di ≤ Dj • Possible f(Ni) = Ni/Nmax CS 414 - Spring 2010

10. SCAN EDF Example (Nmax = 100) CS 414 - Spring 2010

11. Enhanced SCAN-EDF (1) Head Moves Upwards • Use more accurate perturbation of deadline • Consider • Actual track position of disk head ‘N’ • Nmax– max number of disk tracks • Ni– next track to be considered CS 414 - Spring 2010

12. Enhanced SCAN-EDF (2) • Algorithm: • If head moves upwards (towards Nmax), then • (a) • (b) CS 414 - Spring 2010

13. Enhanced SCAN-EDF (3) If head moves downwards (towards 1), then (a) (b) CS 414 - Spring 2010

14. Group Sweeping Algorithms • Policy: • Each Request consists of (Deadline, Block Number ) • Disk Block Requests served in cycles • Requests served in Round-Robin manner • In one cycle, requests divided into groups • As we retrieve blocks, we may need smoothing buffers to ensure continuity CS 414 - Spring 2010

15. Group Sweeping Example CS 414 - Spring 2010

16. Mixed Scheduling (uses SSTF – Shortest Seek Time First) Example of SSTF CS 414 - Spring 2010

17. Mixed Scheduling SSTF (Shortest Seek Time First) + Balanced Strategy CS 414 - Spring 2010

18. Admission Control Client 1 retrieves K1 blocks in one round Client 2 retrieves K2 blocks Server Client 3 retrieves K3 blocks Client 4 retrieves K4 blocks CS 414 - Spring 2010

19. Admission Control • Disk block requests are timed • Media server must determine • admit a stream • serve (schedule) a stream without having negative effect on other streams already serviced. • Deterministic Guarantees • Admission control considers worst case scenario when admitting new stream • Constrained Disk Placement Example: M - size of blocks, G – size of gabs, rdt– data transfer of disk CS 414 - Spring 2010

20. Admission Control α – overhead switching from one round (‘j-1’) To another round (j), and the transmitting the First block of the ‘j’ round β – transmission time of (Ki-1) blocks in ‘j’ round, i=1,..4 Ki – number of blocks retrieved by client ‘i’ ηi – Block granularity retrieved for client ‘i’ Ri – playback rates of client ‘i’ CS 414 - Spring 2010

21. Admission Control • Statistical Guarantees • Deadlines are guaranteed with certain probability • Admission control considers statistical behavior of the disk system while admitting new stream (average performance) • Best effort Service • No guarantees CS 414 - Spring 2010

22. Multimedia File Systems • Real-time Characteristics • Read operation must be executed before well-defined deadline with small jitter • Additional buffers smooth data • File Size • Can be very large even those compressed • Files larger than 232 bytes • Multiple Correlated Data Streams • Retrieval of a movie requires processing and synch of audio and video streams CS 414 - Spring 2010

23. Placement of Mapping Tables • Fundamental Issue: keep track of which disk blocks belong to each file (I-nodes in UNIX) • For continuous files/contiguous placement • don’t need maps • For scattered files • Need maps • Linked lists (inefficient for multimedia files) • File allocation tables (FAT) CS 414 - Spring 2010

24. Indexing and FAT Higher Level Index Table Per File File Allocation Table Block I1 Location PTR I Frame Block I2 Location PTR P Frame Block I3 Location PTR B Frame Block P11 Location PTR P Frame Block P12 Location PTR ……….. Block B1 Location PTR Block P21 Location PTR Block P22 Location PTR ………….. CS 414 - Spring 2010

25. Constant and Real-time Retrieval of MM Data • Retrieve index in real-time • Retrieve block information from FAT • Retrieve data from disk in real-time • Real-time playback • Implement linked list • Random seek (Fast Forward, Rewind) • Implement indexing • MM File Maps • include metadata about MM objects: creator of video, sync info CS 414 - Spring 2010

26. Fast Forward and Rewind(Implementation) • Play back media at higher rate • Not practical solution • Continue playback at normal rate, but skip frames • Define skip steps, e.g. skip every 3rd, or 5th frame • Be careful about interdependencies within MPEG frames • Approaches for FF: • Create a separate and highly compressed file • Categorize each frame as relevant or irrelevant • Intelligent arrangement of blocks for FF CS 414 - Spring 2010

27. Block Size Issues in File Organization Movie Time line Frame index ……… A A A V V V V V V T T CS 414 - Spring 2010 • Small Block Sizes • Use smaller block sizes, smaller than average frame size • Organization Strategy: Constant Time Length • Need Metadata structure, called Frame Index • Frame means a time frame within a movie • Under the time frame read all blocks (audio, video, text) belonging to this time frame

28. Block Size Issues Block Index A V A V A V A V V V • Large Block Size • Use large blocks (e.g., 256 KB) which include multiple audio/video/text frames • Organization Strategy: Constant Data Length • Need Metadata structure, called Block Index • Each block contains multiple movie frames CS 414 - Spring 2010

29. Tradeoffs Frame index : needs large RAM usage while movie is playing, however little disk wastage Block index (if frames are not split across blocks): need low RAM usage, but major disk wastage – internal disk fragmentation Block index(if frames are split across blocks): need low Ram usage, no disk wastage, extra seek times CS 414 - Spring 2010

30. Conclusion The data placement, scheduling, block size decisions are very important for any media server design and implementation. Still need to consider caching – next lecture CS 414 - Spring 2010