GlobalMMCS DS-RT 2005 Tutorial

1. GlobalMMCS DS-RT 2005 Tutorial IEEE DS-RT 2005 Montreal Canada Oct. 9 2005 Geoffrey Fox CTO Anabas Corporation and Computer Science, Informatics, Physics Pervasive Technology Laboratories Indiana University Bloomington IN 47401 gcf@indiana.edu http://www.infomall.org

2. Collaboration and Web Services • Collaboration has • Mechanism to set up members (people, devices) of a “collaborative sessions” • Shared generic tools such as text chat, white boards, audio-video conferencing • Shared applications such as Web Pages, PowerPoint, Visualization, maps, (medical) instruments …. • b) and c) are “just shared objects” where objects could be Web Services but rarely are at moment • We can port objects to Web Services and build a general approach for making Web services collaborative • a) is a “Service” which is set up in many different ways (H323 SIP JXTA are standards supported by multiple implementations) – we should make it a WS

3. Shared Event Collaboration • All collaboration is about sharing events defining state changes • Audio/Video conferencing shares events specifying in compressed form audio or video • Shared display shares events corresponding to change in pixels of a frame buffer • Instant Messengers share updates to text message streams • Microsoft events for shared PowerPoint (file replicated between clients) as in Access Grid • Finite State Change NOT Finite State Machine architecture • Using Web services allows one to expose update events of all kinds as message streams • Need publish/subscribe approach to share messages (NB) plus • System to control “session” – who is collaborating and rules • XGSP is XML protocol for controlling collaboration building on H323 and SIP

4. R R R U U U WSViewer WSDisplay F F F F F F I I I I I I WebService WebService WebService O O O O O O WS Viewer WS Display WS Viewer WSDisplay Shared Input Port (Replicated WS) Collaboration Collaboration as a WSSet up Session with XGSP Master Event(Message)Service OtherParticipants

5. Shared Output Port Collaboration WSDL R U Application orContent source F F WSViewer WSDisplay I I O O Web Service Collaboration as a WSSet up Session with XGSP Web Service Message Interceptor Master WS Viewer WS Display Text Chat Whiteboard Multiple masters Event(Message)Service OtherParticipants WS Viewer WSDisplay

6. GlobalMMCS Design Principles • A unified, scalable, robust “overlay” network is needed to support AV and data group communication over heterogeneous networking environments. • A common AV signaling protocol has to be designed to support interactions between different AV collaboration endpoints. • A core conference control mechanism is required for establishing and managing the multi-point conference. • Finally, we’d like to use Web (Grid) -services to integrate collaboration communities in different technologies and to allow use of powerful services (like schedulers and meta-data catalogs) from general Grid/Web activities. • XGSP defines WSDL and other interoperability specifications

7. Global-MMCS Community Grid • This includes an open source protocol independent Web Service “MCU” which will scale to an arbitrary number of users and provides support for thousands of simultaneous users of collaboration services. • The function of A/V media server is distributed using NaradaBrokering architecture. • Media Servers mix and convert A/V streams • Open XGSP MCU based on the following open source projects • openh323 is basis of H323 Gateway • NIST SIP stack is basis of SIP Gateway • NaradaBrokering is open source messaging • Java Media Framework basis of Media Servers • Helix Communityhttp://www.helixcommunity.org for Real Media • http://www.globalmmcs.org open source release

8. Global-MMCS Service Architecture

9. Session Server XGSP-based Control Media Servers Filters NaradaBrokering All Messaging Admire SIP H323 Access Grid Native XGSP XGSP Web Service MCU Architecture Use Multiple Media servers to scale to many codecs and many versions of audio/video mixing WebServices High Performance (RTP)and XML/SOAP and .. NB Scales asdistributed Gateways convert to uniform XGSP Messaging NaradaBrokering

10. Break up into Web Services • Monolithic MCU becomes many different “Simple Services” • Session Control • Thumbnail “image” grabber • Audio Mixer • Video Mixer • Codec Conversion • Helix Real Streaming • PDA Conversion • H323/SIP Gateways • As independent can replicate particular services as needed • Codec conversion might require 20 services for 20 streams spread over 5 machines • 1000 simultaneous users could require: • 1 session controller, 1 audio mixer, 10 video mixers, 20 codec converters, 2 PDA converters and 20 NaradaBrokers • Support with a stream optimized Grid Farm in the sky • Future billion way “Video over IP” serving 3G Phones and home media centers/TV’s could require a lot of computing

11. GlobalMMCS and NaradaBrokering • All communication – both control and “binary” codecs are handled by NaradaBrokering • Control uses SOAP and codecs use RTP transport • Each stream is regarded as a “topic” for NB • Each RTP packet from this stream is regarded as an “event” for this topic • Can use replay and persistency support in NB to support archiving and late clients • Can build customized stream management to administer replay, and who gets what stream in what codec • NaradaBrokering supports unicast and multicast • Use firewall penetration and network monitoring services in NB to improve Q0S

12. Incorporating Support for Audio/Video Delivery into NaradaBrokering • Added support for an unreliable transport protocol, UDP • Implemented a fixed size (fast) topic (8 bytes). • Designed a new compact event with minimum headers. • Added support for legacy RTP clients (both unicast clients and multicast groups) • Improved the routing algorithm to handle real-time audio and video stream delivery.

13. Average delays per packet for 50 video-clients NaradaBrokering Avg=2.23 ms, JMF Avg=3.08 ms 60 NaradaBrokering-RTP JMF-RTP 50 40 30 Delay (Milliseconds) 20 10 0 0 200 400 600 800 1000 1200 1400 1600 1800 2000 Packet Number

14. Average jitter (std. dev) for 50 video clients. NaradaBrokering Avg=0.95 ms, JMF Avg=1.10 ms 8 NaradaBrokering-RTP JMF-RTP 7 6 5 4 Jitter (Milliseconds) 3 2 1 0 0 200 400 600 800 1000 1200 1400 1600 1800 2000 Packet Number

15. Polycom, Access Grid and RealVideo views of video-mixed streams using GlobalMMCS

16. Improved JMF Performance Fraction of CPU used versus number of received streams The CIF-size video sequence from a 30-second movie with a lot of motions is streamed to the clients. Each stream is encoded in H.261,and has average bandwidth of 400Kbps and 20 fps.

17. MPEG-4 vs. H.261 • We added MPEG4 video to Java Media Framework • Higher quality and flexible video sizes including distributed pixels

18. Coupled Diverse Streams • GlobalMMCS supports many diverse streams managed by “video system” • Different audio and video codecs • Shared display using video codecs (MPEG4 or H261) • Motion JPEG – stream of images to and from PDA • NaradaBrokering represents these and other collaborative streams just a “topics”; collaboration from multiple clients subscribing to a topic • Text Chat • Traditional lossless codec based shared display • White boards • Control streams • Streams can be linked to provide composite topics • eSports project linking video streams and real time annotation of any frame • Can rewind and choose any frame of a real-time stream

19. eSports Snapshot Master Video Annotation Whiteboard Collaborative Video Annotation Whiteboard Synchronized Replay of archived video and annotation

20. Two way RTP link One way RTP link Local Storage access Communication channel Topic eSports Architecture Replay/Archiving Service Stremaing Gateway NB Storage NB RTSP Server/Proxy NB Storage Replay/Archiving Service XGSP Clients Audio Video JMF Whiteboard eSports Player

21. Esports with sport movies

22. Integration of PDA, Cell phone and Desktop Grid Access NB Support for optimizedPDA Communication

23. PDA Download video as images I

24. PDA Download video as images II

25. GlobalMMCS Status/Futures I • 1. New Collaboration tools • Shared IDL (Visualization), PowerPoint, OpenOffice (Applications need a month or so more) • SVG game ( stable ) • Whiteboard ( stable ) • e-Sport ( prototype) • Jabber IM client ( prototype) • XGSP needs extension to support • 2. JMF Audio/Video client ( stable) • performance enhancement finished • new codec ( MPEG4 finished; try H.264) • support different platform ( Linux, Mac – Mac well developed but need to chase bug(s)) • support NAT/firewall transparently like Skype • screen codec for shared display ( prototype )

26. GlobalMMCS Status/Futures II • 3. Replay & Archive (prototype) • Replay Engine based on NaradaBroker Storage Service • XGSP-RTSP gateway • Extend RTSP and NaradaBrokering for Instant Replay • 4. Web Server Portal ( stable) • Standard calendar service ( iCalendar, vCalendar) • Flexible conference management • Need to package UI’s as portlets • 5. Conferencing Media Processing Service ( Stable) • Fix the bugs and add scheduling • Support new codec ( MPEG4 ) • 6. H.323 Gateway ( Stable) • Import it to Linux platform

27. GlobalMMCS Status/Futures III • 7. RealStreaming Gateway ( Stable ) • Import it to Linux • Support Mobile clients • 8. Global-MMCS deployment & test • Core performance measurements complete • Test under the setting of multiple NaradaBroker and NAT/Firewall • support deployment for AFRL, NASA, DOE portals • test with remote sites • 9. PDA Clients (prototype) • 10. Improved video codec-based shared display • 11. Scheduler of dynamic services sensitive to streaming bandwidth requirement as well as CPU use of codec conversion