1 / 23

A Multi-Site Virtual Cluster System for Wide Area Networks

A Multi-Site Virtual Cluster System for Wide Area Networks. Takahiro Hirofuchi, Takeshi Yokoi, Tadashi Ebara, Yusuke Tanimura, Hirotaka Ogawa, Hidetomo Nakada, Yoshio Tanaka, and Satoshi Sekiguchi National Institute of Advanced Industrial Science and Technology (AIST). What is Virtual Cluster?.

ruby-phelps
Download Presentation

A Multi-Site Virtual Cluster System for Wide Area Networks

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. A Multi-Site Virtual Cluster System for Wide Area Networks Takahiro Hirofuchi, Takeshi Yokoi, Tadashi Ebara, Yusuke Tanimura, Hirotaka Ogawa, Hidetomo Nakada, Yoshio Tanaka, and Satoshi Sekiguchi National Institute of Advanced Industrial Science and Technology (AIST) LASCO08

  2. What is Virtual Cluster? • Virtualization • Virtual Machine, SAN, VLAN • Cloud Computing • Get computing powers on demand • Virtualized Datacenters, Amazon EC2 (1VM = $0.1/hour) • Virtual cluster • Large-scale computing environment composed of VMs • Allow quick and easy deployment of many virtual nodes • Share physical clusters for multiple applications Application Scientific application Web 2.0 services Emulation testbed Application LASCO08

  3. Background • Our virtual cluster system • Virtualization toolkitfor physical clusters • Easy to install, low cost • Allow users to customize virtualization • Problem • Physical resource limitation at a single site • Scalability and flexibility • => Use multi-site, many-site resources • Exponential increase of management cost • How to manage widely-distributed virtual nodes? LASCO08

  4. Research Focus • Multi-site virtual cluster • Solve large-scale deployment and configuration issues of distributed virtual nodes • Break resource limitations at a single site • Small management cost, easy-to-use UI • Quick deployment and easy customization • Contribution • Design and implementation • Evaluation in WAN environments • The minimum WAN traffic for management • Quick configuration time like physical clusters LASCO08

  5. Scenario Comparison(Single-Site Virtualization) • Software deployment and configuration are very tiresome work! • Manage many virtual nodes distributed in many sites • Update and customize them day by day Application Application Site B Site A LASCO08

  6. Scenario Comparison(Multi-Site Cluster Virtualization) • Software deployment and configuration are performed through the view of a single cluster • Many virtual nodes at many sites can be seen like one physical cluster. • Easy management like a local single cluster Application Application Deployment and configuration Scientific application Web 2.0 services Emulation testbed Site B Site A LASCO08

  7. Requirement • Powerful management support for large-scale virtual nodes • Scalable and robust • Flexible customization of node by node • Support heterogeneous resources in many sites • Intuitive administration interface • Allow quick deployment, minimize administrative network traffic LASCO08

  8. Design Criteria • Exploit an existing cluster management system for distributed virtual nodes • Designed for physical clusters • Virtual node management (add, delete, and configure) • Integrate distributed VMs by Ethernet VPN • Bridge internal networks of single-site virtual clusters • Transparency for admins and users • Transparent package caching at each site • Package-based software installer • Quick installation and reconfiguration • Flexible customizability LASCO08

  9. Cluster Management System(NPACI Rocks) Installation Configuration Operation Frontend Node Compute Node Private Network （LAN） • Automatic node install by PXE network boot • Cluster-wide application package called “Roll” • Parallel command execution tools (e.g., cluster-fork, tentakel) • Node status monitoring (e.g., Ganglia) • Automatic recovery from node down Node Config. Database LASCO08

  10. Virtual Cluster Management Installation Configuration Operation Virtual Frontend Virtual Node Private Network Node Config. Database Allocate a group of VMs Isolate them by VLAN Ethernet VPN LASCO08

  11. Virtual Node (Re-)Configuration(1) Virtual Frontend Virtual Node Ethernet VPN Node Config. DB NodeA：　App P＆Conf X NodeB：　App Q＆Conf Y NodeC：　App R＆Conf Z … PXE Boot Installation Package ＆Configuration Download LASCO08

  12. Virtual Node (Re-)Configuration(2) Virtual Frontend Virtual Node Ethernet VPN Huge Download Traffic Packages x #Nodes Node Config. DB NodeA：　App P＆Conf X NodeB：　App Q＆Conf Y NodeC：　App R＆Conf Z … PXE Boot Installation Package ＆Configuration Download LASCO08

  13. Virtual Node (Re-)Configurationwith Package Caching Virtual Frontend Virtual Node Ethernet VPN Node Config. DB NodeA：　App P＆Conf X NodeB：　App Q＆Conf Y NodeC：　App R＆Conf Z … Package Cache Server Minimize admin. traffic for node configuration LASCO08

  14. PrototypeImplementation • REST API • Create a virtual cluster • Add/Delete a VM • Start/Stop a VPN • Reservation portal • Site resource monitoring • Automatic reservation • NPACI Rocks 4.2 • VMware Server • VLAN, iSCSI • OpenVPN 2.0 • Squid 3.0 LASCO08

  15. Evaluation • Current status • Distributed VM configuration • Parallel command line tools • Node status monitoring • Points of evaluation • Feasibility • WAN environment • Large number of virtual nodes • Reconfiguration time • Network traffic Work fine ! LASCO08

  16. Experiment Setting Node Config. DB 10 or 134 nodes, reconfigured with 900MByte packages 16 nodes AMD Opteron 244, 3GB Mem, Gb Eth x2 AMD Opteron 246, 6GB Mem, Gb Eth x2 LASCO08

  17. Experiment without Package Caching Installation Time over WAN (10 Nodes) Even for only 10 nodes,it is too slow due to WAN traffic. (s) RTT (ms) LASCO08

  18. Experiment with Package Caching VPN Throughput • RTT 20ms 8MB/s The package cache server reduces download traffic to a unique set of installed packages (800MB). 22 min Cache Server Throughput 80MB/s LASCO08

  19. Summary • Minimized WAN traffic • A unique set of installed packages • Pre-caching is possible • Rapid configuration of virtual nodes • 20 – 30 min / 134 nodes (RTT20ms) • Not so far from local physical nodes • 15 – 25 min / 134 nodes LASCO08

  20. Related Work • Virtual cluster + VPN • Virtuoso [Northwestern Univ., 2003] • Remote VM access via VPN • VioCluster [Purdue Univ., 2005] • Automatic shrink/expand by VPN topologies • Virtual Workspace • Globus-based virtual cluster services • Amazon EC2 • Xen-image-based deployment • PlanetLab • Distributed network emulation testbed LASCO08

  21. Conclusion • Large-scale application environment • Scalability beyond a single site • Smart management framework • Flexible configuration • Multi-site virtual cluster • Exploit a powerful cluster manager • Single cluster view by Ethernet VPN • Package caching at each site • Evaluation • Quick configuration of distributed VMs • Minimized WAN traffic LASCO08

  22. Future Work • Ongoing Project • Xen support • VM migration over WAN • Diskless boot • Other OS support • Availability • http://code.google.com/p/grivon LASCO08

  23. Node ID Time Node Installation Progress Generating node configuration scripts. Download an Installer image Download packages 22 min LASCO08

More Related