Ganeti Open Source Xen Cluster Management

1. GanetiOpen Source Xen Cluster Management Carlos Rodriguez HPCf

2. Virtualization • Abstraction of computer resources • Virtual machines (VM) a software implementation to emulate a real machine. • VM Operating systems are “fooled” as if they were running on a physical machine (kind of) • Makes possible partitioning and sharing of resources.

3. Why Virtual? • Utilization of hardware resources (multi-cores) • Concurrently (same hardware) run conflicting environments (OSes) • Isolate workloads and avoid attacks • Fast provisioning & deployment

4. Full Virtualization Guest OS doesn’t realize it is running on a virtual machine Virtualization software creates an interface where guest OS can run unmodified Virtual box, Qemu, VMWare Para-virtualization Guest OS is aware of virtualization layer Guest OS must be ported to a para-API “improved” efficiency XEN Paradigms of Virtualization

5. What is Xen • neXt gENeration virtualization • Virtualization Hypervisor • Allows several guest operating systems to execute on the same computer hardware concurrently • Supports x86, x86-64, Itanium, Power PC, and ARM • Paravirtualized & hardware assisted • Developed bt Cambridge University acquired by Cytrix

6. Xen Lego

7. Xen Hypervisor • Thin, privileged abstraction layer between the hardware and operating systems • Defines the virtual machine that guest domains see instead of physical hardware • Grants portions of the full physical resources to each guest • Exports simplified devices to guests • Modifies hard-to-virtualize portions of x86 arch. • Enforces isolation among guests Running Xen, p. 48-49

8. Why? Open Source Paravirtualization (“efficiency”) & HA Early adopters standardized in linux kernel humble hardware requirements Works with OLD hardware Why not? Commercially controlled Management Management Management Management Management Why or why not Xen

9. Cluster Buy new hardware Buy a SAN Buy a Cytrix licensed management product. Cluster Use discarded machines Use network mirroring with drbd Use Ganeti Ready to virtualize with XenNow what?

10. What is Ganeti • Ganeti is a cluster virtual server management software tool • Poor man solution using existing open source tools without any shared storage (SAN) requirements • Works with Xen or KVM • Uses DRBD for data replication • Fail tolerant • Floating master node

11. Ganeti Lego Physical Hardware Dom0Ganeti tools DomUApps DomUApps DRBD LVM Xen Hypervisor Ganeti tools Comm. Physical Hardware Dom0Ganeti tools DomUApps DomUApps DRBD LVM Xen Hypervisor

12. Nodes and VMs need to be resolvable FQDN (/etc/hosts) Single NIC is required Password-less ssh Floating master node IP Virtual interfaces for VMs Ganeti networking gnode1.hpcf.upr.edu136.145.54.134 client VIF1 VIF2 VIF3 ganeti.hpcf.upr.edu 136.145.54.133 gnode2.hpcf.upr.edu136.145.54.135 VIF1 VIF2 VIF3 gnode3.hpcf.upr.edu136.145.54.136 VIF1 VIF2 VIF3

13. gnt-instance activate-disks - Activate an instance's disks add - Creates and adds a new instance to the cluster add-tags - Add tags to the given instance batch-create - Create a bunch of instances based on specs in the file. console - Opens a console on the specified instance deactivate-disks - Deactivate an instance's disks failover - Stops the instance and starts it on the backup node, using the remote mirror (only for instances of type drbd) grow-disk - Grow an instance's disk info - Show information on the specified instance(s) list - Lists the instances and their status. The available fields are (see the man page for details): status, oper_state, oper_ram, name, os, pnode, snodes, admin_state, admin_ram, disk_template, ip, mac, bridge, sda_size, sdb_size, vcpus, serial_no, hypervisor. The default field list is (in order): name, hypervisor, os, pnode, status, oper_ram. list-tags - List the tags of the given instance migrate - Migrate instance to its secondary node (only for instances of type drbd) modify - Alters the parameters of an instance reboot - Reboots an instance reinstall - Reinstall a stopped instance remove - Shuts down the instance and removes it remove-tags - Remove tags from given instance rename - Rename the instance replace-disks - Replaces all disks for the instance shutdown - Stops an instance startup - Starts an instance

14. gnt-os • List or diagnose OSes available for instances • OS are added by creating a script with an api

15. gnt-cluster add-tags - Add tags to the cluster command - Runs a command on all (or only some) nodes copyfile - Copies a file to all (or only some) nodes destroy - Destroy cluster getmaster - Shows the cluster master info - Show cluster configuration init - Initialises a new cluster configuration list-tags - List the tags of the cluster masterfailover - Makes the current node the master modify - Alters the parameters of the cluster queue - Change queue properties redist-conf - Forces a push of the configuration file and ssconf files to the nodes in the cluster remove-tags - Remove tags from the cluster rename - Renames the cluster repair-disk-sizes - Updates mismatches in recorded disk sizes search-tags - Searches the tags on all objects on the cluster for a given pattern (regex) verify - Does a check on the cluster configuration verify-disks - Does a check on the cluster disk status version - Shows the cluster version

16. gnt-backup export - Exports an instance to an image import - Imports an instance from an exported image list - Lists instance exports available in the ganeti cluster remove - Remove exports of named instance from the filesystem.

17. Ganeti caveats • NO High Availabilty - Split brain is a concern since it doesn’t use cluster aware filesystem or locking mechanism (STONITH) • Don’t try it on Ubuntu karmic koala - Ubuntu turn their focus away from Xen to KVM and their cloud computing solution (Xen is broken!!) • OS scripts are (in my opinion) too much work.

18. Conclusions • Good alternative to make use of older hardware • Provisioning and managing is fairly easy • Great for poor people • Future - KVM Amazon EC2 compatible