An SLA-aware load balancing scheme for cloud datacenters

1. An SLA-aware load balancing scheme for cloud datacenters 指導教授：王國禎 學生：黎中誠 國立交通大學資訊工程系 行動計算與寬頻網路實驗室

2. Outlines • Introduction • Related work • Proposed two-level decentralized load balancer architecture • Proposed SLA-aware load balancing scheme • Experimental environment • Experimental results • Conclusion • Future work • References

3. Introduction • Cloud computing is the delivery of computing as a service rather than a product, whereby shared resources, software, and information are provided to computers and other devices as a utility (like the electricity grid) over a network (typically the Internet) [1] The cloud scales: Amazon EC2 growth [2]

4. Introduction (cont.) • The load of a cloud computing system is highly dynamic • Different users may require different services, and it may lead to load unbalance between the virtual machines (VMs)in a datacenter • User requests are sending to a load balancer and the load balancer then forward them to the appropriate VMs for processing in cloud datacenters

5. Introduction (cont.) • A service-level agreement (SLA) is a part of a service contract where the level of service is formally defined [3] • The SLA will typically have a technical definition in terms of response time, throughput, or similar measurable details [3]

6. Introduction (cont.) • The following figure illustrates a classic load balancing architecture in a cloud computing environment [4]

7. Introduction (cont.) • We focus on two issues • Load balancing control [5] • Centralized or decentralized • Three-level centralized load balancer [6] • Two-level decentralized load balancer [5] • Two-level decentralized SLA-aware load balancer (tldlb, proposed) • Load balancing algorithm [5] • Static or dynamic • Weighted round-robin (wrr) [4] • Dynamic weighted round-robin (dwrr, proposed)

8. Related work • Wang et al. proposed a three-level load balancer architecture [6], as shown below • The first level has a Request Manager that is used to assign a task to a suitable Service Manager • The second level includes several Service Managers that are used to divide tasks into some logical independent subtasks • The third level includes Service Nodes that are used to execute subtasks

9. Related work Three-level centralized load balancer architecture [6]

10. Related work (cont.) Structure of decentralized load balancer [5] • Rajavel proposed a two-level decentralized load balancer [5] • The Meta-Scheduler are connected to each other by network connections and each Meta-Scheduler in turn is connected to a group of clusters (resources)

11. Related work (cont.) Structure of Meta-Scheduler [5] • The structure of Meta-Scheduler works as follows: • User submits their jobs to aMeta-Scheduler, and the jobs are stored in the queue of a request handler • Dispatch Manager obtains the submitted job periodically from the queue

12. Related work (cont.) • Load Balancer will perform the load balancing by exploiting the information gathered from Load Monitor and Information Manager • Information Manager will query Load Monitor and sends the host load information to the Load Balancer • Transfer Manager gives permission rights for the execution of a given job to a remote host • Execution Manager will keep updating the job status to the Dispatch Manager

13. Related work (cont.) Qualitative comparison of different load balancing architectures

14. Related work (cont.) The weighted round-robin (wrr) scheduling algorithm will assign a fixed weight to each VM, an integer value that indicates the VM’s processing capacity [4] The capacity based scheduling algorithm [7] monitors the resources of each VM and distributes more requests to the VM which has more remaining resources Artificial neural network (ANN) based load predictor has the lowest prediction errors [8], and the proposed dwrr(Dynamic weighted round-robin) combine ANN and SLA requirements together to avoid SLA violation

15. Related work (cont.) Qualitative comparison of different load balancing algorithm

16. Proposed two-level decentralized load balancer architecture We propose a new architecture, called SLA-aware two-level decentralized load balancer (tldlb), to support dynamic load balancing in cloud data center There are two levels in our load balancer design: global load balancer and local load balancer Each global load balancer connected to a SLA-awarelocal load balancer that forms a virtual zone

17. Proposed two-level decentralized load balancer architecture (cont.) • The proposed load balancer architecture is shown as follows:

18. Proposed two-level decentralized load balancer architecture (cont.) • Local load balancer • Monitoring local VMs load which are in the same virtual zone • Choosing a proper VM to handle a request by Request Scheduler • Global load balancer • Global balancers are connected to each other via P2P connections • When the responsible zone of a global load balancer is overloaded, it will select another virtual zone to share the load by forwarding requests to the zone

19. Proposed two-level decentralized load balancer architecture (cont.) • The local load balancer architecture is shown as follows:

20. Proposed two-level decentralized load balancer architecture (cont.) • Request Handler • This module receives user requests and forward them to the Request Scheduler module • Request Scheduler • This module assigns user requests from Request Handler to correspond VMs based on the weights from the Weight Adjustment module • Load Monitor • It will monitor current four load metrics (CPU, MEM, NET, Disk I/O utilization) of each VM

21. Proposed two-level decentralized load balancer architecture (cont.) • History Storage • The load data collected by Load Monitor will be stored in this module • These load data can support the Load Prediction to predict the load at next time slot • Load Prediction • This module uses load history of each VM, which is stored in the History Storage module to predict the future load of each VM • The load prediction results (NIi) will send to Weight Adjustment module

22. Proposed two-level decentralized load balancer architecture (cont.) • SLA Engine • This module will record the response time of each requestand check of the request satisfies its SLA requirements • Weight Adjustment • This module will adjust the weight of each virtual machine according to the capacity index (CIi) from Load Monitor and the neural index (NIi) from Load Prediction

23. Proposed two-level decentralized load balancer architecture (cont.) • Spare VM pool • There are some suspended VMs in the spare VM pool • When active VMs can’t handle incoming requests to meet the SLA requirements, Request Handler will wake up some VMs to service requests

24. Proposed SLA-aware load balancing scheme • We propose a novel load balancing algorithm, called dwrr (dynamic weighted round-robin), to dispatch requests to different VMs • We give each active VM a weight according to the capacity index (CIi) from Load Monitor and the neural index (NIi) from Load Prediction • The Request Scheduler module distributes the requests to active VMs by these weights

25. Proposed SLA-aware load balancing scheme (cont.) • Capacity index (CI) • The critical resource is different when cloud datacenter provides varies services • The load Monitor module collects four load metrics, utilizations of CPU, memory, network bandwidth, and disk I/O

26. Proposed SLA-aware load balancing scheme (cont.) • Neural index (NI) • Artificial Neural Network (ANN) has the ability to do optimization and prediction • We use the delta learning rule in our ANN design • In our ANN design, we consider avoiding SLA violations, such as the response time violation

27. Proposed SLA-aware load balancing scheme (cont.) We use following formula to determine the weight of each VM

28. Proposed SLA-aware load balancing scheme (cont.) • The following figure shows schematic representation of the artificial neural network model for deriving the neural index

29. Proposed SLA-aware load balancing scheme (cont.) • We train the weights in the hidden layer via delta learning rule

30. Proposed SLA-aware load balancing scheme (cont.) • Transfer function: (log-sigmoid) • : pre-reaction rate • :response time specified in the SLA • : actual response time • : learning rate • : input

31. Proposed SLA-aware load balancing scheme (cont.) Capacity index Neural index Weight

32. Proposed SLA-aware load balancing scheme (cont.)

33. Experimental environment We built a small test bed that include three active VMs (VM1, VM2, VM3) with different capability and two spare VM (VMs1, VMs2), which running in an apache web server in a virtual zone for our experiment Requests are based on a real web service We use the modified apache benchmark (ab) to collect real web traffic

34. Experimental environment (cont.) We set up an experimental environment as shown below

35. Experimental environment (cont.) Our experimental environment setup andrelated parametersareshown in the following table

36. Experimental environment (cont.) VM configuration

37. Experimental results • We compare four scheduling algorithms • Response time specified in the SLA: 2000 ms

38. Experimental results (cont.) • The proposeddwrr is 1.86 times faster than wrr, 1.49 times faster than capacity based, and 1.21 times faster than ANN scheduling algorithms in terms of average response time • Response time specified in the SLA: 2000 ms

39. Experimental results (cont.) • We compare four scheduling algorithms • Response time specified in the SLA: 1000 ms

40. Experimental results (cont.) • We compare four scheduling algorithms • Response time specified in the SLA: 432 ms

41. Experimental results (cont.) The proposed tldlb can reduce the SLA violation rate by activating VMs in the spare VM pool module in time

42. Conclusions We have presented SLA-aware decentralized load balancer architecture We combine of VM load metrics monitoring (CPU, memory, network, disk I/O utilization) and neural network to adjust the weight of each VM Experimental results support that dwrr is 1.86 times faster than wrr, 1.49 times faster than capacity based, and 1.21 times faster than ANN scheduling algorithms in terms of average response time in the limited resources

43. Conclusions (cont.) The experiment results support that our proposed dwrr algorithm has faster response time, which means we can handle more requests per second This load balancing algorithm can avoid SLA violations via activating VMs in the spare VM pool Since our scheduling algorithm is simple and efficient, it is well-suited for cloud computing environments to service a large number of requests with short response time

44. Future work Implementing the global balancer and do some experiments about user requests redirection performance Deploying our load balancer to the real cloud datacenter

45. References [1] Cloud computing – Wiki http://en.wikipedia.org/wiki/Cloud_computing. [2] Average daily instance launch counts http://jackofallclouds.com/wp-content/uploads/2010/12/cloudkick_yearly_ec2_launch_counts.png. [3] Service-level agreement – Wiki http://en.wikipedia.org/wiki/Service-level_agreement. [4] Linux Virtual Server http://www.linuxvirtualserver.org. [5] R. Rajavel, “De-Centralized Load Balancing for the Computational Grid environment,”in Proceeding of International Conference on Communication and Computational Intelligence (INCOCCI), pp. 419-424, Dec. 2010. [6] S. C. Wang, K. Q. Yan, W. P. Liao, and S. S. Wang, “Towards a Load Balancing in a three-level cloud computing network,” in Proceeding of IEEE International Conference on Computer Science and Information Technology (ICCSIT), vol. 1, pp. 108 - 113, Jul. 2010. [7] C. C. Li, and K. C. Wang, “SLA-aware Load Balancing for Cloud Data Centers,” report, 2012 [8] V. Nae, A. Iosup, R. Prodan, “Dynamic Resource Provisioning in Massively Multiplayer Online Games,” IEEE Transactions on Parallel and Distributed Systems, vol. 22, no. 3, pp. 380-395, Mar. 2011.