1 / 29

Hierarchical Model-based Autonomic Control of Software Systems

Hierarchical Model-based Autonomic Control of Software Systems. Marin Litoiu, IBM CAS Toronto Murray Woodside, Tao Zheng, Carleton University. Outline. Motivation Hierarchical Control Performance Models Conclusions. Client. Client. Data Server. Data Server. Web Server. Web Server.

autumn-lancaster
Download Presentation

Hierarchical Model-based Autonomic Control of Software Systems

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. Hierarchical Model-based Autonomic Control of Software Systems Marin Litoiu, IBM CAS Toronto Murray Woodside, Tao Zheng, Carleton University

  2. Outline • Motivation • Hierarchical Control • Performance Models • Conclusions DEAS 05, St. Louis

  3. Client Client Data Server Data Server Web Server Web Server App Server App Server Client Client A Typical Deployment: Data Centres SLAs component Application Data Centre DEAS 05, St. Louis

  4. Response time time Self - optimization • Automated allocation of software and hardware resources to accomplish a performance goal by optimizing a cost function in the presence of • Workload variations • Perturbations • Change in the environment • Aims at • Reducing the cost of ownership • Improve the QoS (dependability) DEAS 05, St. Louis

  5. Outline • Motivation • Hierarchical Control • Performance Models • Conclusions DEAS 05, St. Louis

  6. Hierarchical Control(1) SLO SLO • Level 1: Component tuning • Level 2: Application tuning • Level 3: Provisioning DEAS 05, St. Louis

  7. Hierarchical Control(2) DEAS 05, St. Louis

  8. Hierarchical Control(3) • The Controller • Monitor the managed component performance metrics and the input workload and setpoints (SLOs) • Use the performance model to estimate future metrics and future adjustments of controlled parameters • If the future workloads cannot be accommodated by local adjustments, alert the upper level; otherwise perform the local adjustments. DEAS 05, St. Louis

  9. Hierarchical Control(4): Advantages “When in doubt, mumble; when in trouble, delegate; when in charge, ponder.” • The targeted systems are hierarchical • Structure is hierarchical (containment) • Goals are hierarchical • Authority is hierarchical • Provides several homeostatic control levers: If the 1st one fails, engage the 2nd, if the 2nd one fails, engage the 3rd… • Solves time scale and scope issues • Reduces cognition, control and communication complexity DEAS 05, St. Louis

  10. Agenda • Motivation • Hierarchical control • Performance Models • Conclusions DEAS 05, St. Louis

  11. The Role of Performance Models • “All models are wrong, some models are useful.” • Prediction (forecast) role: tell what happens in the future • if the workload increases by 100 users, the response time will increase by 5s • Estimation role: what happens now • If I increase the number of threads, servers, alter the software architecture, what is the estimated change in performance • Problem determination role • Where is the bottleneck DEAS 05, St. Louis

  12. Client Data Server Web Server App Server Client Queuing Network Models Di=service demand; X=throughput Ui=utilization of device i Qi=queue length at device i X=N/(R) Ui=X *Di DEAS 05, St. Louis

  13. Client Client Data Server CPU, Disk Web Server CPU, Disk App Server CPU, Disk Client CPU, Disk Layered Queuing Models (LQM) • Layered Queuing Models (LQM) are analytic performance models that • Extend Queuing Network Models (QNMs) • Model queuing at software components: threading and data connection pools, locks and critical sections • Model multiple classes of requests • LQM Structure • Software resource interactions: synchronous, asynchronous, forward call • Demands at hardware resources for each class of request, one user per class in the system • Queuing centers: CPU, DISK, network, threading and data connections pools… Layer 1 Layer 0 DEAS 05, St. Louis

  14. Linearized Dynamic Models • Linear regression models x(k)=Ax(k-1)+Bu(k) y(k)=Cx(k)+Du(k) x,u, y - vectors A,B,C,D- matrixes • Consider multiple input, output and state variables • Take into account the tendencies • Advantage • Take advantage of the controller design techniques from system control • Capture the transient behaviour Disadvantage • Assume the system is linear • The matrixes A,B,C,D are experimentally identified • Any change in the system will invalidate the model R(N)=C*N DEAS 05, St. Louis

  15. Threshold and Policy Models • Decision is part of the Model • Monitor an output variable (utilization, queue length..) • Increase/decrease a state variable • “ If the number of users increases by 10, increase the number of threads by 5” • “ if the utilization is greater than 50%, then provision a new server” • More complex models • Monitor more output variables • Increase/decrease one output variable • Advantages • Very simple • Very fast • Disadvantages • Not globally optimal, sometimes not even locally optimal • Not able to handle changes in the system DEAS 05, St. Louis

  16. Tradeoffs.……. • Layered Queuing Models • Model non-linearities across wide domains • Appropriate at application and system level • Work with mean values • Difficult to obtain service times per individual transactions( Kalman filters seem to help) • Dynamic Models • Model transitory behaviour • Appropriate at component level • For mean values, can be deduced from LQMs • In general, built experimentally (hard) • Enable System Control • Threshold and Policy Models • Can capture rules of thumb and domain experience • Can be deduced from the queuing or dynamic models • Appropriate at any level, as a default model • Simple and fast • Hard to maintain… DEAS 05, St. Louis

  17. Conclusions • Hierarchical control • Solves the problem of timescale and scope • Provide flexibility of choice of control algorithms • Supports accelerated decision making by LQM at upper levels • Self Optimization=Component tuning + Application Tuning (Load balancing) + Provisioning • Models for Self-optimization • Threshold or Policy Based Models • Linearized Dynamic Models • Network or Layered Queuing Models • Further Work: • End to end evaluation • Optimization • Stability DEAS 05, St. Louis

  18. Backup slides DEAS 05, St. Louis

  19. Model Builder Prediction: = previous estimate of x ŷ = prediction of observation y based on and the model, Feedback: z = new observation vector = K ( z - ŷ ) * DEAS 05, St. Louis

  20. Dynamic Models  link AC to System Control Automatic Control 1952: Bellman’s optimal control (dynamic programming) 1878: Maxwell's “On Governors” 1931: Black&Nyquist’s electronic negative feedback amplifier 1789: Watt’s fly-ball governor 1960: Kalman fillter Experimental AC Classic AC Modern AC Autonomic Computing 2001:AC 1970’s: Time Sharing OS 1978: Cerf et al. TCP/IP DEAS 05, St. Louis

  21. Solvers for LQMs • Algorithms • Layers of QNMs • The output of a lower lever is used as input for the upper level • Continue until a fix-point is reached • Public Solvers • Method of Layers (Rolia, Sevcik) • Based on Linearizer approximation algorithm • LQNS (Woodside ) • Based on approximate MVA • APERA (Litoiu) • Based on approximate MVA, two layers, on alphaworks Layer 2 Layer 1 Layer 0 (hardware) DEAS 05, St. Louis

  22. LQMs- Predicting the application response time • Depending on the class mix, response time varies widely even when the number of users is constant ( 800) • Each class may reach its maximum response time for a different workload mix • Workload mixes produce changes in the bottlenecks DEAS 05, St. Louis

  23. Monitored number of threads with HTTPConnection: Keep-Alive Estimated average number of threads Monitored number of threads with HTTPConnection:close LQMs- Predicting the Threading Level in WAS* • 100 clients ------------------------------- * From Litoiu M., "Migrating to Web services: a performance engineering approach," Journal of Software Maintenance and Evolution: Research and Practice, No 16, pp. 51-70, 2004. DEAS 05, St. Louis

  24. Dynamic Models  link AC to System Control Automatic Control 1952: Bellman’s optimal control (dynamic programming) 1878: Maxwell's “On Governors” 1931: Black&Nyquist’s electronic negative feedback amplifier 1789: Watt’s fly-ball governor 1960: Kalman fillter Experimental AC Classic AC Modern AC Autonomic Computing 2001:AC 1970’s: Time Sharing OS 1978: Cerf et al. TCP/IP DEAS 05, St. Louis

  25. Component tuning • Example of control parameters • Threading level • Admission control • Scheduling • Session length • Live versus closed connections • Specialized controllers • Specialized decision making • How general can you be at the component level? • How can one systematically inject variability at this level? DEAS 05, St. Louis

  26. Application tuning • Load balancing • horizontal scaling • vertical scaling • Inter-component optimization • DB2 connection pool in WAS • Component allocation • Scheduling • Controller and models • Become more complex • Become more general DEAS 05, St. Louis

  27. Provisioning • Hardware and software provisioning • Add/remove hardware • Add/remove software components • Needs long term prediction DEAS 05, St. Louis

  28. Building Accurate Models “If the map and the terrain don’t match, trust the terrain." Swiss Army Rule DEAS 05, St. Louis

  29. THANKS! DEAS 05, St. Louis

More Related