1 / 35

Datacenter Basics

Datacenter Basics. Fred Chong 290N Green Computing. Figure : Storage hierarchy of a Warehouse-scale computer. Storage Hierarchy. Figure : Latency, bandwidth and capacity of a Warehouse-scale computer. Performance Variations. Server Comparison. Cost proportional to Power.

mollie-sargent
Download Presentation

Datacenter Basics

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. Datacenter Basics Fred Chong 290N Green Computing

  2. Figure : Storage hierarchy of a Warehouse-scale computer Storage Hierarchy

  3. Figure : Latency, bandwidth and capacity of a Warehouse-scale computer Performance Variations

  4. Server Comparison

  5. Cost proportional to Power • Cost proportional to power delivered • Typically $10-20/W • Power delivery • 60-400kV transmission lines • 10-20kV medium voltage • 110-480V low voltage

  6. UPS • Uninterruptible Power Supply • Batteries or flywheel • AC-DC-AC conversion • Conditions the power feed • Removes spikes or sags • Removes harmonic distortions • Housed in a separate UPS room • Sizes range from hundreds of kW to 2MW

  7. PDUs • Power Distribution Units • Breaker panels • Input 200-480V • Output many 110V or 220V • 75-225kW in 20-30A circuits (max 6 kW) • Redundancy from two independent power sources

  8. Paralleling • Multiple generators or UPSs • Feed a shared bus • N+1 (one failure) • N+2 (one maintenance, one failure) • 2N (redundant pairs)

  9. Cooling

  10. Cooling Steps • 12-14 C coolant • 16-20 C air at CRAC (Computer Room AC) • 18-22 C at server intake • Then back to chiller

  11. “Free Cooling” • Pre-cool coolant before chiller • Water-based cooling towers use evaporation • Works in moderate climate – freezes if too cold • Glycol-based radiator outside the building • Works in cold climates

  12. Cooling is Critical • Datacenter would fail in minutes without cooling • Cooling backed up by generators and UPSs • Adds > 40% critical electrical load

  13. Airflow • 100 cfm (cubic feet per minute) per server • 10 servers would require 1000 cfm from perforated tiles • Typically no more than 150-200W / sq ft power density • Recirculation from one server’s hot air into the intake of a neighbor • Some avoid with overhead ducts

  14. Variations • In-rack cooling • Water cooled coils next on the server • Cost of plumbing • Damage from leaks (earthquake zones!) • Container-based datacenters • Shipping container 8’ x 8.5’ x 40’ • Similar to in-rack cooling but for the whole container • Higher power densities

  15. Power Efficiency • PUE – power usage efficiency • Datacenter power infrastructure

  16. Poor PUEs • 85% of datacenters PUE > 3 • Only 5% PUE = 2.0 • Chillers take 30-50% overhead • CRAC 10-30% overhead • UPS 7-12% overhead (AC-DC-AC) • Humidifiers, PDUs, lighting • EPA “achievable” PUE of 1.4 by 2011

  17. Improvements • Evaporative cooling • Efficient air movement • Eliminate power conversion losses • Google PUE = 1.21 • Several companies PUE = 1.3

  18. A more comprehensive metric • (b) SPUE – server power usage efficiency • (c) computation energy efficiency

  19. SPUE • Power delivered to components directly involved in computation: • Motherboad, disks, CPUs, DRAM, I/O cards • Losses due to power supplies, fans, voltage regulators • SPUE of 1.6-1.8 common • Power supplies less than 80% efficient • Voltage regulators less than 70% efficient • EPA feasible SPUE < 1.2 in 2011

  20. TPUE • Total PUE = TPUE = PUE * SPUE • Average of 3.2 today (2.2 Watts wasted for every Watt in computation) • PUE 1.2 and SPUE 1.2 would give 2X benefit • TPUE of 1.25 probably the limit of what is economically feasible

  21. Computing Efficiency • Area of greatest potential • Hardest to measure • SPECpower • Joulesort • Storage Network Industry Association

  22. SPECPower Example

  23. Server Load

  24. Load vs Efficiency

  25. Human Dynamic Range

  26. Component Efficiency

  27. CPU Voltage Scaling

  28. Disks • As much as 70% power to keep drives spinning • 1000X penalty to spin up and access • Multiple head, low RPM drives [Gurumurthi]

  29. Server Power Supplies

  30. Power Provisioning • $10-22 per deployed IT Watt • Given 10 year depreciation cycle • $1-2.20 per Watt per year • Assume $0.07 per kilowatt-hr and PUE 2.0 • 8766 hours in a year • (8766 / 1000) * $0.07 * 2.0 = $1.22724 • Up to 2X cost in provisioning • eg. 50% full datacenter = 2X provisioning cost

  31. Time at Power Level 80 servers 800 servers 8000 servers

  32. Oversubscription Opportunity • 7% for racks (80) • 22% for PDUs (800) • 28% for clusters (8000) • Could have hosted almost 40% more machines

  33. Underdeployment • New facilities plan for growth • Also discretization of capacity • Eg 2.5kW circuit may have four 520W servers • 17% underutilized, but can’t have one more

More Related