Making Storage the Invisible Datacenter Resource

1. Making Storage the Invisible Datacenter Resource Lukas Lundell, Nutanix KarthikRanganathan, Nutanix STO5209 #STO5209

2. Agenda • Introduction and Background • The Utopian Datacenter • What You Really Want from Your Storage • Enabling Technologies • Operating at Scale – Lessons from Big Tech • Q & A

3. Introduction and Background Lukas Lundell Email: lukas@nutanix.com Twitter: @LukasLundell Web: LukasLundell.com Current: Sr. Manager of Solutions Engineering Team, Nutanix Background: Datacenter Architect for Accenture’s R&D group, Accenture Technology Labs. Designed, built, and managed Tier 3 and Tier 4 datacenters and associated infrastructure for federal government and financial services clients. KarthikRanganathan Email: karthik@nutanix.com Twitter: @KarthikR Current: Sr. Development Engineer, Nutanix Background: Technical Engineering Lead at Facebook. Co-built Cassandra for Facebook Inbox Search and improved performance and resiliency of Hbase for Facebook Messages and Search Indexing.

4. Agenda • Introduction and Background • The Utopian Datacenter • What You Really Want from Your Storage • Enabling Technologies • Operating at Scale – Lessons from Big Tech • Q & A

5. The Utopian Datacenter Today’s sexiest datacenters. What does an ideal datacenter look like? Lenoir, North Carolina Prineville, Oregon Chicago, Illinois • What’s missing? • Fibre Channel Networks • Disk shelves • Physical SAN/NAS controllers • What happened to all of the network storage??? • What’s in common? • Uniformity • x86 hardware • Local Disks. • Neon lighting. • Green and Blue are very popular colors.

6. Agenda • Introduction and Background • The Utopian Datacenter • What You Really Want from Your Storage • Enabling Technologies • Operating at Scale – Lessons from Big Tech • Q & A

7. The Storage Trinity How storage resources are presented and consumed today File Object Block Object File What does my storage look like today?

8. The Storage Trinity How storage resources are presented and consumed today File Block Object SCSI Read(10) Command Descriptor Block (CDB): There are about 60 SCSI commands defined, of which we commonly use less than 20.

9. The Storage Trinity How storage resources are presented and consumed today File Block Object

10. The Storage Trinity How storage resources are presented and consumed today File Block Object 3.3.0 NULL: Do nothing . . . . . . . . . . . . . . . . . . . . 31 3.3.1 GETATTR: Get file attributes . . . . . . . . . . . . . . 32 3.3.2 SETATTR: Set file attributes . . . . . . . . . . . . . . 33 3.3.3 LOOKUP: Lookup filename . . . . . . . . . . . . . . . . 37 3.3.4 ACCESS: Check access permission . . . . . . . . . . . . 40 3.3.5 READLINK: Read from symbolic link . . . . . . . . . . . 44 3.3.6 READ: Read from file . . . . . . . . . . . . . . . . . . 46 3.3.7 WRITE: Write to file . . . . . . . . . . . . . . . . . . 49 3.3.8 CREATE: Create a file . . . . . . . . . . . . . . . . . 54 3.3.9 MKDIR: Create a directory . . . . . . . . . . . . . . . 58 3.3.10 SYMLINK: Create a symbolic link . . . . . . . . . . . . 61 3.3.11 MKNOD: Create a special device . . . . . . . . . . . . . 63 3.3.12 REMOVE: Remove a file . . . . . . . . . . . . . . . . . 67 3.3.13 RMDIR: Remove a directory . . . . . . . . . . . . . . . 69 3.3.14 RENAME: Rename a file or directory . . . . . . . . . . . 71 3.3.15 LINK: Create link to an object . . . . . . . . . . . . . 74 3.3.16 READDIR: Read From directory . . . . . . . . . . . . . . 76 3.3.17 READDIRPLUS: Extended read from directory . . . . . . . 80 3.3.18 FSSTAT: Get dynamic file system information . . . . . . 84 3.3.19 FSINFO: Get static file system information . . . . . . . 86 3.3.20 PATHCONF: Retrieve POSIX information . . . . . . . . . . 90 3.3.21 COMMIT: Commit cached data on a server to stable storage 92 NFSv3 Commands (RFC #1813): Sample Sun ONC UDP RPC v2: There are 22 NFSv3 commands defined in RFC #1813. NFSv3 RFC#1813: http://tools.ietf.org/html/rfc1813

11. The Storage Trinity How storage resources are presented and consumed today File Block Object

12. The Storage Trinity How storage resources are presented and consumed today File Block Object Sample Amazon S3 GET Object request:

13. What you really want from your storage What we ask from network storage is simple… 1 Data Integrity + Security Don’t corrupt my data. Keep my data safe. Also, keep some snapshots around just in case. 2 Availability 100% uptime. No single points of failure. Also, replicate my VMs and data in case I have a disaster. What else do you want from your storage? 3 Manageability Provisioning, proactive monitoring, and scaling should be simple. Provide an API to use for automation. Integrate well with VMware (my key app). 4 Performance Be fast enough to handle all my servers, applications, and databases. 5 Economics Do the other four things very well and don’t cost me too much.

14. What you really want from your storage … Why is it so hard to get it right? 1 Data Integrity + Security Don’t corrupt my data. Keep my data safe. Also, keep some snapshots around just in case. 2 Availability I have to run at 50% capacity to tolerate a failure with my dual controller system? Why can’t I replicate data to my DR site on a per-VM basis? 3 Manageability Need to hire PS to set it up? No scale out. Capacity planning is a pain. Forced to manage LUNs or volumes rather than VMs. Our SAN guy is always busy. 4 Performance As I add more workloads, my performance gets worse. Why is my VDI Environment so slow? I can’t set QoSfor certain VMs? 5 Economics Why do you cost me so much money and time while failing to solve my storage problems.

18. How do they do it? How does big tech transcend these issues? Lenoir, North Carolina Prineville, Oregon Chicago, Illinois • Making storage an invisible resource… • Use local devices on x86 hardware as the fundamental storage resource. Basic building block. • Build intelligent distributed software-based storage services on top of these systems that are purpose built for the use case (ex: Google GFS, Facebook Haystack, Windows Azure Drives/Blobs) • Scale out not scale up. • Scale out not shell out. • *Your use case is providing storage services to VMs on your chosen hypervisor (VMware)*

19. Agenda • Introduction and Background • The Utopian Datacenter • What You Really Want from Your Storage • Enabling Technologies • Operating at Scale – Lessons from Big Tech • Q & A

20. Enabling Technologies What industry developments are enabling storage to become an invisible part of the datacenter… 1 Advancement of Distributed Systems Big tech has made great progress in distributed systems. Their technologies were open sourced and can be generalized to fit the enterprise What’s enabling the shift of how we approach enterprise storage? 2 Flash Technology Very low latency reads. Causes network latencies to become noticeable. Very fast persistent storage without requiring a high spindle count. 3 Denser Compute Nodes Great compute power allows us to seamlessly virtualize the storage controller. Much like your iPhone displaced your iPod. 4 Paxos Practical development of protocols for solving consensus in distributed systems. 5 Protocol Buffers Google developed protocol buffers to improve the performance and efficiency of communication in a distributed system. Awesome serialization tech.

21. Enabling Technologies What industry developments are enabling storage to become an invisible part of the datacenter… 1 Advancement of Distributed Systems Big tech has made great progress in distributed systems. Their technologies were open sourced and can be generalized to fit the enterprise. 2 Flash Technology Very low latency reads. Causes network latencies to become noticeable. Very fast persistent storage without requiring a high spindle count. 3 Denser Compute Nodes Great compute power allows us to seamlessly virtualize the storage controller. Much like your iPhone displaced your iPod. 4 Paxos Practical development of protocols for solving consensus in distributed systems. 5 Protocol Buffers Google developed protocol buffers to improve the performance and efficiency of RPC communication in a distributed system. Awesome serialization tech.

22. Agenda • Introduction and Background • The Utopian Datacenter • What You Really Want from Your Storage • Enabling Technologies • Operating at Scale – Lessons from Big Tech • Q & A

23. Building Distributed Systems Core principles of ‘s big data architecture • Core Strategy: • Use commodity hardware • Scaling out simply is #1 • Efficiency is #2 • Design for failures • Metrics, Metrics, Metrics • Health Checks: • Auto remediation of nodes • Detect and adjust to node failures (auto exclude on node failure, but do not auto exclude on rack failures) • Auto include nodes once they have been fixed (but rate limit remediations) • Dashboards and Metrics: • Single view of the entire distributed system • SLA hits/misses • RPC calls • Cluster and node health • Performance and utilization • Design for Failures: • No single point of failure. Killing any process is legit. • Minimize manual intervention. (especially for common failures like disks) • Rolling upgrades are the norm. Maintain uptime during them. • Survive rack failures.

24. Technology behind massive scale Advancement of distributed systems

25. Agenda • Introduction and Background • The Utopian Datacenter • What You Really Want from Your Storage • Enabling Technologies • Operating at Scale – Lessons from Big Tech • Q & A

26. Customers Who Can Scale Forward thinking companies who’ve embraced distributed systems tech.

27. Q&A Feel free to ask us anything… Lukas Lundell Email: lukas@nutanix.com Twitter: @LukasLundell Web: LukasLundell.com Current: Sr. Manager of Solutions Engineering Team, Nutanix Background: Datacenter Architect for Accenture’s R&D group, Accenture Technology Labs. Designed, built, and managed Tier 3 and Tier 4 datacenters and associated infrastructure for federal government and financial services clients. KarthikRanganathan Email: karthik@nutanix.com Twitter: @KarthikR Current: Sr. Development Engineer, Nutanix Background: Technical Engineering Lead at Facebook. Co-built Cassandra for Facebook Inbox Search and improved performance and resiliency of Hbase for Facebook Messages and Search Indexing.

28. Please come visit us at booth #1521! X marks the spot. NUTANIX INC. – CONFIDENTIAL AND PROPRIETARY

29. Other VMware Activities Related to This Session STO5209 • HOL:HOL-SDC-1304vSphere Performance Optimization HOL-SDC-1308Virtual Storage Solutions • Group Discussions:STO1000-GDOverall Storage with Patrick Carmichael

30. THANK YOU

31. FILL OUT A SURVEY Every Completed Survey Is Entered Into a Drawing for a $25 VMware Company Store Gift Certificate

32. Making Storage the Invisible Datacenter Resource Lukas Lundell, Nutanix KarthikRanganathan, Nutanix STO5209 #STO5209