1 / 41

An Introduction to Cloud-based Services

An Introduction to Cloud-based Services. Paul Watson Newcastle University, UK paul.watson@ncl.ac.uk. e.g. Amazon. Plan. What is Cloud Computing? Potential Advantages Lessons from our own experiences Cloud Issues. What is Cloud Computing? . “.. a broad array of

Leo
Download Presentation

An Introduction to Cloud-based Services

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. An Introduction to Cloud-based Services Paul Watson Newcastle University, UK paul.watson@ncl.ac.uk

  2. e.g. Amazon

  3. Plan • What is Cloud Computing? • Potential Advantages • Lessons from our own experiences • Cloud Issues

  4. What is Cloud Computing? “.. a broad array of web-based services aimed at allowing users to obtain a wide range of functional capabilities on a ‘pay-as-you-go’ basis that previously required tremendous hardware/software investments and professional skills to acquire.” Irving Wladawsky Berger

  5. What’s New? • illusion of Infinite computing resources On Demand • no up-front commitment by users • Pay for use of resources on a short-term basis as needed (from “Above the Clouds: A Berkeley View of Cloud Computing”)

  6. Example – Amazon Web Services • Based on Xen VMs • run any OS & software stack • CPU: 1.0Ghz x86 instance @ $0.10 /hour • Blob Storage @ $0.12 /GB month • External Data Transfer @ $0.10 /GB • Also queue, key store, block store, range of instances

  7. Why is this Important (I): Internal IT Problems (slide by permission of Arjuna Technologies) Silos = Inflexibility

  8. Why is this Important (II)? Time to put Ideas into action Research • Have good idea • Write proposal • Wait 6 months • If successful.. • Buy Computers • Install Computers • Start Work Science Start-ups • Have good idea • Write Business Plan • Ask VCs to fund If successful.. • Buy computers • Install Computers • Start Work

  9. Why is this a Good idea: using commercial clouds • Have good idea • Grab nodes as needed from Cloud provider • Start Work • Pay for what you used

  10. Cloud Services Continuum (based on Robert Anderson) http://et.cairene.net/2008/07/03/cloud-services-continuum/ Software (SaaS) Google Docs Salesforce.com Platform (PaaS) Flexibility Complexity Google AppEngine Microsoft Azure Infrastructure (IaaS) Amazon EC2 & S3

  11. Example Lessons from CARMEN Project • Design began in 2006 • Commercial clouds not an option • Designed own “private” cloud • Experimenting with Commercial Cloud

  12. UK EPSRC e-Science Pilot £4M (2006-10) 20 Investigators CARMEN Project Stirling St. Andrews Newcastle York Manchester Sheffield Leicester Cambridge Warwick Imperial Plymouth

  13. Industry & Associates

  14. Research Challenge Understanding the brain is the greatest informatics challenge • Enormous implications for science: • Medicine • Biology • Computer Science

  15. Collecting the Evidence 100,000 neuroscientists generate huge quantities of data • molecular (genomic/proteomic) • neurophysiological (time-series activity) • anatomical (spatial) • behavioural

  16. Epilepsy Exemplar Data analysis guides surgeon during operation Further analysis provides evidence WARNING! The next 2 Slides show an exposed human brain

  17. enables sharing and collaborative exploitation of data, analysis code and expertise that are not physically collocated CARMEN

  18. CARMEN e-Science Requirements • Store • very large quantities of data (100TB+) • Analyse • suite of neuroinformatics services • support data intensive analysis • Automate • workflow • Share • under user-control

  19. Background: North East Regional e-Science Centre • 25 Research Projects across many domains: • Bioinformatics, Ageing & Health, Neuroscience, Chemical Engineering, Transport, Geomatics, Video Archives, Artistic Performance Analysis, Computer Performance Analysis,.... • Same key needs:

  20. Result: e-Science Central • Integrated Store-Analyse-Automate-Share infrastructure • Generic • CARMEN neuroinformatics & chemistry as pilots

  21. e-Science Central • Web based • Works anywhere e-Science Central • Dynamic Resource • Allocation • Pay-as-you-Go* • Controlled Sharing • Collaboration • Communities

  22. Science Cloud Architecture Access over Internet (typically via browser) Upload data & services Run analyses Data storage and analysis

  23. Science Cloud Options Users Science App 1 Science App n Service Developers .... Science Platform Science App 1 Science App n .... Cloud Infrastructure: Storage & Compute Cloud Infrastructure: Storage & Compute

  24. .... App App App API e-Science Central Security Analysis Services Social Networking Science Cloud Platform Workflow Enactment Processing Cloud Infrastructure Storage

  25. Editing and Running a Workflow on the Web

  26. Workflow Result File Viewing the output of Workflow Runs

  27. Viewing results

  28. Blogs and links Communicating Results Linking to results & workflows

  29. What we learnt: Moving into a Cloud • Moving existing technologies into a cloud can be difficult • some can’t run in a Cloud at all

  30. Raw Data Exploration with Signal Data Explorer

  31. What we learnt : Scalability • Clouds offer the potential for scalability • grab compute power only when needed • Developers have to manage scalability • for Infrastructure as a Service Clouds • scale up as well as down

  32. Adaptive Dynamic Deployment with Dynasoar Commercial “pay-as-you-go” clouds would allow us to avoid this limit Adding Processors as you need them optimises resources and saves money in pay-as-you-go clouds Ensure system can also release unwanted nodes

  33. Microsoft Azure Cloud for e-Science Demo • Recent experiments with Microsoft Azure Cloud • running Chemical analyses • Silverlight App Thanks to: - Paul Appleby & Team at the Microsoft Technology Centre, Reading - & MS External Research e-Science Group

  34. Microsoft Azure Cloud Demo

  35. When not to use Clouds? • Large data transfers • Time & Cost • High Performance • cpu/io/network bandwidth/low latency • Predictable Performance • Confidentiality • High Availability? • High Server Utilisation? • private clouds better?

  36. Create Private Cloud (slides by permission of Arjuna Technologies)

  37. Private Cloud Examples • Eucalyptus • Amazon API • Private Cloud deployments of Microsoft Azure • Arjuna Agility

  38. Federating Private & Public Clouds Public Cloud Public Cloud e.g. Amazon App1 Service Agreement Arjuna Agility App1 App1 & 2 Service Agreement Internal Cloud Dept A Dept B

  39. Public Cloud e.g. Amazon App1 App1 Public Cloud e.g. FlexiScale Arjuna Agility App1 App1 & 2 Internal Cloud Dept A Dept B Arjuna

  40. Summary • Cloud computing can revolutionise e-science • provide sustainable infrastructure • reduce time from idea to realisation • Don’t underestimate complexity • building scalable distributed systems is still hard • can Science Clouds help by lowering the hurdles? • e-Science Central • Store-Analyse-Automate-Share e-science platform • adding content from a range of domains • CARMEN is evaluating it for neuroinformatics

More Related