1 / 31

KEY VALUE STORAGE IN THE CLOUD

KEY VALUE STORAGE IN THE CLOUD. AGENDA. review cloud storage review providers review differences between storage clouds jclouds. review cloud storage. what is key value storage?. global name space key, value with metadata http accessible sites on demand unlimited scaling.

tea
Download Presentation

KEY VALUE STORAGE IN THE CLOUD

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. KEY VALUE STORAGE IN THE CLOUD

  2. AGENDA • review cloud storage • review providers • review differences between storage clouds • jclouds

  3. review cloud storage

  4. what is key value storage? • global name space • key, value with metadata • http accessible • sites on demand • unlimited scaling

  5. key value storage concepts • GLOBAL NAME SPACE • organizes and sets boundaries for content • called a “bucket” or “container” • KEY/ VALUE • called objects, blobs, or files • unstructured content • can be small or large • can have metadata (key, value pairs)

  6. review vendors

  7. THE VENDORS

  8. review differences between storage clouds

  9. FILE SIZE • Can you store 5 GB or unlimited? • UPLOAD LARGE FILES • One shot or chunked? • DOWNLOAD LARGE FILES • One shot or chunked?

  10. CONTENT DELIVERY NETWORK Cache content close to users? GLOBAL PRESENCE Where do you write? What level of redundancy? SLA How many 9’s? Maintenance notification Replication count

  11. Consistency Model Immediate or Eventual?

  12. SUB-ACCOUNTS AND QUOTAS ACCESS CONTROLS AUTHORIZATION

  13. PROTOCOLS • ReST • SOAP • WebDAV • CIFS • NFS

  14. pricing • PARKING FEES • TEXT MESSAGING • AIR TIME • SUPPORT

  15. Value Adds • Promotions with other services and tools

  16. api complexity • How difficult is it to use?

  17. code and sign the http request PUT /adriansmovies/sushi.avi HTTP/1.1 Host: <account>.blob.core.windows.netContent-Length: 734859264 Date: Wed, 01 Mar 2006 12:00:00 GMTAuthorization: SharedKey <app>:signature x-ms-meta-Chef: Kawasaki PUT /sushi.avi HTTP/1.1 Host: adriansmovies.s3.amazonaws.comContent-Length: 734859264Date: Wed, 01 Mar 2006 12:00:00 GMTAuthorization: signature x-amz-meta-Chef: Kawasaki POST /namespace/adriansmovies/sushi.avi HTTP/1.1 Content-Length: 734859264Date: Wed, 01 Mar 2006 12:00:00 GMT x-emc-uid: <uid> x-emc-signature: signature x-emc-meta: Chef=Kawasaki PUT /<api version>/<account>/adriansmovies/sushi.avi HTTP/1.1 Host: storage.clouddrive.com Transfer-Encoding: chunkedX-Auth-Token: session-token X-Object-Meta-Chef: Kawasaki

  18. CODE AND SIGN THE HTTP REQUEST GET /ws/IMFS/GetStorageNodeExtended.ashx?&fileOverwrite=true&ipRestricted=true&destFolderPath= adriansmovies&sizeBytes=734859264&firstByteExpiration=6000&lastByteExpiration=259200&sessionToken=session-token HTTP/1.1 POST /Upload.ashx?uploadToken=from_above&destFolderPath=adriansmovies HTTP/1.1 Host: from_above Content-Length: 734859382 Content-Type=multipart/form-data; boundary=--jclouds-- Authorization=Basic GpjbG9= ----jclouds-- Content-Disposition: form-data; name="sushi.avi"; filename="sushi.avi" Content-Type: application/octetstring ... PUT /ws/Metadata/SetMetadata.ashx?&path=Folders/adriansmovies/sushi.avi&sessionToken=session-token&metadata=Chef:Kawasaki HTTP/1.1

  19. CODE AND SIGN THE HTTP REQUEST POST /<api version>/containers/id_of_ adriansmovies/contents HTTP/1.1 Content-Length: 734859382 Content-Type=multipart/form-data; boundary=--jclouds-- Authorization=Basic GpjbG9= ----jclouds-- Content-Disposition: form-data; name="sushi.avi"; filename="sushi.avi" Content-Type: application/octetstring ... PUT /<api version>/files/from_above/metadata/Chef HTTP/1.1 Content-Length: 8 Content-Type: text/plain Authorization: Basic GpjbG9= Kawasaki

  20. do you want to • Deal with Errors • Deal with Concurrency • Deal with Cloud Complexity

  21. enterprise grade tooling is critical • JCLOUDS BLOBSTORE • open source • feels like java • portability between clouds • deal with web complexity • unit testability • thread-safe and scalable

  22. jclouds-blobstore • No HTTP, weird URLs, or complicated authorization • Portable atomic commands for blobs and containers. • Easy to switch clouds (or stub!) BlobStoreContext context = S3ContextFactory.createContext(account, key); Map<String, InputStream> map = context.createInputStreamMap("adriansmovies"); InputStream myphoto = map.get(“sushi.avi”); BlobMap blobMap = context.createBlobMap("adriansmovies"); BlobMetadata sushiMetadata = map.get(“sushi.avi”).getMetadata(); String chef = sushiMetadata.getUserMetadata(“chef”); String contentType = sushiMetadata.getContentType(); context.close();

  23. architecture

  24. questions? • http://www.jclouds.org

  25. DEMO • Goals • Show multi-cloud • Show Google AppEngine support • Show something cool

  26. DEMO architecture

  27. Lets try it out • Send a tweet @jclouds • Wait a minute • Check to see if it landed in the cloud • http://jclouds-tweetstore.appspot.com/tweets/get

  28. what’s next? • COMPUTE • VCloud Express, EC2 and GoGrid Support • Create Abstraction Layer via Dasein • STORAGE • Content Delivery Network • Eucalyptus Support

  29. appendix • other notes of interest

  30. feature and pricing comparison

More Related