1 / 12

Bulk Data API

Bulk Data API. Nick Simha Technical Alliance Manager. Agenda. Bulk Data API basics Demo Best practices Resource list. What is the Bulk Data API?. REST based, asynchronous API optimized for loading large sets of data. Why is it useful?.

lavender
Download Presentation

Bulk Data API

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. Bulk Data API Nick Simha Technical Alliance Manager

  2. Agenda Bulk Data API basics Demo Best practices Resource list

  3. What is the Bulk Data API? • REST based, asynchronous API optimized for loading large sets of data.

  4. Why is it useful? • Enable high volume integration with Salesforce (volume) • Enable integration that has to finish in a certain window of time (speed) • Part of suite of features that enable our customers to store very large data volumes in Salesforce • Batch Apex • Skinny tables • Divisions • Custom Indexing • Etc.

  5. How does the Bulk Data API works Loop until all records sent (e.g. 50 times for 500k rows) Decoupled phases. One doesn't wait for the other, each can run in parallel Loop until all files processed

  6. How can I call the Bulk Data API? • Through Data Loader • From any Web Services Client – Java, C# etc. • From the command line! • Support by our integration partners

  7. Using the Bulk API from a client • Create Job • Create Batch (es) and add to Job • Number of batches determined by the amount of data and the limits on batch size. • Close Job • Retrieve Batch Status • Retrieve Batch Result

  8. Sample Request POST /services/async/17.0/job HTTP/1.1 User-Agent: curl/7.19.6 (i386-pc-win32) libcurl/7.19.6 OpenSSL/09.8k zlib/1.2.3 Host: na6.salesforce.com Accept: */* X-SFDC-Session:00D80000000MD0n!AQgAQI1EfPPEyWvwuaD_IRpvSlrwm7Kr00e Content-Type: application/xml; charset=UTF-8 Content-Length: 195 <?xml version="1.0" encoding="UTF-8"?> <jobInfo xmlns="http://wwwforce.com/2009/06/asyncapi/dataload"> <operation>insert</operation> <object>Contact</object><contentType>CSV</contentType></jobInfo> See Getting Started Chapter in the API guide. Use curl –trace-ascii <file_name> to capture messages.

  9. Demo • Load 100K Addresses of medical providers • Cleanse the data

  10. Bulk API - Some Additional Information • Can Monitor Bulk Loads in Builder • Monitoring -> Bulk Data Load Jobs • Doesn’t handle attachments • Governor limits – 500 batches per 24 hour limit. 10,000 records per batch. So theoretical limit of 5M records per day. • Caveat – Batch size also needs to be less than 10MB • Batch limits can be increased by engineering • Need to contact your SE

  11. Best practices • Combine Bulk API with Batch Apex to get optimal performance • Faster than complex triggers • Similar to the demo – this is a generic pattern that you can use in many scenarios • Stick with parallel processing unless there is a reason not to do so • See FAQ for scenarios when you would serial processing • Handling Very Large Data Volumes requires a comprehensive, holistic approach • Bulk API is one part of the solution

  12. Resource list • Bulk API doc http://www.salesforce.com/us/developer/docs/api_asynch/index.htm

More Related