1 / 29

@ andy_pavlo

Making Fast Databases. FASTER. @ andy_pavlo. Fast. +. Cheap. Legacy Systems. TPC-C NewOrder. 12.3%. 29.6%. 10.2%. CPU Cycles. 18.7%. 21.1%. 8.1%. OLTP Through the Looking Glass, and What We Found There SIGMOD 2008. OLTP Transactions. Fast. Repetitive. Small.

cathleen-monroe
Download Presentation

@ andy_pavlo

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. Making Fast Databases FASTER @andy_pavlo

  2. Fast + Cheap

  3. Legacy Systems TPC-C NewOrder 12.3% 29.6% 10.2% CPU Cycles 18.7% 21.1% 8.1% OLTP Through the Looking Glass,and What We Found There SIGMOD 2008

  4. OLTP Transactions Fast Repetitive Small

  5. Main Memory • Parallel • Shared-Nothing Transaction Processing H-Store: A High-Performance, DistributedMain Memory Transaction Processing SystemVLDB vol. 1, issue 2, 2008

  6. Stored Procedure Execution Procedure Name Input Parameters Transaction Result Client Application Database Cluster Database Cluster

  7. TPC-C NewOrder

  8. Optimization #1: Partition database to reduce the number of distributed txns. Skew-Aware Automatic Database Partitioning in Shared-Nothing, Parallel OLTP SystemsSIGMOD 2012

  9. CUSTOMER ITEM CUSTOMER ORDERS CUSTOMER CUSTOMER CUSTOMER ORDERS ORDERS ORDERS ITEM ITEM ITEM

  10. ? ? ? NewOrder(5, “Method Man”, 1234) ? ? ? Client Application CUSTOMER CUSTOMER CUSTOMER ORDERS ORDERS ORDERS ITEM ITEM ITEM

  11. DDL CUSTOMER NewOrder SELECT * FROM WAREHOUSE WHERE W_ID = 10; SELECT * FROM DISTRICT WHERE D_W_ID = 10 AND D_ID =9; INSERT INTO ORDERS (O_W_ID, O_D_ID, O_C_ID,…) VALUES(10, 9, 12345,…); ⋮ SELECT * FROM WAREHOUSE WHERE W_ID = 10; SELECT * FROM DISTRICT D_W_ID = 10 AND D_ID =9; INSERT INTO ORDERS (O_W_ID, O_D_ID, O_C_ID) VALUES (10, 9, 12345); ⋮ SELECT * FROM WAREHOUSE WHERE W_ID = 10; INSERT INTO ORDERS (O_W_ID, O_D_ID, O_C_ID) VALUES (10, 9, 12345); ⋮ ITEM DDL SELECT * FROM WAREHOUSE WHERE W_ID = 10; INSERT INTO ORDERS (O_W_ID, O_D_ID, O_C_ID) VALUES (10, 9, 12345); ⋮ Schema ORDERS DTxn Estimator Skew Estimator ------- -------------- Workload Large-Neighorhood Search Algorithm … CUSTOMER CUSTOMER CUSTOMER CUSTOMER ORDERS ORDERS ORDERS ORDERS ITEM ITEM ITEM ITEM

  12. Large-Neighborhood Search Restart DDL Schema ------- -------------- Workload Initial Design Relaxation Local Search

  13. Throughput Horticulture State-of-the-Art (txn/s) TATP TPC-C TPC-C Skewed +88% +16% +183%

  14. Search Times TATP SEATS TPC-C TPC-C Skewed % Single-Partitioned Transactions TPC-E AuctionMark

  15. ? ? Undo Log ? Client Application ? Database Cluster Database Cluster

  16. Optimization #2: Predict what txns will do before they execute. On Predictive Modeling for OptimizingTransaction Execution in Parallel OLTP SystemsVLDB, vol 5. issue 2, October 2011

  17. » Partitions Touched? » Undo Log? » Done with Partitions? Client Application Database Cluster Database Cluster

  18. Current State: Input Parameters: w_id=0 i_w_ids=[0,1] i_ids=[1001,1002] ? ? GetWarehouse: SELECT * FROM WAREHOUSEWHERE W_ID = ?

  19. Estimated Execution Path Input Parameters: w_id=0 i_w_ids=[0,1] i_ids=[1001,1002] Transaction Estimate:

  20. Throughput Houdini Assume Single-Partitioned (txn/s) TATP TPC-C AuctionMark +57% +126% +117%

  21. Prediction Overhead TATP TPC-C AuctionMark

  22. Conclusion: Achieving fast performance ismore than just using only RAM. Future Work: Reduce distributed txn overhead through creative scheduling.

  23. h-store hstore.cs.brown.edu github.com/apavlo/h-store

  24. Help is Available +1-212-939-7064 Graduate Student Abuse HotlineAvailable24/7 Collect Calls Accepted

More Related