1 / 18

Clustering the Reliable File Transfer Service

Clustering the Reliable File Transfer Service. Jim Basney and Patrick Duda NCSA, University of Illinois. This material is based upon work supported by the National Science Foundation under Grant No. 0426972. Goal. Provide a highly available Reliable File Transfer (RFT) Service

eydie
Download Presentation

Clustering the Reliable File Transfer Service

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. Clustering the Reliable File Transfer Service Jim Basney and Patrick DudaNCSA, University of Illinois This material is based upon work supported by the National Science Foundation under Grant No. 0426972. TeraGrid '07

  2. Goal • Provide a highly availableReliable File Transfer (RFT) Service • Tolerate server failures • Hardware/software faults and resource exhaustion • Continue to handle incoming requests • Continue to make forward progress on file transfers in the queue TeraGrid '07

  3. GridFTP GridFTP Globus ToolkitReliable File Transfer Service Client RFT TeraGrid '07

  4. RFT and GridFTP Clustering GridFTPdata RFT GridFTPcontrol GridFTPdata RFT GridFTPdata RFT GridFTPcontrol GridFTPdata TeraGrid '07

  5. Clustering Approach RFT HADBMS LoadBalancer RFT RFT TeraGrid '07

  6. RFT State Management Web ServiceContainer DelegationService Client RFT DBMS TeraGrid '07

  7. RFT DB Tables Added Fields TeraGrid '07

  8. New Tables TeraGrid '07

  9. RFT Fail-Over • Based on time-outs • Periodically query database for pending requests with no recent activity • Stalled requests could be caused by RFT service crash, hardware failure, RFT service overload, etc. • If found, obtain DB write lock, query again, claim stalled requests, and release lock • Configuration values: • Query interval (default: 30 seconds) • Recent interval (default: 60 seconds) TeraGrid '07

  10. Evaluation Environment • Dedicated 12 node Linux cluster • Red Hat Enterprise Linux AS Release 3 • Switched Gigabit Ethernet • 2 GB RAM • dual 2GHz Intel Xeon CPUs 512KB cache • Globus Toolkit 4.0.3 • MySQL Standard 5.0.27 TeraGrid '07

  11. Evaluation • Correctness / Effectiveness • Submitted multiple RFT requests of different sizes to 12 RFT instances • Verified fail-over and notification functionality • Performance • Evaluate overhead of shared DBMS • Stress test: transfer many small files TeraGrid '07

  12. 60 second fail-over interval web servicescontainer stopped fail-over TeraGrid '07

  13. TeraGrid '07

  14. 95% 82% 57% 43% 22% 14% 10% 6% 4% TeraGrid '07

  15. Related Work HAND: Highly Available Dynamic Deployment Infrastructure for GT4 Migrate services between containers to maintain availability during planned outages Does not address management of persistent service state or fail-over for unplanned outages myGrid DBMS persistence of WS-ResourceProperties in Apache WSRF Points to a general-purpose approach for DBMS-based persistence of stateful WSRF services TeraGrid '07

  16. Conclusion Clustering RFT provides load-balancing and fail-over with acceptable performance for small clusters Clustering is a promising approach for application to other grid services TeraGrid '07

  17. Future Work Correctly handle replay of FTP deletes Implement credentialRefreshListener Evaluate use of different DBMS solutions Investigate GT4 DBMS persistence in general Investigate use of WS-Naming TeraGrid '07

  18. Thanks! Questions? Comments? This material is based upon work supported by the National Science Foundation under Grant No. 0426972. Performance experiments were conducted on computers at the Technology Research, Education, and Commercialization Center (TRECC), a program of the University of Illinois at Urbana-Champaign, funded by the Office of Naval Research and administered by the National Center for Supercomputing Applications. We thank Tom Roney for his assistance with the TRECC cluster. We also thank Ravi Madduri from the Globus project for answering our questions about RFT. TeraGrid '07

More Related