1 / 18

xrootd Update

xrootd Update. OSG All Hands Meeting University of Nebraska March 19-23, 2012 Andrew Hanushevsky, SLAC http://xrootd.org. Outline. Additions & Changes (available now in 3.1.1) On the horizon (anticipated for 3.2 release 3Q12) Near future (anticipated for 3.3 release)

mandel
Download Presentation

xrootd Update

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. xrootd Update OSG All Hands Meeting University of Nebraska March 19-23, 2012 Andrew Hanushevsky, SLAC http://xrootd.org

  2. Outline • Additions & Changes (available now in 3.1.1) • On the horizon (anticipated for 3.2 release 3Q12) • Near future (anticipated for 3.3 release) • The xrootd collaboration • Conclusion • Acknowledgements

  3. Additions & Changes (3.1) • Extended Attribute Framework • Integrated checksums • Shared-Everything File System Support • Static ENOENT redirection • Enhances creation of new topologies • Caching proxy server • Federated site shares • Monitoring Extended • Read-fork-read client

  4. Agnostic Extended Attributes • FS-Independent Extended Attribute Framework • Used to save file-specific information • File system must support extended attributes • Current Attributes • XrdCks.xxxxxx (e.g. md5) checksum • XrdFrm.yyyFile Residency Manager information • Future Attributes • Original file creator • Extended Access Information • Number of parallel readers, % read, access time • Can be used for better migration/purging

  5. Integrated Checksums • Originally, checksums supported via callout • Now, xrootd internally handles checksums • xrootd.chksum [maxnum] {adler32|crc32|md5} • Add new ones via ofs.ckslib directive and plug-in • Checksum saved in extended attributes • Returned on query • Automatically recomputed when file changes • Can be disabled • External program not needed; but still supported

  6. Shared FS Support • The problem dCache GPFS HDFS Lustre etc Data Server Client Redirector Data Server Shared FS Data Server open(x) have(x) stat(x) • The solution • Use cms.dfs directive to eliminate duplicate hits • Many tuning options available • Lookup at redirector, limits, caching, etc • See http://xrootd.org/doc/prod/cms_config.htm

  7. Static ENOENT Redirection ? xyzzy:1234 open(x) Data Server goto xyzzy:1234 if redirector_host xrootd.redirect ? / xyzzy:1234 fi Client Redirector Data Server open(x) Data Server have(x) • What we envisioned. . . • xyzzy is a caching proxy server or proxy cluster • Provides high performance WAN access • Client accesses data via WAN in this case

  8. What Others Envisioned! open(x) Global Redirector Client all.manager meta C all.manager meta global if redirector C xrootd.redirect ? / global:1234 fi goto xyzzy:1234 exclude region A open(x) ? Redirector A Redirector B Redirector C Site 8 Site 7 Site 5 Site 6 Site 4 Site 3 Site 2 Site 1 Region A Region B Region C

  9. Caching Proxy Server • The proxy server plug-in has in-memory cache • Must be enabled via pss.memcache directive • Many tuning options; some of which are . . . • Cache and page size • Read ahead size • Maximum block size to cache • Root file access optimization • See http://xrootd.org/doc/prod/ofs_config.htm

  10. Federated Site Shares • The problem Each site gets an equal number of requests all things being equal Site A Meta Manager Site B Site C Client • What to do if sites do not want to be equal? • Use cms.sched directive to establish site share • Use the gshr and gsdflt options • See http://xrootd.org/doc/prod/cms_config.htm

  11. Monitoring • Readv requests fully monitored • By default, only a readv request summary provided • Can request a full unwind of a readv request • Per client I/O monitoring now flushable • Before, I/O statistics flushed when buffer full • Can specify a flush window • Based on code provided by MatevzTadel, CMS • Authentication can now be fully monitored

  12. Read-Fork-Read Client • The current client allows forking • This allows sharing data between processes • Read conditions data • Fork n times for parallel processing • Read event data using pre-read conditions data • Extensively used by CMS • Substantially reduces memory load • Critical for large multi-core worker nodes

  13. git & cmake & EPEL • Now using git repository for source code • git clone http://xrootd.org//repo/xrootd.git • Standardized on cmake for builds • See README file in the top level source directory • Adhering to EPEL guidelines • All ‘.a’ files are replaced by ‘.so’ files • Caused increase in number of installed .so files • We consolidated libraries; but more than we would like • Unfortunately, changing things afterwards is difficult in EPEL

  14. On the horizon (3.2) • New fully asynchronous client • Guards against low performing servers • To be added in subsequent release • Extended monitoring • Redirect information • Authentication summary information • Integrated 3rd party copy • Allows client directed server-to-server copies • Dropping RH4 support

  15. Things Within A Year (3.3+) • Disk Caching Proxy Server • Extension of memory caching (UCSD effort) • Automatic checksum validation • Integrated alerts • New more effective async I/O model • IPV6

  16. Xrootd Collaboration • Mutually interested institutions contributing effort for development and maintenance • SLAC (founder) • CERN (2010) • Duke (spring 2011) • JINR (fall 2011) • UCSD (winter 2011) • Newest member!

  17. Conclusion • xrootdis under active development • Always looking for new ideas • Feel free to suggest them • Be a contributor • You too can contribute to the code base • Consider joining the xrootdcollaboration • It costs no money to join • See more at http://xrootd.org/

  18. Acknowledgements • Current Software Contributors • ATLAS: Doug Benjamin, Patrick McGuigan, DanilaOleynik, ArtemPetrosyan • CERN: FabrizioFurano, Lukasz Janyst, Andreas Peters,David Smith • CMS: Brian Bockelman (unl), MatevzTadel (ucsd) • Duke: Douglas Benjamin • Fermi/GLAST: Tony Johnson • LBNL: Alex Sim, JunminGu, VijayaNatarajan(BeStMan team) • Root: Gerri Ganis, BeterandBellenet, FonsRademakers • OSG: Tim Cartwright, Tanya Levshina • SLAC: Andrew Hanushevsky,WilkoKroeger, Daniel Wang, Wei Yang • Operational Collaborators • ANL, BNL, CERN, FZK, IN2P3, SLAC, UCSD, UTA, UoC, UNL, UVIC, UWisc • US Department of Energy • Contract DE-AC02-76SF00515with Stanford University

More Related