1 / 79

ROOT Project Status Major developments Directions

ROOT Project Status Major developments Directions. DESY 5 December 2005 Ren é Brun CERN. ROOT: a 10 years old project. Started in January 1995 in NA49 First public presentation in November 95 Cooperation with Masa Goto/CINT in April 96

whitney
Download Presentation

ROOT Project Status Major developments Directions

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. ROOT Project StatusMajor developmentsDirections DESY 5 December 2005 René Brun CERN Overview, Major Developments, Directions

  2. ROOT: a 10 years old project • Started in January 1995 in NA49 • First public presentation in November 95 • Cooperation with Masa Goto/CINT in April 96 • First implementation of ROOT with gAlice in December 97 (TGeant3)->VMC • FNAL chooses ROOT for I/O and data analysis • Announced at CHEP98 in Sept 98 in Chicago • RHIC experiments follow immediately • Hoffmann computing Review in 2000/2001 • LCG project in 2002 (Blueprint RTAG) • ROOT: LCG project in 2005 : Overview & Major Developments

  3. Applications Area Organizationin LCG Phase I decisions strategy Architects forum Applications manager ROOT User - provider Applications area meeting POOL project SEAL project PI project Simulation project SPI project consultation Overview & Major Developments

  4. Applications Area Organizationin LCG Phase II Same organization as in Phase I. ROOT and SEAL projects merge Applications area meeting POOL project ROOT +SEAL project Simulation project SPI project consultation ROOT project structured in work-packages Overview & Major Developments

  5. New ROOT team structure The work-packages SEAL Lorenzo Moneta DICT Philippe Canal BASE Fons Rademakers MATH Lorenzo Moneta I/O & Trees Philippe Canal 2-D/3D graphics Olivier Couet GUI Ilka Antcheva PROOF Fons Rademakers GEOM/VMC Andrei Gheata Overview & Major Developments

  6. ROOT version 5 • pro release 4.04/02 3 May (new Users Guide) • 1st dev release 5.02 28 June • 2nd dev release 5.04 20 September • 3rd dev release 5.06 2 Nov • Pro release 5.12 15 December (New Guide) See detailed Release Notes http://root.cern.ch/root/Version50400.news.html Overview & Major Developments

  7. ROOT 2005 workshop • September 28-30 at CERN • 115 registered participants • 41 talks • 9 posters • See talks at ROOT web page Overview & Major Developments

  8. ROOT: large user community ROOT is LGPL Overview & Major Developments

  9. Base Work Package New infrastructure features Miscellaneous Overview, Major Developments, Directions

  10. TArchiveFile and TZIPFile • TArchiveFile is an abstract class that describes an archive file containing multiple sub-files, like a ZIP or TAR archive. • The TZIPFile class describes a ZIP archive file containing multiple ROOT sub-files. Notice that the ROOT files should not be compressed when being added to the ZIP file, since ROOT files are normally already compressed. To create the file multi.zip do: • The ROOT files in an archive can be simply accessed like this: • A TBrowser and TChain interface will follow shortly. zip –n root multi file1.root file2.root TFile *f = TFile::Open("multi.zip#file2.root") or TFile *f = TFile::Open("root://mymachine/multi.zip#2") Overview & Major Developments

  11. Replica of a DB subset T0 T1 local TZipFile remote TZipFile http, xrootd, castor, dcache.. Overview & Major Developments

  12. SLAC’s New File Server - xrootd • The file server xrootd (eXtended ROOT daemon) has been developed by Andy Hanushevsky of SLAC. • The server exploits a multithreaded architecture to provide high-performance file based access, focusing on scalability and fault tolerance. The server is being extensively used by BaBar, Star and Alice. • The xrootd file server will in the near future replace the current daemon rootd. • xrootd and its plugin facility is heavily used by PROOF Overview & Major Developments

  13. The New xrootd Client - TXNetFile • The new client class TXNetFile implements the xrootd protocol and is provided to open a file via the xrootd daemon. • TXNetFile can detect when it talks to on old rootd daemon and return a TNetFile. • To open a file via xrootd, just use the standard static method TFile::Open() as for opening via rootd. Overview & Major Developments

  14. Class TGrid (abstract interface) //--- General GRID const char *GridUrl() const const char *GetGrid() const const char *GetHost() const const char *GetUser() const const char *GetPw() const const char *GetOptions() const Int_t GetPort() const //--- Catalogue Interface virtual TGridResult *Command(const char *command, Bool_t interactive = kFALSE, UInt_t stream = kFALSE) virtual TGridResult *Query(const char *path, const char *pattern, const char *conditions, const char *options) virtual TGridResult *LocateSites() virtual TGridResult *ls(const char*ldn ="", Option_t*options ="") virtual Bool_t cd(const char*ldn ="",Bool_t verbose =kFALSE) virtual Bool_t mkdir(const char*ldn ="", Option_t*options ="") virtual Bool_t rmdir(const char*ldn ="", Option_t*options ="") virtual Bool_t register(const char *lfn , const char *turl , Long_t size, const char *se, const char *guid) virtual Bool_t rm(const char*lfn , Option_t*option ="") //--- Job Submission Interface virtual TGridJob *Submit(const char *jdl) virtual TGridJDL *GetJDLGenerator() //--- Load desired plugin and setup conection to GRID static TGrid *Connect(const char *grid, const char *uid, const char *pw, const char *options) Overview & Major Developments

  15. Access to File Catalogues eg Alien FC Same style interface could be implemented for Other GRID File Catalogues Overview & Major Developments

  16. TGrid example with Alien // Connect TGrid alien = TGrid::Connect(“alien://”); // Query TGridResult *res =alien.Query (“/alice/cern.ch/user/p/peters/analysis/miniesd/”, ”*.root“); // List of files TList *listf = res->GetFileInfoList(); // Create chain TChain chain(“Events", “session"); Chain.AddFileInfoList(listf); // Start PROOF TProof proof(“remote”); // Process your query Chain.Process(“selector.C”); Overview & Major Developments

  17. Auto Loading of Plugins • Support for auto-loading libraries when an unknown class is being referenced. • The auto-loading mechanism reads the files $ROOTSYS/etc/system.rootmap, ~/.rootmap and ./.rootmap (via TEnv) to try to map the unknown class to a library. • If the library is found it, and the libraries on which it depends, are loaded. • The rootmap files are created with the rlibmap tool when executing "make map". • Example: in an interactive session, one can do directly without having to do TLorentzVector v; gSystem->Load(“libPhysics”); Overview & Major Developments

  18. TMacro • This class allows for storing a C++ macro in a ROOT file. • In addition to being stored in a ROOT file a TMacro can be executed, edited, etc. TMacro m("Peaks.C"); //macro m with name "Peaks" is created //from file Peaks.C m.Exec(); //macro executed with default arguments m.Exec("4"); //macro executed with argument m.SaveSource("newPeaks.C"); TFile f("mymacros.root","recreate"); m.Write(); //macro saved to file with name "Peaks" Overview & Major Developments

  19. Using PCRE for Reg Exp’s • new class TPRegexp which uses the Perl Compatible Regular Expressions library. • Well know, rich, regular expression syntax. • It is interfaced to TString and other class and methods now using TRegexp. • TRegexp will of course stay for backward compatibility. Overview & Major Developments

  20. Dict Work Package New version of Reflex New version of rootcint rootcint  CINT rootcint -> Reflex ->Cintex ->CINT rootcint ->gccxml -> Reflex -> CINT Adapt PyRoot to Reflex Adapt CINT to Reflex Overview, Major Developments, Directions

  21. Dictionaries : root only Root meta C++ CINT DS ROOT X.h CINT API rootcint CINT XDictcint.cxx Overview & Major Developments

  22. Dictionaries : situation today REFLEX API lcgdict XDictlcg.cxx REFLEX DS X.xml Root meta C++ cintex gccxml CINT DS ROOT CINT API X.h rootcint CINT XDictcint.cxx Overview & Major Developments

  23. Dictionaries : situation in the future Python CINT Root meta C++ Reflex/Cint DS ROOT CINT/Reflex API rootcint -cint XDictcint.cxx rootcint -reflex X.h rootcint -gccxml Overview & Major Developments

  24. IO work-package Consolidation, Consolidation, Consolidation Support for STL collections More cases in auto schema evolution Better support for references Bitmap index TreeSQL Overview, Major Developments, Directions

  25. ROOT I/O: STL Collections • ROOT now supports I/O of all STL containers • std::vector<T> std::list<T> std::set<T> std::deque<T>std::map<K,T> std::multimap<K,T> • And implicitly (through std::deque) std::queue<T> std::stack<T> • STL collections are saved in split mode • Objects are split (but: NOT if pointers) • Quick pre-selections on trees • Interactivity: Trees can be browsed • Save space (see $ROOTSYS/test/bench):std::vector<THit>:compression 5.38std::vector<THit*>:compression 3.37 Overview & Major Developments

  26. Float, double and space…(1) • Math operations very often require doubleprecision, but on saving single precision is sufficient… • New data type: Double32_tIn memory: doubleOn disk: float or integer Overview & Major Developments

  27. Float, double and space… (2) • Usage (see tutorials/double32.C): Double32_t m_data; // [min,max<,nbits>] • No nbits,min,max: saved as float • min, max: saved as int 32 bits precision explicit values or expressions of values known to Cint (e.g. “pi”) • nbits present: saved as int with nbit precision higher precision than float for same persistent space Overview & Major Developments

  28. Float, double and space… (3) Increase precision Save space Overview & Major Developments

  29. File types & Access in 5.06 user Local File X.xml TTreeSQL TFile TKey/TTree TStreamerInfo TSQLServer TSQLRow TSQLResult http rootd/xrootd Oracle Local File X.root MySQL Dcache Castor PgSQL RFIO Chirp SapDb Overview & Major Developments

  30. Bitmap Indices • Bitmap indices are efficient data structures for accelerating multi-dimensional queries: • E.g. pT > 195 AND nTracks < 4 AND muonTight1cm > 12.4 • Supported by most commercial database management systems and data warehouses • Optimized for read-only data • However, because an efficient index may be as big as the data, we think that it is only appropriate for things like event meta data catalogues Overview & Major Developments

  31. Query Performance - TTreeFormula vs. Bitmap Indices Bitmap indices 10X faster than TTreeFormula Overview & Major Developments

  32. Data analysis with bitmap indices Event catalogue Bitmap index Direct use by PROOF slaves to select events query Event list Overview & Major Developments

  33. Math work-package MathCore MathMore Minuit2 Smatrix Linear & Robust Fitter Splot Overview, Major Developments, Directions

  34. MATH work-package : News • MathCore library with basic Math functionality • Basic Special and statistical functions • Physics and geometry vectors • MathMore library • C++ interface to function and algorithm from GSL • Extra math functions, Adaptive integration, derivation, root finders • Minuit2 • New OO implementation of Minuit • Interface to ROOT TVirtualFitter • Linear and Robust Fitter • sPlot Overview & Major Developments

  35. New Math Libraries organization Overview & Major Developments

  36. MATH work-package : Plan • Complete MathCore with Random numbers • Adapt ROOT classes to MathCore • TF1,2,3, Fitting • Virtual Fitter extensions • corresponding changes in ROOT fitting and roofit • Fully integrate and extend new Minuit • Fitting GUI • Box plots, qqplots • Many new tools required for LHC Physics analysis (PHYSTAT05 Oxford) Overview & Major Developments

  37. Linear Fitter (0) • To fit functions linear in parameters • Polynomials, hyperplanes, linear combinations of arbitrary functions • TLinearFitter can be used directly or through TH1, TGraph, TGraph2D::Fit interfaces • When used directly, can fit multidimensional functions Overview & Major Developments

  38. Linear Fitter (1) • Special formula syntax: • Linear parts separated by “++” signs: • “1 ++ sin(x) ++ sin(2*x) ++ cos(3*x)” • “[0] + [1]*sin(x) + [2]*sin(2*x) + [3]*cos(3*x)” • Simple to use in multidimensional case • “x0 ++ x1 ++ exp(x2) ++ log(x3) ++ x4” • Polynomials (pol0, pol1…) and hyperplanes (hyp1, hyp2, …) are the fastest to compute • By default, polynomials in TH1, TGraph::Fit functions now go through Linear Fitter • Data to be used for fitting is not copied into the fitter Overview & Major Developments

  39. Linear Fitter (2) • Advantages in separating linear and non-linear fitting: • Doesn’t require setting initial parameter values • The gain in speed Overview & Major Developments

  40. Robust fitting (0) • Least Trimmed Squares regression – extension of the TLinearFitter class • Motivation: least-squares fitting is very sensitive to bad observations • Robust fitter is used to fit datasets with outliers • The algorithm tries to fit h points (out of N) that have the smallest sum of squared residuals Overview & Major Developments

  41. Robust fitting (1) • Highbreakdown point - smallest proportion of outliers that can cause the estimator to produce values arbitrarily far from the true parameters Graph.Fit(“pol3”, “rob=0.75”, -2, 2); 2nd parameter – fraction h of the good points Overview & Major Developments

  42. Multivariate covariance • Minimum Covariance Determinant Estimator – a highly robust estimator of multivariate location and scatter • Motivation: arithmetic mean and regular covariance estimator are very sensitive to bad observations • Class TRobustEstimator • The algorithm tries to find a subset of h observations (out of N) with the minimal covariance matrix determinant Overview & Major Developments

  43. Multivariate covariance • Left – covariance ellipses of a 1000-point dataset with 250 outliers • Right – distances of points from the robust mean, calculated using robust covariance matrix • High breakdown point • Indices of outlying points can be returned Overview & Major Developments

  44. sPlot – A statistical tool to unfold data distributions Left – Projection plot cut on the likelihood ratio Excess of events – Signal? Background? Right – sPlot – no cut getting rid of background by statistical methods Signal!!! Overview & Major Developments

  45. News in TH1 and TF1 • TH1: • Chi2 test • Mean & RMS error, skewness and kurtosis • TF1: • Derivatives (1st, 2nd and 3rd) • Improved minimization – a combination of grid search and Brent’s method (golden section search and parabolic interpolation) Overview & Major Developments

  46. Graphics work-package zillions of micro/mini features http://couet.home.cern.ch/couet/POW_files/frame.htm reimplement (TGaxis) GL with new GUI GL for dynamic tracks GL in Pad Overview, Major Developments, Directions

  47. Overview & Major Developments

  48. TImageDump • Many extensions to the libAfterImage library to support line, marker and (filled) polygon drawing. Accessible via TASImage. • The new class TImageDump uses TASImage and derives from TVirtualPS to allow the saving of canvases in gif, jpg, png, tiff, etc., image formats in batch mode: • Or to display any gif, jpg, png, tiff in a canvas, do: $ root –b root [0] .x hsimple.C root [1] c1->Print("c1.gif"); TCanvas *c1; TImageDump *imgdump = new TImageDump("test.png"); c1->Paint(); imgdump->Close(); Overview & Major Developments

  49. GL in Pad Overview & Major Developments

  50. Overview & Major Developments

More Related