1 / 72

ROOT Project Status Summary of major developments

ROOT Project Status Summary of major developments. ROOT Workshop 2004 SLAC 25 February Ren é Brun CERN. Project History. Jan 95: Thinking/writing/rewriting/??? November 95: Public seminar, show Root 0.5 Spring 96: decision to use CINT Jan 97: Root version 1.0 Jan 98: Root version 2.0

janetkirkman
Download Presentation

ROOT Project Status Summary of major developments

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. ROOT Project StatusSummary of major developments ROOT Workshop 2004 SLAC 25 February René Brun CERN Overview & Major Developments

  2. Project History • Jan 95: Thinking/writing/rewriting/??? • November 95: Public seminar, show Root 0.5 • Spring 96: decision to use CINT • Jan 97: Root version 1.0 • Jan 98: Root version 2.0 • Mar 99: Root version 2.21/08 (1st Root workshop FNAL) • Feb 00: Root version 2.23/12 (2nd Root workshop CERN) • Mar 01: Root version 3.00/06 • Jun 01: Root version 3.01/05(3rd Root workshop FNAL) • Jan 02: Root version 3.02/07 (LCG project starts: RTAGs) • Oct 02: Root version 3.03/09 (4th Root workshop CERN) • Dec 03: Root version 3.10/02 (last PRO release) • Feb 04: Towards version 4.00 (5th Root workshop SLAC) 9 years !! Overview & Major Developments

  3. Plan of talk • The ROOT Team • Summary of developments in 3.04,3.05,3.10 • Version 4.00 highlights • Medium term and long term • Relations with LCG/AA Overview & Major Developments

  4. Today full time on the project • Ilka Antcheva (LCG staff) (since 1st Aug 2002) GUI/Doc • Maarten Ballintijn (MIT/Phobos) PROOF (since Sep 2001) • Rene Brun: PH/SFT group and Alice part time • Philippe Canal (FNAL/CD) (since 1998) • Olivier Couet CERN (since 1st Jun 2002) Graphics • Gerri Ganis (LCG) (since Dec 2002)Authentication • Andrei Gheata: (Alice) Geometry package (since Sep 2001) • Masa Goto (Agilent technologies) CINT fulltime! • Eddy Offermann (from Finance world. Sabatical for 1 year) • Valeriy Onuchin (LCG) GUI/Win32gdk (since 1st Feb 2002) • Fons Rademakers: Alice and PH/SFT group Overview & Major Developments

  5. Also a substantial fraction of their time • Bertrand Bellenot (Alcan) Win32gdk (since June 2000) • Valery Fine (BNL/STAR) TVirtualX/Qt + Coin3D • Victor Perevoztchikov (BNL/STAR) STL, foreign classes • AND more than 70 important contributions from people spending a substantial fraction of their time on the project. See $ROOTSYS/README/CREDITS Overview & Major Developments

  6. New Splash Screen To see the list of all contributors, select the HELP button in the browser or canvas. The window auto scrolls Until you click on it. Appears 4s When executing root Overview & Major Developments

  7. CINT & Masa • Continuous and impressive list of developments by Masa (see his talk) • towards full C++ standard ? • Still problems with object scope in loops • CINT is fundamental to ROOT • ROOT puts many difficult requirements to CINT • Hook when object does not exist • Hook when an object is deleted • Many thanks to Masa for 9 years of very fruitful collaboration and 12 years of work on CINT ! Overview & Major Developments

  8. Special thanks to: • FermiLab for supporting Philippe since many years. Philippe is a KEY developer • CERN/LCG for providing manpower for GUI, Windows support and Authentication. • BNL for supporting Valeri and Victor since many years (via Star->Atlas->Star). • BaBar for supporting the xrootd development • Our colleagues in LCG/AA and PH/SFT for stimulating discussions Overview & Major Developments

  9. Roottalk Mails to roottalk + mails to rootdev and root-bugs 1323 in 1997 2515 in 1998 3004 in 1999 3567 in 2000 4686 in 2001 5370 in 2002 5393 in 2003 +1419 in Forum (August 2003->Feb 2004) + mails about-root-fnal + direct mails to team members + spam mail PLEASE: -Always indicate a mail subject -Always indicate which version of ROOT and which OS -Send a x.tar.gz file instead of many files in attachement Overview & Major Developments

  10. Short summary of main developments since ROOT2002 • Dec 2002: version 3.04/07 • Jun 2003: version 3.05/07 • Dec 2003: version 3.10/02 • Feb 2004: version 4.00/02 (released in June) http://root.cern.ch/root/htmldoc/examples/Version30402.news.html http://root.cern.ch/root/htmldoc/examples/Version31002.news.html http://root.cern.ch/root/html/examples/V4.00.txt.html We have implemented more than 80% of our plan announced at the previous workshop in October 2002. See: ftp://root.cern.ch/root/R2002/future.pdf We have implemented many new features not foreseen in Oct 2002 Overview & Major Developments

  11. Summary of 3.04, 3.05 and 3.10 October 2002 ->December 2003 Overview & Major Developments

  12. ROOT Documentation Overview & Major Developments

  13. New ports • Added port to IBM's xlC compiler under MacOS X, by Federico Carminati. • Port to Microsoft's VC++ 7.1. • Port to Intel's icc v8.0. • Port to RedHat 8.x, 9.x. • Port to Fedora Core 1. • Port to Cygwin gcc, by Axel Naumann. • Port to AMD Opteron and Athlon64 Overview & Major Developments

  14. Plug-in Manager name class Shared lib How to call Plugin.TFile: ^rfio: TRFIOFile RFIO "TRFIOFile(const char*,Option_t*,const char*,Int_t)" +Plugin.TFile: ^castor: TCastorFile RFIO "TCastorFile(const char*,Option_t*,const char*,Int_t,Int_t)" +Plugin.TFile: ^dcache: TDCacheFile DCache "TDCacheFile(const char*,Option_t*,const char*,Int_t)" +Plugin.TFile: ^chirp: TChirpFile Chirp "TChirpFile(const char*,Option_t*,const char*,Int_t)" Plugin.TSystem: ^rfio: TRFIOSystem RFIO "TRFIOSystem()" Plugin.TSQLServer: ^mysql: TMySQLServer MySQL "TMySQLServer(const char*,const char*,const char*)" +Plugin.TSQLServer: ^pgsql: TPgSQLServer PgSQL "TPgSQLServer(const char*,const char*,const char*)" +Plugin.TSQLServer: ^sapdb: TSapDBServer SapDB "TSapDBServer(const char*,const char*,const char*)" +Plugin.TSQLServer: ^oracle: TOracleServer Oracle "TOracleServer(const char*,const char*,const char*)" Plugin.TGrid: ^alien TAlien RAliEn "TAlien(const char*,const char*,const char*,const char*)" Plugin.TVirtualPad: * TPad Gpad "TPad()" Plugin.TVirtualHistPainter: * THistPainter HistPainter "THistPainter()" Plugin.TVirtualTreePlayer: * TTreePlayer TreePlayer "TTreePlayer()" Plugin.TVirtualTreeViewer: * TTreeViewer TreeViewer "TTreeViewer(const TTree*)" Plugin.TVirtualGeoPainter: * TGeoPainter GeomPainter "TGeoPainter()" Plugin.TVirtualUtil3D: * TUtil3D Graf3d "TUtil3D()" Plugin.TVirtualUtilHist: * TUtilHist Hist "TUtilHist()" Plugin.TVirtualUtilPad: * TUtilPad Gpad "TUtilPad()" Plugin.TVirtualFitter: Minuit TFitter Minuit "TFitter(Int_t)" +Plugin.TVirtualFitter: Fumili TFumili Fumili "TFumili(Int_t)" Plugin.TVirtualPS: ps TPostScript Postscript "TPostScript()" +Plugin.TVirtualPS: svg TSVG Postscript "TSVG()" Plugin.TViewerX3D: x11 TViewerX3D X3d "TViewerX3D(TVirtualPad*,Option_t*)” +Plugin.TViewerX3D: qt TQtViewerX3D QtX3d "TQtViewerX3D(TVirtualPad*,Option_t*)” Overview & Major Developments

  15. ACLIC • The '+' option now properly recreates the library when the script or one of the file it includes is newer than the shared library. • When asked to load a script (file.C) which has already been compiled (into file_C.so), ACLiC looks for a dependency file (file_C.d). If this dependency file is older than the library or does not exist, ACLiC generates a new dependency file using ROOT's rmkdepend tool and the list of -I and -D directives it knows about (see ACLiC description on how to set those). • Then ACLiC compare the time stamp of all the files listed in the dependency file and rebuild the library if any of them is newer than the library Root >.x file.C++ Root > .x file.C+ Root >.x file.C++g Overview & Major Developments

  16. Rootcint (see Philippe talk) • Many developments. Consolidation • Digestion of STL collections • Better support for templates • Dictionary size: a problem ? Overview & Major Developments

  17. ROOT Dictionary size If all classes have a dictionary, the size of the dictionary may become a large fraction of the executable module! Overview & Major Developments

  18. Full native support for 64 bit integers • Support on all platforms for 64 bits integers via the portable typedefs Long64_t and ULong64_t. • Long long on Unix • _int64 with VC++ • Big-endian, little-endian byte swapping • Many internal class tables moved to Long64_t • Still more classes/functions to be converted • fEntries in TH1, TTree • Replacememt of TTree::BuildIndex internal tables with an array of ULong64_t • TEventList ? Overview & Major Developments

  19. Trees improvements • Huge number of improvements (see Philippe) • Support for split mode in more complex cases • Performance improvement • Support for foreign classes as top branches #include “TObject.h” Class MyClass : public TObject { int i,j,k; ClassDef(MyClass,1) }; TTree T(“T”,”test foreign”) MyClass *a = new MyClass; T.Branch(“event”,”MyClass”,&a); Class MyClass { int i,j,k; }; Overview & Major Developments

  20. TTreeViewer • Several layout improvement and changes. New Refresh button. Overview & Major Developments

  21. TSelector • The TSelector class has been upgraded to improve its general functionality and better support its use in PROOF. Specific changes: • - Introduction of a Version() function to provide backward compatibility for the old version and support future upgrades. • - Replace ProcessCut() and ProcessFill() with a single Process() function. Reduce the information that needs to be shared between functions. Promote a more granular access to data by reading branches as they are needed. • Add SlaveBegin() and SlaveTerminate(), in the case of PROOF these routines are called on the slave servers. The Begin() and Terminate() function now always run on the (local) client machine even when using PROOF. Overview & Major Developments

  22. rootd • The rootd daemon uses the new authentication methods described in detail at: http://root.cern.ch/root/Auth.html • Client/Server Authentication Client/Server authentication for access to data on remote hosts (via the rootd daemon) or to a PROOF cluster has been deeply restructured.. • See Gerri talk Overview & Major Developments

  23. PROOF (see Maarten & Fons) • improved packetizer (work distributor). • interface to AliEn grid system via the TGridProof/TAlienProof classes. • interface to Condor Computing on Demand via TCondor class. - query progress dialog • introduced the Package Manager and Sandbox: • gProof->UploadPackage() • gProof->EnablePackage() • gProof->ClearPackage() Overview & Major Developments

  24. Fitting/TMinuit/TFumili Choose Fitter at run time or via system.rootrc void FitBench(Int_t npass=100) { TCanvas *c1 = new TCanvas("c1"); c1->Divide(1,2); fitFcn = new TF1("fitFcn",fitFunction,0,3,6); fitFcn->SetNpx(200); gStyle->SetOptFit(); gStyle->SetStatY(0.6); //with Minuit c1->cd(1); TVirtualFitter::SetDefaultFitter("Minuit"); DoFit(npass); //with Fumili c1->cd(2); TVirtualFitter::SetDefaultFitter("Fumili"); DoFit("Fumili",npass); } Overview & Major Developments

  25. TSpectrum • New improvements in the Spectrum classes by Miro Morach. Overview & Major Developments

  26. TGraph2D • This new class is a set of N points x[i],y[i],zi[i] in a non uniform grid. Several visualization techniques are implemented, including Delaunay triangulation. TCanvas c; Double_t x, y, z, P = 6.; Int_t np = 200; TGraph2D dt; TRandom r; for (Int_t N=0; N<np; N++) { x = 2*P*(r.Rndm(N))-P; y = 2*P*(r.Rndm(N))-P; z = (sin(x)/x)*(sin(y)/y)+0.2; dt.SetPoint(N,x,y,z); } gStyle->SetPalette(1); dt.Draw("surf1"); dt.Draw(“same p0”); Overview & Major Developments

  27. TMultiLayerPerceptron package(see Christophe Delaere talk) • New MultiLayerPerceptron (mlp) package, by Christophe Delaere. The package has 3 classes TMultiLayerPerceptron, TNeuron, TSynapse These classes describe a neural network. There are facilities to train the network and use the output. • The input layer is made of inactive neurons (returning the normalized input), hidden layers are made of sigmoids and output neurons are linear. The basic input is a TTree and two (training and test) TEventLists. • For classification jobs, a branch (maybe in a TFriend) must contain the expected output. 6 learning methods are available: kStochastic, kBatch, kSteepestDescent, kRibierePolak, kFletcherReeves and kBFGS. This implementation is *inspired* from the mlpfit package from J.Schwindling et al. Overview & Major Developments

  28. New tutorials in $ROOTSYS/tutorial • graph2dfit.C: TGraph2D example. • graph2derrorsfit.C: TGraph2DErrors example. • markerwarning.C illustrating the danger of using asymmetric markers. • logscales.C illustrating the use of axis with log scales. • triangles.C illustrating how to interact with closed polylines. • multigraph.C illustrating the use of TMultiGraph. • mlpHiggs.C illustring the use of the NN classes TMultiLayerPerceptron, etc. • bent.C illustrating the use of the new class TGraphBentError • peaks.C illustrating how to use the peak finder (class TSpectrum) associated with a complex minimization problem. • TestAuth.C to test the ROOT Authentication methods. • threads.C illustrating the use of the Thread classes. • geoshapes.C illustrating the use of the TGeo shapes. • timeonaxis2.C illustrating the basics of the time units on the histogram axis. Overview & Major Developments

  29. New tutorials in $ROOTSYS/tutorial • clonesA_Event.C illustrating complex situations of inheritance and multiple inheritance. • jets.C illustrating the use of a Tree using the JetEvent class. The JetEvent class has several collections (TClonesArray) and other collections (TRefArray ) referencing objects in the TClonesArrays. • tree2a.C with the same functionality than tree2.C but using a class instead of a C-struct. • copytree3.C showing how to copy selected entries to another Tree. Overview & Major Developments

  30. Version 4.00 hightlights Overview & Major Developments

  31. ROOT version 4 Highlights (1)(already in 4.00/02) • Support for Automatic Schema evolution for foreign classes without using a class version number. • Support for large files (> 2 GBytes) • New data type Double32_t (double in memory, saved as float on the file) • Native support for STL: automatic Streamers. • Tree split mode with STL vector/list • New Linear Algebra package • New class TRolke by Jan Conrad • New class TGraph2DErrors. More features in TGraph2D • Consolidation of win32gdk. • More features in the authentication system. • Consolidation of the PROOF classes. • GUI code generation See Philippe See Eddy See Gerri See Fons & Maarten See Ilka Overview & Major Developments

  32. ROOT version 4 Highlights (2)(will be in final release in June) • Plug-in Manager upgrade (rlimap) with automatic library/class discovery/load. • xrootd • Virtual MC (G3, Fluka, G4?) • Support for Qt (alternative option to x11) • GUI builder • PROOF/Alien in production • Interface with Python (PyRoot with SEAL) • Interface with Ruby (ruby-root) • Geometry interface to OpenGL/Coin3D (V.Fine/STAR) • MakeClass/MakeSelector upgrade. Support for Tree friends • New GUI Histogram editor and Fitter see BaBar talks see Andrei Gheata /Alice see Valeri Fine/STAR see Ilka see Fons see Wim /Atlas/LBL Overview & Major Developments

  33. Foreign classes checksum foreign class #include “Rtypes.h” class MyClass { int i,j,k; ClassDef(MyClass,1) }; class MyClass { int i,j,k; }; buffer .... Bytecount 0 checksum ObjectN Bytecount 0 checksum objectN+1 .... buffer 4 bytes .... bytecountversion ObjectN bytecountversion objectN+1 .... 2 bytes 4 bytes 4 bytes 2 bytes Overview & Major Developments

  34. ClassDef advantages • The IsA function generated by ClassDef speeds up considerably the access to the TClass for a given object. • The version number (2 bytes maximum) consumes less space on disk than the “0+checksum” Overview & Major Developments

  35. Support for large files • By default, a TFile will grow above 2 GBytes if necessary. • A TTree will only go above 2 GBytes if TTree::SetMaxTreeSize has been called. • Below 2 GBytes, all pointers are 4 bytes. Above 2 GBytes, they are 8 bytes. This saves some space when writing many small keys/baskets. • If a file is > 2GBytes, fSeekKey or fBasketSeek are always 8 bytes. • A file written with version 4 can be read by version 3 if the file size is < 2 GBytes. Overview & Major Developments

  36. New Data type Double32_t • A new data type Double32_t is defined in Rtypes.h • Same as Double_t in memory • Same as Float_t on disk • The Automatic Schema evolution supports conversions from • Double32_t to Double_t and vice-versa • Double32_t to Float_t and vice-versa • Double32_t* to Double_t* and vice-versa • Double32_t* to Float_t* and vice-versa • This new data type may save a non negligible space on disk. • However be careful with loss of precision when making write->read->write->read->write… Overview & Major Developments

  37. New Graphics Editor 1-Toggle Editor mode 3-Edit object attributes 2-Click On the object to be edited Overview & Major Developments

  38. Gui Builder/Gui Code Generator Type CRTL/S anywhere on a widget To generate the C++ code for this widget Then do root > .x guisave.C Overview & Major Developments

  39. Virtual Monte Carlo VMC • The VMC has considerably evolved in the past two years.The VMC was designed to have • A common input for the kinematics • A common input for the geometry • A common output for the hits (StepManager) • The original version was feeding the final target system (G3 or G4) via a common API. • In addition the new version has interfaces from G3 and Fluka to the Root geometry (G4 in plan) • In case of G3, one can run with the native G3 geometry or the ROOT geometry with no changes in the application. (See Andrei Gheata talk) Overview & Major Developments

  40. Geometry classes TGeoxxx Modelling Visualisation Interactivity Where am I? Distance to boundary Closest boundary Persistency Simulation program Geant3-based Geant4-based Fluka-based C++ classes Geometry package MySQL XML Reconstruction program Visualisation Overview & Major Developments

  41. Geometry and Geometries C++ classes G3 Geant3 rz file C++ classes C++ classes geometry G4 ROOT file geometry Geometry In memory Fluka XML Files Eg, GDML Recons Overview & Major Developments

  42. Geometries: not the same goal ! External description only Used as input to a real geometry (G4, ROOT) Checker , Viewer may be implemented Requires some data structure in memory This has very limited functionality. Interesting (may be?) for input. Too much emphasis on this solution XML Files Eg, GDML Simulation/Reconstruction oriented C++ API for the construction Input can be via first solution Checker, Viewer must be (are) implemented Provide interface to navigators THIS IS THE MAIN HORSE TO BE OPTIMIZED Geometry In memory (G3,G4,ROOT) Overview & Major Developments

  43. New Linear Algebra classes • Eddy Offermann has kindly maintained the old TMatrix, TVector classes for a few years. • Eddy is now spending one year with us at CERN.He has developed a new set of classes with more functionality and better performance. • See Eddy’s talk on Friday TDecompBase TDecompChol TDecompLU TDecompQRH TDecompSVD TMatrix TMatrixD TMatrixDBase TMatrixDCramerInv TMatrixDEigen TMatrixDLazy TMatrixDSym TMatrixDSymEigen TMatrixDUtils TMatrixF TMatrixFBase TMatrixFCramerInv TMatrixFLazy TMatrixFSym TMatrixFUtils TVector TVectorD TVectorF Overview & Major Developments

  44. New Linear Algebra classes Overview & Major Developments

  45. New class TRolke by Jan Conrad • This class computes confidence intervals for the rate of a Poisson in the presence of background and efficiency with a fully frequentist treatment of the uncertainties in the efficiency and background estimate using the profile likelihood method. • The signal is always assumed to be Poisson. • It allows the following Models: • 1: Background - Poisson, Efficiency - Binomial (cl,x,y,z,tau,m) • 2: Background - Poisson, Efficiency - Gaussian (cl,xd,y,em,tau,sde) • 3: Background - Gaussian, Efficiency - Gaussian (cl,x,bm,em,sd) • 4: Background - Poisson, Efficiency - known (cl,x,y,tau,e) • 5: Background - Gaussian, Efficiency - known (cl,x,y,z,sdb,e) • 6: Background - known, Efficiency - Binomial (cl,x,z,m,b) • 7: Background - known, Efficiency - Gaussian (cl,x,em,sde,b) • See example in tutorial Rolke.C Overview & Major Developments

  46. Plugin manager & rlimap • An extension of the Plugin Manager and a new tool rlimap are currently under development (now in CVS). • $ROOTSYS/bin rlimap is a new tool generating the list of classes for a shared lib • rlimap libxxx.so libXXX.map • At run time, when CINT finds an unknown class, it calls a new Root module that is able to find the shared lib containing the class by looking into all libXXX.map files in the LD_LIBRARY_PATH. Eg. • TLorentzVector v; • will automatically trigger gSystem->Load(“libPhysics”); • gSystem->Load(libXXX”) can still be called Overview & Major Developments

  47. Python/Root & Ruby/Root interfaces • We consider CINT + ACLIC to be the right combination for interactive data analysis • Having in addition an interface with other scripting languages is a bonus. We do not see these scripting languages to be an alternative to CINT/ACLIC, but an interesting complement. We are collaborating with: • LCG/AA for PyRoot (Pere Mato, Wim Lavrijsen) • Juan Alcaraz for PythonRoot & RubyRoot • Elias Athanasopoulos for ruby-root • It would be interesting to have a similar interface for PERL. • See also JavaRoot • http://sarkar.home.cern.ch/sarkar/jroot/main.html root > .x script.C root > .x script.C+ root > gSystem->Load(“libClasses”) Overview & Major Developments

  48. PyRoot • PyRoot (successor of RootPython by Pere Mato) is an automatic interface between Python and any class having a CINT dictionary. • It does not require code generation. • We will distribute PyRoot as a sub-ROOT component, once PyRoot is stable. • See talk by Wim Lavrijsen Overview & Major Developments

  49. PythonRoot http://alcaraz.home.cern.ch/alcaraz/PythonRoot/ Overview & Major Developments

  50. RubyRoot http://alcaraz.home.cern.ch/alcaraz/RubyRoot/ Overview & Major Developments

More Related