1 / 29

A TLAS M etadata I nterface. Portal page. ami2p3.fr/

A TLAS M etadata I nterface. Portal page. http://ami.in2p3.fr/ A generic cataloging framework – used in ATLAS for dataset discovery. AMI. Data in AMI is organized in CATALOGUES. Catalogues are assigned to a project and sub-project. mc08-Production (Holds 2008 MonteCarlo datasets)

jera
Download Presentation

A TLAS M etadata I nterface. Portal page. ami2p3.fr/

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. ATLAS Metadata Interface. Portal page. http://ami.in2p3.fr/ A generic cataloging framework – used in ATLAS for dataset discovery AMI • Data in AMI is organized in CATALOGUES. Catalogues are assigned to a project and sub-project. • mc08-Production (Holds 2008 MonteCarlo datasets) • data08-RealData (cosmics and first beam) 1

  2. Dataset Discovery • Means finding the names of valid datasets to use in your analysis. • An ATLAS (container) dataset is either • A number of files output of DAQ, and managed by Tier 0. (RAW data and fast reconstruction) • A number of files produced by one or more production system tasks. Each task makes a set of (TID) datasets. (MC and reprocessing.) • The distributed data management (DDM) manages datasets as containers of task datasets. • Physicists doing analysis normally should worry only about the containers, and let the infrastructure take care of the rest.

  3. An introduction to datasets and their nomenclature Project.runNumber.streamType.productionStep.dataType.version[/] Project.datasetNumber.physicsShort.productionStep.dataType.version[/] dataNN_* or mcNN_* ESD, AOD digit, recon, merge, etc… Concatanation of configurations

  4. mc09.106099.PhojetCdiff.evgen.EVNT.e456

  5. Where does AMI get its data? Real data : From the Tier 0 : DAQ data, and first reconstruction is registered in AMI < 5 minutes after it is declared “ready” (both datasets and files). Config tags for real data. Monte Carlo and reprocessing. From the Task Request DB : Tasks, dataset names, MC and reprocessing config tags From the production DB : Finished tasks – files and metadata. From physicists. M.C. GEN datasets MC Dataset number info, physics group owner,… DPD tags …

  6. A precision about REAL DATA registration in AMI Users may find data in DQ2 several hours before AMI sees the dataset. Start of Run. Tier 0 declares empty datasets in DQ2 data09_something Tier 0 adds files to the dataset as they arrive from SFO (LB by LB) End of Run. Dataset closed in DQ2. AMI Flag raised. Arrives in AMI within 5 minutes.

  7. Overview page :https://ami.in2p3.fr/AMI/servlet/net.hep.atlas.Database.Bookkeeping.AMI.Servlet.Command?linkId=246 Notice that not all datasets are VALID. Invalid because all tasks failed, declared bad, or deleted. Invalid datasets are hidden by default. Project tags are controlled, for official datasets; they follow the nomenclature specification and are declared by data preparation coordination. (data09_7TeV for example) If you know the project tag this is perhaps the simplest way to get to the list.

  8. The “Simple” search Simple Search – search on part of the name. Example “L1Calo “ Note that “%” is used for wild carding. Search on keywords, and comments – a search in many fields – takes longer. Tutorial link: http://ami.in2p3.fr:8080/opencms/opencms/AMI/www/Tutorial/Simple_search_interface.html

  9. The “Advanced” search Examples to try: Physics group = “phys-beauty” Notice that there are results in several catalogues. Tutorial link: http://ami.in2p3.fr:8080/opencms/opencms/AMI/www/Tutorial/Advanced_search_interface.html This interface will de redesigned in the next few months.

  10. Looking at the results. Use any search interface to get “physics_MinBias ” data for 2009 Use the “group by” under data_type to find the AOD datasets. (409 2010-02-08) “Filter” on the dataset name, or the runNumber to find 142383. http://ami.in2p3.fr:8080/opencms/opencms/AMI/www/Tutorial/Exploring_the_results.html http://ami.in2p3.fr:8080/opencms/opencms/AMI/www/Tutorial/Refine_the_Results.html

  11. Looking at the results some more Click on “additional fields” . Notice the prodsysStatus. Choose one which has “EVENTS_AVAILABLE”. (more) Click on “details” – to get the “child” elements. Click on “event_range” to get the prodsys task details. Click on “files” to browse the files. Notice the “history” buttons (“bread crumb trail”). You can even remove them. Click on “Provenance” of dataset. You can use this to navigate along the production chain. For REAL data one gets back to the RAW type. (For MC to EVGEN and even GEN input. From EVGEN you can see the jobOptions.)

  12. Getting to the data Look for an mc09 dataset with “EVENTS_AVAILABLE” mc09_900GeV.105004.pythia_ddiff.merge.AOD.e466_s667_s668_d258_r1026_r1051. Click on “DQ2”. This page shows an aggregation of information from DQ2. You can subscribe to the dataset from this page if you like, going through the DDM permission system of course. AMI will also make you “dq2-get” commands, but in general you should not do it.

  13. Config Tags Example: e466_s667_s668_d258_r1026_r1051 (last field of dataset name) e466 -> event generation parameters s667, s668 -> simulation parameters (simul.HITS, merge.HITS) d258 -> digitization r1026,r1051 -> reconstruction parameters Interpretation of Config tagshttp://ami.in2p3.fr/opencms/opencms/AMI/www/ReferenceTables/ Searching starting from the Config Tag.http://ami.in2p3.fr/opencms/opencms/AMI/www/Tutorial/ConfigTags Comparing tags (try r1026 and r1051) from the simple search page. 13

  14. AMI Accounts Logging on to AMI. In general you do not need to log on to read (at the moment) You can make an AMI account to access a personal page. You must log on for any writing operation. Once you log on to AMI you can make bookmarks. Tutorial link : http://ami.in2p3.fr/opencms/opencms/AMI/www/Tutorial/Other_AMI_basic_functionalities.html

  15. Other stuff. pyAMI. Everything in AMI can be obtained from the python client. http://ami.in2p3.fr/opencms/opencms/AMI/www/Client/pyAMISecure_and_cmt Ad Hoc queries. (For really advanced users!) http://ami.in2p3.fr/opencms/opencms/AMI/www/Tutorial/Refine_the_search.html

More Related