1 / 18

Ariadne: Tutorial

Ariadne: Tutorial. Illya Shapoval CERN, KIPT, UNIFE, INFN-FE. 2 nd LHCb Computing Workshop 4th-8th November 2013 CERN. Content. Introduction Querying Ariadne Updating Ariadne U se case of compatibility. Introduction: terminology. Ariadne knowledge graph (AKG)

dunn
Download Presentation

Ariadne: Tutorial

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. Ariadne:Tutorial Illya Shapoval CERN, KIPT, UNIFE, INFN-FE 2nd LHCb Computing Workshop 4th-8th November 2013 CERN

  2. Content Introduction Querying Ariadne Updating Ariadne Use case of compatibility 2nd LHCb Computing Workshop , 4th-8th November, CERN

  3. Introduction: terminology Ariadne knowledge graph (AKG) A graph holding all LHCb metadata entities (nodes) and relationships (edges) between them (to the best of managers’ knowledge). Ariadne domain data model An abstract graph, describing by its pattern topology of relationships in a specific application domain. Ariadne solution Subgraph of the Ariadne knowledge graph, matching concrete graph pattern, requirements on nodes’ attributes and relationships types. Ariadne thread An Ariadne solution that matches entirely domain data model. 2nd LHCb Computing Workshop , 4th-8th November, CERN

  4. Introduction: metadata identifiers Raw identifier: dictionary of attributes, for use in low-level CL tools, e.g.: {'type': 'application', 'name': 'DaVinci', 'version': 'v33r8'} {'type': 'detector_type', 'name': '2013'} {'type': 'reco_type', 'name': 'Reco14'} Special case: {'type': 'tag', 'partition': 'DDDB', 'name': 'dddb-20120831', 'release_date': '2012-08-31'} Shortcut identifier: Ariadne node index value, for use in high-level CL tools: application:DaVinci:v33r8 detector_type:2013 reco_type:Reco14 tag:DDDB:dddb-20120831 2nd LHCb Computing Workshop , 4th-8th November, CERN

  5. Introduction: Ariadne CL tools Querying Ariadne ariadne_query,high-level query tool, used by you up to now available in the LbLogin environment from anywhere with web access no authentication and authorization (AA) required Updating Ariadne ariadne_add, high-level admin tool, used by you up to now available in the LbLogin environment from anywhere with web access AA required: talks through CERN SSO (valid CERN Kerberos ticket) AriadneAdmin.py: powerful low-level admin tool, avoid using it; available in the “SetupProject Tools --runtime LCGCMT --use Ariadne” environment from anywhere with web access AA required: talks through CERN SSO (valid CERN Kerberos ticket) New:generic tool for all tasks, recommended to use ariadne, high-level admin tool for both updating and querying available in the LbLogin environment from anywhere with web access AA is conditional: no AA for querying, and AA for updating (valid CERN Kerberos ticket) Note1: only members of lhcb-core-soft are authorized to update AKG Note2: where AA is required, cern-get-sso-cookie has to be installed 2nd LHCb Computing Workshop , 4th-8th November, CERN

  6. Querying Ariadne (with CL tool)ariadne q[uery] [OPTIONS] IDENTIFIER [IDENTIFIER2] (see ariadne help, or ariadne query -h ) Check DaVinci v33r8 is tracked: ariadne query application:DaVinci:v33r8 Fetch entities, directly related to DaVinci v33r8: ariadne query -r application:DaVinci:v33r8 Fetch DetectorTypes, directly related to DaVinciv33r8: ariadne q -r application:DaVinci:v33r8 | grep detector Fetch DDDB tags, directly related to DaVinci v33r8: ariadne q -r application:DaVinci:v33r8 | grep DDDB Check DaVinciv33r8 and DetectorType 2013 have direct relationship: ariadne q application:DaVinci:v33r8 detector_type:2013 Let’s go around an application, for instance, DaVinci v33r8 2nd LHCb Computing Workshop , 4th-8th November, CERN

  7. Querying Ariadne (from Python) A simple XMLRPC framework is provided to be able to send questionsto Ariadne, and get solutionsfrom it: from xmlrpclib import ServerProxy ariadne = ServerProxy("http://ariadne-lhcb.cern.ch/xmlrpc/", allow_none = True) solution = ariadne.aquery(question, criterion) A common question/solution skeleton provided to help users in communicating with Ariadne, e.g.: question = { 'application': 'Brunel', 'application_version': 'v43r2p2', 'detector_type': '2012', 'reco_type': 'Reco14', 'DDDBTag': None, 'LHCBCONDTag': None, 'DQFLAGSTag': None } solution= { 'application': 'Brunel', 'application_version': 'v43r2p2', 'detector_type': '2012', 'reco_type': 'Reco14', 'DDDBTag': 'dddb-20120831', 'LHCBCONDTag': 'cond-20130404', 'DQFLAGSTag': 'dq-20121016'} Note: as criterion use 'latest_LHCBCOND_DDDB‘ to get this solution 'DDDBConf' already got the AutoTags feature, based on such query mechanism (>=LHCB_v35r6) from Configurables import DDDBConf DDDBConf( DataType = "2012", AutoTags = True ) 2nd LHCb Computing Workshop , 4th-8th November, CERN

  8. Updating AriadneResponsibility of application manager Compatibility data model Consider the use case of compatibility. Imagine, new Brunel vXrY is being released. Relationships, needed to be considered by application manager A A New node: Brunel vXrY ? ? ? ? R R TQ D D TC TC TD TD 1-level deep only! 2nd LHCb Computing Workshop , 4th-8th November, CERN

  9. Updating AriadneResponsibility of CondDB manager Compatibility data model Consider the use case of compatibility. Imagine, new DDDB tag is being released. Relationships, needed to be considered by CondDB manager A A ? R R TQ New DDDB tag D D ? TC TC ? TD TD ? 1-level deep only! 2nd LHCb Computing Workshop , 4th-8th November, CERN

  10. Updating Ariadne (with CL tool)ariadneadd [OPTIONS] IDENTIFIER [INSTRUCTION1 [INSTRUCTION2 […]]] (see ariadne help, or ariadne add -h ) Ariadne Knowledge Graph space Compatible entities identified, and exist in AKG R D TC TD A (zillions of other AKG nodes are not shown) 2nd LHCb Computing Workshop , 4th-8th November, CERN

  11. Updating Ariadne (with CL tool)ariadneadd [OPTIONS] IDENTIFIER [INSTRUCTION1 [INSTRUCTION2 […]]] (see ariadne help, or ariadne add -h ) Ariadne Knowledge Graph space • Compatible entities identified, and exist in AKG • Brunel vXrY node added: • ariadne add application:Brunel:vXrY B A New node: Brunel vXrY R D TC TD A (zillions of other AKG nodes are not shown) 2nd LHCb Computing Workshop , 4th-8th November, CERN

  12. Updating Ariadne (with CL tool)ariadneadd [OPTIONS] IDENTIFIER [INSTRUCTION1 [INSTRUCTION2 […]]] (see ariadne help, or ariadne add -h ) Ariadne Knowledge Graph space • Compatible entities identified, and exist in AKG • Brunel vXrY node added: • ariadne add application:Brunel:vXrY • Brunel vXrY is declared compatible with RecoType “Reco” • ariadne add application:Brunel:vXrY \ • +reco_type:Reco B A New node: Brunel vXrY C R D TC TD A (zillions of other AKG nodes are not shown) 2nd LHCb Computing Workshop , 4th-8th November, CERN

  13. Updating Ariadne (with CL tool)ariadneadd [OPTIONS] IDENTIFIER [INSTRUCTION1 [INSTRUCTION2 […]]] (see ariadne help, or ariadne add -h ) Ariadne Knowledge Graph space • Compatible entities identified, and exist in AKG • Brunel vXrY node added: • ariadne add application:Brunel:vXrY • Brunel vXrY is declared compatible with RecoType “Reco” • ariadne add application:Brunel:vXrY \ • +reco_type:Reco • Brunel vXrY is declared compatible with the rest of entities: • ariadne add application:Brunel:vXrY \ • +detector_type:2013 +tag:DDDB:dddb-ZZZ \ • +tag:LHCBCOND:cond-WWW B A New node: Brunel vXrY C D D D R D TC TD A (zillions of other AKG nodes are not shown) 2nd LHCb Computing Workshop , 4th-8th November, CERN

  14. Updating Ariadne (with CL tool)ariadneadd [OPTIONS] IDENTIFIER [INSTRUCTION1 [INSTRUCTION2 […]]] (see ariadne help, or ariadne add -h ) Ariadne Knowledge Graph space • Compatible entities identified, and exist in AKG • Brunel vXrY node added: • ariadne add application:Brunel:vXrY • Brunel vXrY is declared compatible with RecoType “Reco” • ariadne add application:Brunel:vXrY \ • +reco_type:Reco • Brunel vXrY is declared compatible with the rest of entities: • ariadne add application:Brunel:vXrY \ • +detector_type:2013 +tag:DDDB:dddb-ZZZ \ • +tag:LHCBCOND:cond-WWW B A New node: Brunel vXrY C D D D R D TC TD A (zillions of other AKG nodes are not shown) Or, equivalently, in a single command: ariadne add application:Brunel:vXrY+reco_type:Reco+detector_type:2013 \ +tag:DDDB:dddb-ZZZ+tag:LHCBCOND:cond-WWW 2nd LHCb Computing Workshop , 4th-8th November, CERN

  15. Updating Ariadne (with CL tool)ariadneadd [OPTIONS] IDENTIFIER [INSTRUCTION1 [INSTRUCTION2 […]]] (see ariadne help, or ariadne add -h ) Ariadne Knowledge Graph space • Compatible entities identified, and exist in AKG • Brunel vXrY node added: • ariadne add application:Brunel:vXrY • Brunel vXrY is declared compatible with RecoType “Reco” • ariadne add application:Brunel:vXrY \ • +reco_type:Reco • Brunel vXrY is declared compatible with the rest of entities: • ariadne add application:Brunel:vXrY \ • +detector_type:2013 +tag:DDDB:dddb-ZZZ \ • +tag:LHCBCOND:cond-WWW B A New node: Brunel vXrY E D D D R D TC TD A (zillions of other AKG nodes are not shown) • Next day we figure out Brunel vXrY is NOT really compatible with RecoType “Reco”: • ariadne add application:Brunel:vXrY -reco_type:Reco 2nd LHCb Computing Workshop , 4th-8th November, CERN

  16. Updating Ariadne (with CL tool)ariadneadd [OPTIONS] IDENTIFIER [INSTRUCTION1 [INSTRUCTION2 […]]] (see ariadne help, or ariadne add -h ) Ariadne Knowledge Graph space If new Brunel vXrZ, instead, occurs to be entirely compatible with all entities the previous Brunel vXrY is compatible with, then: ariadne add Brunel:vXrZ =Brunel:vXrY A vXrY A vXrZ (zillions of other AKG nodes are not shown) 2nd LHCb Computing Workshop , 4th-8th November, CERN

  17. Visual Mining in Ariadne (with Neo4j WI)https://ariadne-lhcb.cern.ch/webadmin/ 2nd LHCb Computing Workshop , 4th-8th November, CERN

  18. Updating Ariadne (with Neo4j WI)https://ariadne-lhcb.cern.ch/webadmin/ 2nd LHCb Computing Workshop , 4th-8th November, CERN

More Related