1 / 16

RGMA Deployment

RGMA Deployment. Laurence Field / RAL <l.field@rl.ac.uk> Steve Fisher / RAL <s.m.fisher@rl.ac.uk> . R-GMA. Use the GMA from GGF A relational implementation Powerful data model and query language Applied to both information and monitoring Creates impression that you have one RDBMS per VO.

lilah-horton
Download Presentation

RGMA Deployment

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. RGMA Deployment Laurence Field / RAL <l.field@rl.ac.uk> Steve Fisher / RAL <s.m.fisher@rl.ac.uk>

  2. R-GMA • Use the GMA from GGF • A relational implementation • Powerful data model and query language • Applied to both information and monitoring • Creates impression that you have one RDBMS per VO Producer Store location Registry execute or stream Lookup location Consumer R-GMA Deployment

  3. Relational Data Model • Not a general distributed RDBMS system, but a way to use the relational model in a distributed environment where global consistency is not important. • Producers announce: SQL “CREATE TABLE” publish: SQL “INSERT” • Consumers collect: SQL “SELECT” • Some producers, the Registry and Schema make use of RDBMS as appropriate – but what is central is the relational model. R-GMA Deployment

  4. Producer  Consumer • Consumer can issue one-off queries • Similar to normal database query • Consumer can also start a continuous query • Requests all data published which matches the query • Can be seen as an alert mechanism R-GMA Deployment

  5. 3 Kinds of Query StreamProducer Tuple Tuple Tuple Tuple Tuple Tuple Tuple Tuple Tuple Tuple Tuple HistoryProducer Tuple Tuple Tuple Tuple Tuple Tuple Tuple Tuple Tuple Tuple Tuple LatestProducer Tuple Tuple Tuple Tuple Tuple R-GMA Deployment

  6. Producers • DataBaseProducer – Supports History Queries • Information not lost • Supports joins • Clean up strategy • StreamProducer – Supports Continuous Queries • In memory data structure • Can define minimum retention period • ResilientStreamProducer – Supports Continuous Queries • Like the StreamProducer but won’t lose data if system crashes • So slightly slower • LatestProducer – Supports Latest Queries • Just holds the latest information for any “primaryish” key • Supports joins • CanonicalProducer – Supports anything • Offers anything as relations R-GMA Deployment

  7. Archiver (Re-publisher) • It is a combined Consumer-Producer • You just have to tell it what to collect and it does so on your behalf • Re-publishes to any kind of “Insertable” (i.e. not to the CanonicalProducer) • Can support joins if archiving to a DataBaseProducer or a LatestProducer R-GMA Deployment

  8. Functionality - mediator • Queries posed against a virtual data base • The Mediator must: • find the right Producers • combine information from them • Hidden component – but vital to R-GMA Now supports Joins! R-GMA Deployment

  9. Topologies • Normally publish via a StreamProducer • Archivers instantiated with a Producer and a Predicate • Must avoid cycles in the graph SP SP A A A SP LP SP SP A A SP HP SP R-GMA Deployment

  10. edg-rgma • show tables • describe Service • show producers of Service • latest select * from Service • old continuous select * from Service Now produces decent error messages as it uses new C API internally. R-GMA Deployment

  11. GIN and GOUT (Gadget IN and Gadget OUT) Archiver (LatestProducer) LDAP InfoProvider GLUE Schema Consumer (CE) ConsumerAPI Latest Producer Consumer (SE) GIN RDBMS Consumer (SiteInfo) R-GMA Stream Producer GOUT Stream Producer LDAP Server R-GMA Consumers GIN LDAP InfoProvider R-GMA Deployment

  12. WP3 testbed • Asked to create a WP3 testbed. • Integration testing before releasing to EDG Dev. • Building a complete testbed (including RB) • Significant effort • Impacts a number of sysadmins • Hope to get a good return • Bristol, Glasgow, Imperial College, NIKHEF, RAL x3 R-GMA Deployment

  13. Performance Testing • Document – based on criteria from Steven Burke • No of sites R-GMA can scale to. • Availability of information • Accuracy of information • Propagation speed • etc. • Analyzing code using OptimizeIt. • Looking at bottlenecks in code. • Memory, CPU usage etc. • Increasing performance of the code. • Reducing huge number of threads in Gout • Optimising and simplifying Gin R-GMA Deployment

  14. Resilience Testing • Looking at the Resilience of R-GMA. • Random Reboots, Power Cuts, network glitches etc. • Developing a resilience testing framework. • Automating testing of different components. • Will test the fail-over mechanisms. • Test registry replication. • Will lead to more robust code. R-GMA Deployment

  15. Current Status • Jobs have been submitted on Dev testbed • Minor bugs appear less frequently • Mostly fixed very quickly. • Can support 50 70 simulated sites R-GMA Deployment

  16. Immediate Future • Bringing it to the standards of the “performance document” and demonstrating it • Security • Increased resilience • Multiple registry instances • Multiple schema instances R-GMA Deployment

More Related