jGMA: A Reference Implementation of the Grid Monitoring Architecture

jGMA: A Reference Implementation of the Grid Monitoring Architecture Mat Grove Distributed Systems Group University of Portsmouth http://dsg.port.ac.uk/~mjeg/

Outline • Introduction and Motivation. • The Grid Monitoring Architecture (GMA). • The jGMA Implementation. • Summary and future work. • jGMA Demo. http://dsg.port.ac.uk/projects/jGMA/

Introduction • GridRM is: • A generic resource monitoring system – basically a framework for gathering data from heterogeneous data sources, • A scalable means of monitoring resources distributed over the Grid, • Written in Java, using commons standards: • Such as SQL, JDBC, Java Servlets, GLUE. • GridRM is made up of local and global layers; the latter basically is the means of transporting GridRM data between grid sites. • GridRM needed a “standards-based” transport layer to bind together these its sites at the global layer. • A GMA-based transport appeared to be the best candidate. And and the the a http://dsg.port.ac.uk/projects/jGMA/

GridRM (gridrm.org) http://dsg.port.ac.uk/projects/jGMA/

The Grid Monitoring Architecture • The GGF’s GMA document is their suggested “informational” means for monitoring within the Grid: • Note Informational documents are used to inform the community about a useful idea or set of ideas. • The GMA document (2001) recommends: • A publish, subscribe and bind architecture, • An implementation that is: • Secure and scalable, • Has a low-latency and capable of high data rates, • Imposes minimal impact on the infrastructure. • GMA compliancy means implementing approximately 20 features. And and the the a http://dsg.port.ac.uk/projects/jGMA/

GMA Components Registry Publish/Search Publish/Search Bind Producer Consumer Events http://dsg.port.ac.uk/projects/jGMA/

Some GMA Implementations • In the first instance we wanted to take up an off-the-shelf GMA package, several out there. • Embedded (part of a larger s/w package): • e.g. AutoPilot and Globus MDS: • MDS can be extracted into a standalone infrastructure with a little effort. • Standalone: • pyGMA (Berkeley National Lab), and R-GMA (DataGrid/LCG). • None of these implementations met GridRM’s requirements: • Easy to integrate within GridRM (Java-based), • Simple installation with minimal dependencies, • No need to “fork” parent software tree, • GMA compliance. And and the the a http://dsg.port.ac.uk/projects/jGMA/

jGMA Components • jGMA has the following components: • A registry, • A producer and consumer (shared code-base), • A PC Servlet (provides WAN access between producers and consumers). • jGMA features: • Written in Java, so takes advantage of usual features and libraries, • Small API (<20 calls), • Released as single “jar” of currently 28Kbytes. • Limited dependencies: • Tomcat and Java. http://dsg.port.ac.uk/projects/jGMA/

jGMA Implementation http://dsg.port.ac.uk/projects/jGMA/

Example jGMA Addressing. • LAN address: • WAN address: socket://148.197.150.3:56099/producer/cam http://dsg.port.ac.uk:8080/jGMA/PC?8723623 http://dsg.port.ac.uk/projects/jGMA/

jGMA Optimisations • The jGMA Implementation: • Non-blocking communications: • Local communications use Java sockets: • RMI, • Serialising objects, • Packed binary arrays, • WAN communication use HTTP: • HTTP POST, • Encapsulating binary data. • Performance Tests: • Latency (Over Ethernet 1.5ms, Localhost 0.8ms), • Throughput (~1200 messages per second), • Scalability (tested stability under heavy loads). http://dsg.port.ac.uk/projects/jGMA/

Consumer pseudo code. Consumer(){ Connector=new JgmaConnector(“holly.dsg.port.ac.uk”); Connector.register(“pinger”); } Registered(){ Connector.registryQuery(“select * from producers”); } RegistryEvent(result){ for each result as producer{ Connector.sendMessage(producer, “ping”); } } http://dsg.port.ac.uk/projects/jGMA/

Producer pseudo code. Producer(){ Connector=new JgmaConnector(“holly.dsg.port.ac.uk”); Connector.register(“ponger”); } MessageEvent(message){ Connector.replyMessage(message, “pong”); } http://dsg.port.ac.uk/projects/jGMA/

Summary • Summary: • jGMA is a reference implementation of GMA written in Java, • jGMA is a easy to use, simple, scalable and a robust system, • Our early tests show that jGMA has good performance and scalability. • Some issues need dealing with: • Throttling. http://dsg.port.ac.uk/projects/jGMA/

Future Work • Future developments: • Implement a persistent registry: • File (ASCII text), • R-DBMS (i.e. MySQL), • XML (i.e. Xindice). • Completing the leasing infrastructure, • Virtual Registry, • Monitoring via PC Servlet, • Hooks for debugging, • Being deployed in GridRM, • Look at working with other GMA developers to get an agreement on a standardised wire API. http://dsg.port.ac.uk/projects/jGMA/

Demo http://dsg.port.ac.uk/projects/jGMA/

Download • Available: jGMA-0.3.7-beta • New version for October: • http://dsg.port.ac.uk/projects/jGMA http://dsg.port.ac.uk/projects/jGMA/

Links • Project Web page: • http://dsg.port.ac.uk/projects/jGMA/ • The DSG Web page: • http://dsg.port.ac.uk/ • GridRM: • http://gridrm.org/ • Related links: • GGF: http://www.gridforum.org/ • R-GMA: http://www.r-gma.org/ http://dsg.port.ac.uk/projects/jGMA/

Xcx vxv Xcv xcv Sdf Sd F S Dfs Df Xcx vxv Xcv xcv http://dsg.port.ac.uk/projects/jGMA/

jGMA: A Reference Implementation of the Grid Monitoring Architecture