1 / 10

GridX Globus has come of age

GridX Globus has come of age. Francis Tang, Sebastian Ho with Arun Krishnan, Ryan Lim. Overview. Grid computing with Globus Grid proxy Staging Globusrun Grid computing with GridX Example 1: warming up Example 2: pre-built binary Example 3: development cluster Other features. Globus.

aerona
Download Presentation

GridX Globus has come of age

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. GridXGlobus has come of age Francis Tang, Sebastian Ho with Arun Krishnan, Ryan Lim The new baby

  2. Overview • Grid computing with Globus • Grid proxy • Staging • Globusrun • Grid computing with GridX • Example 1: warming up • Example 2: pre-built binary • Example 3: development cluster • Other features The new baby

  3. Globus • To run a job using globus: • Initialise proxy grid_proxy_init • Choose a grid node (e.g. turing.gridx.bii OR church.gridx.bii) • Stage files to grid node • Start remote GASS server (globusrun … globus-gass-server?) • Globus-url-copy • Determine local scheduler on grid node, then submit job globusrun –b –r turing.gridx.bii:/jobmanager-grd ‘&(directory=…..)’ OR: globusrun –b –r church.gridx.bii:/jobmanager-pbs • Poll status of job globus-job-status https://....... • If job has finished, then stage files back • Globus-url-copy The new baby

  4. GridX • To submit a job using GridX • Point browser to http://babbage.gridx.bii • Login • Provide job information on job submit page • Executable name • Files location • Click submit • Wait for results through email, or check status using GridX web interface The new baby

  5. What GridX does • Given job details, GridX will • choose a suitable grid node • stage the files to that node • submit job to remote scheduler (e.g. PBS, SGE) • When job finishes, GridX will • stage files back to the GridX server • optionally email results to you • Once results are on the GridX server • user can view them using the web interface • or, download them The new baby

  6. Point browser to http://babbage.gridx.bii Submit Click “Submit Job – advanced” Type: Job Name: “warm up” Executable: “/bin/hostname” Processors Required: “1 shell” Click “Submit” GridX shows summary Check status On summary page, click “Job Submission History” Your most recently submitted jobs should appear on the top Click “refresh” to get latest status information (but not too quickly!) When status is done, click “view” in Result column Example 1: warming up The new baby

  7. Scenario: you want to run a pre-built binary (called cpi) which is stored on your local harddisk. If you want to try this, download cpi from http://www.bii.a-star.edu.sg/~francis/GridX/files/cpi Submit Click “Submit Job – advanced” Type: Job Name: “pre-built binary” Executable: “cpi” Processors Required: “4 mpi” Click “Data File 2 – upload” and upload cpi Click “Submit” GridX shows summary Check status as before Example 2: pre-built binaries The new baby

  8. Scenario: You are developing code on a development cluster You have been testing the code and would like to run it on some real data, BUT The development cluster is heavily loaded The development cluster does not have enough resources (e.g. cpu count, RAM) You want to run your program on the Grid GridX can help you Submit the job, BUT tell GridX that the files are on the development cluster GridX will run the job on the suitable cluster GridX will perform file staging over the network if necessary Step-by-step example… Development cluster is goedel.gridx.bii Example 3: development cluster The new baby

  9. Ssh into goedel.gridx.bii Create directory: mkdir gridx-ex3 Edit source / compile binaries: cd gridx-ex3 cp /usr/local/mpich-1.2.5-/examples/cpi .c vi cpi.c gcc –o mycpi cpi.c “Export” your files for staging mkdir temp cp mycpi temp cd temp ~gridx/bin/gridx-publish.sh ex3 The export script will display the directory for staging Submit: Click “Submit Job – advanced” Type: Job Name: “example 3” Executable: “mycpi” Processors Required: “6 mpi” 3rd party transfer - Hostname: “goedel.gridx.bii” 3rd party transfer – Directory: (copy paste from other window) Click “Submit” GridX shows summary Check status as before Example 3 (continued) The new baby

  10. Other features • If your program writes output to many files, GridX can optionally stage them back • To enable staging back, select “Retrieve all files” option on submit page • When the job is done, you can view your files through the results page • If you prefer, GridX can mail the results back to you • Beware: GridX stages all files back, including the binary, input data and temporary files generated by your program The new baby

More Related