1 / 15

Compiling and Executing Applications on BlueGene

Compiling and Executing Applications on BlueGene. Subba R Bodda and Ravi K Komanduri IBM India Systems and Technology Laboratory . Agenda. BlueGene setup Compiling applications on BlueGene Executing applications on BlueGene. Redbooks.

zarifa
Download Presentation

Compiling and Executing Applications on BlueGene

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. Compiling and Executing Applications on BlueGene Subba R Bodda and Ravi K Komanduri IBM India Systems and Technology Laboratory

  2. Agenda • BlueGene setup • Compiling applications on BlueGene • Executing applications on BlueGene

  3. Redbooks Link: http://www.redbooks.ibm.com/cgi-bin/searchsite.cgi?query=blue+AND+gene • Blue Gene System Administration, SG24-7417-00 • IBM System Blue Gene Solution: System Administration, SG24-7178-04 • IBM System Blue Gene Solution: Configuring and Maintaining Your Environment, SG24-7352-00 • Blue Gene/L: Hardware Overview and Planning, SG24-6796-02 • IBM System Blue Gene Solution: Hardware Installation and Serviceability, SG24-6743-00 • Blue Gene Safety Considerations, REDP-4257-00 • Blue Gene/L: Safety Considerations, REDP-3983-01 • Blue Gene/L: Performance Analysis Tools, SG24-7278-00 • IBM System Blue Gene Solution: Application Development, SG24-7179-04 • Unfolding the IBM eServer Blue Gene Solution, SG24-6686-00 • GPFS Multicluster with the IBM System Blue Gene Solution and eHPS Clusters, REDP-4168-00 • IBM System Blue Gene Solution Problem Determination Guide, SG24-7211-00 • Evolution of the IBM System Blue Gene Solution, REDP-4247-00

  4. BlueGene setup • bglfen01(10.16.28.30),bglfen02 is the front end node • bglsn is the service node • Login to bglfen and compile your code • Use mpirun or LL to submit your jobs • bglsn has the service daemons which would spawn your jobs on actual BG nodes

  5. IBM Compilers for Blue Gene • IBM compilers (Fortran, C, C++) for Blue Gene are located on the frontend node in /opt/ibmcmp • Fortran: /opt/ibmcmp/xlf/bg/10.1/bin/blrts_xlf /opt/ibmcmp/xlf/bg/10.1/bin/blrts_xlf90 /opt/ibmcmp/xlf/bg/10.1/bin/blrts_xlf95 • C: /opt/ibmcmp/vac/bg/8.0/bin/blrts_xlc • C++: /opt/ibmcmp/vacpp/bg/8.0/bin/blrts_xlC Note : Users can add the respective directories to their PATH.

  6. GNU Compilers for Blue Gene • GNU compilers (Fortran, C, C++) for Blue Gene are located in /opt/blrts-gnu/ • Fortran: /opt/blrts-gnu/bin/ powerpc-bgl-blrts-gnu-g77 • C: /opt/blrts-gnu/bin/ powerpc-bgl-blrts-gnu-gcc • C++: /opt/blrts-gnu/bin/ powerpc-bgl-blrts-gnu-g++

  7. MPI on Blue Gene • MPI implementation on Blue Gene is based on MPICH-2 from Argonne National Laboratory. • Include files mpi.h and mpif.h are at the location: -I/bgl/BlueLight/ppcfloor/bglsys/include • Libraries to link for MPI are: -L/bgl/BlueLight/ppcfloorbglsys/lib -lmpich.rts -lmsglayer.rts -lrts.rts -ldevices.rts Note : The linking order for the MPI libraries should be as mentioned above.

  8. Compiler Optimizations

  9. Compiler Optimizations

  10. IBM ESSL libraries • The IBM ESSL (Engineering Scientific Subroutine Library) is located in /opt/ibmmath/essl/4.2/

  11. Executing jobs using mpirun • cd /gpfs/fcHome/scratch/userid mpirun –np 32 –partition R000 -cwd /gpfs/fcHome/scratch/ -exe /gpfs/fcHome/scratch/hello_c • Common options: -args : List of arguments to the executables in double quotes -env : List of environment variables in double quotes. “VARIABLE=value” -mode : CO or VN (coprocessor/virtual-node) -only_test_protocol : This options is used only to test communication protocol • coprocessor mode : one MPI process per node, 1 GB limit per process. Default mode • virtual-node mode : two MPI processes per node, 512 MB limit per process; L3 cache, memory, networks, are shared • For more details type the following at a command prompt : mpirun -h Note : A user can have only one job per Blue Gene partition

  12. Executing jobs using submit_job • Login to bglsn • cd /bgl/BlueLight/ppcfloor/bglsys/bin • ./mmcs_db_console • mmcs$ allocate M03B_J214_64 • mmcs$ submitjob M03B_J214_64 /bgl/hello.rts /bgl/ • OK • jobId=25 • mmcs$ waitjob M03B_J214_64 25 • OK • mmcs$ free M03B_J214_64 • OK • mmcs$ quit

  13. Environment Variables • Environment variables can be specified using –env “XYZ=123 ABC=456” • Some of the environment variables are • BGLMPI_EAGER • BGL_APP_L1_WRITE_THROUGH • BGL_APP_L1_SWOA • BGLMPI_COPRO_SENDS • BGLMPI_INTERRUPT

  14. Redbooks Link: http://www.redbooks.ibm.com/cgi-bin/searchsite.cgi?query=blue+AND+gene • Blue Gene System Administration, SG24-7417-00 • IBM System Blue Gene Solution: System Administration, SG24-7178-04 • IBM System Blue Gene Solution: Configuring and Maintaining Your Environment, SG24-7352-00 • Blue Gene/L: Hardware Overview and Planning, SG24-6796-02 • IBM System Blue Gene Solution: Hardware Installation and Serviceability, SG24-6743-00 • Blue Gene Safety Considerations, REDP-4257-00 • Blue Gene/L: Safety Considerations, REDP-3983-01 • Blue Gene/L: Performance Analysis Tools, SG24-7278-00 • IBM System Blue Gene Solution: Application Development, SG24-7179-04 • Unfolding the IBM eServer Blue Gene Solution, SG24-6686-00 • GPFS Multicluster with the IBM System Blue Gene Solution and eHPS Clusters, REDP-4168-00 • IBM System Blue Gene Solution Problem Determination Guide, SG24-7211-00 • Evolution of the IBM System Blue Gene Solution, REDP-4247-00

  15. Summary • Compilation • IBM XLF/XLC and GNU Compilers • ESSL Libraries • submit_job • mpirun • LL • Environment Variables

More Related