GridSuperscalar A programming model for GRID applications

1. GridSuperscalarA programming model for GRID applications José Mª Cela cela@ac.upc.es cela@ciri.upc.es

2. What we have? What we want? LCFIB Campus Nord 500GFlops 40PB disk space 10TB RAM ... CEPBA Campus Nord AC Campus Castelldefels Conected heterogeneous resources Transparent access to basic resources What is GRID?

3. OPEN CODE • DOCUMENTATION • JAVA • C (Unix) • TCP/IP SERVICES LIBRARIES UTILTIES Basic brick in a GRID • Globus Toolkit 2.x

4. CLIENT UTILITIES LIBRARIES SERVER Data transfer Access to sytem information Secure Access Manage and execute processes Globus Resource Allocation Manager (GRAM) Grid Security Infrastructure (GSI) Monitoring and Discovery Service (MDS) Global Access to Secondary Storage (GASS) What is GLOBUS?

5. Resource GRID-MAPFILE GRID User Certificate GRID User -> Local User Autorization GATEKEEPER Public Key Private Key Resource certificate Temporal Proxy Public Key Private Key Security Autentification

6. GISS (Grid Index Information Service) GISS-ALL GRIS (Grid Resource Information Service) ... Monitoring Systems

7. GRID-MAPFILE GRID User -> Local User Data transfer RECURSO High Speed It has implementation problems GSIFTP - gsiftp:// GSIFTP RESOURCE CERTIFICATION OK! Temporal Proxy GASS GASS - https:// Always available Slow speed Local User Temporal Proxy

8. RSL SCRIPT Resource WITH Load Balancing Control '&(parameter=valorY) (parameter1=valorX) ...' LoadLeveler, NQE GRAM SERVER executable=<@ejecutable> arguments=<argumentos> stdin=<@origen entrada estándar> stdout=<@destino salida estándar> queue=<cola> environment=<variables de entorno> directory=<directorio inicial> ... GRAM CLIENT Resource WITHOUT it GRAM SERVER Grid Job No Grid Job Execute processes

9. Execute processes main() { globus_module_activate( GLOBUS_GRAM_CLIENT_MODULE ); err = globus_gram_client_callback_allow( Callback, NULL, &contact ); if (err != GLOBUS_SUCCESS) ERROR(); desc = "(&executable=/home/ac/cela/Dimemas)(arguments= -o kk.out file.cfg) (scratch_dir=/scratch/ac/cela)(directory=/home/ac/cela/) (file_stage_in=(https://kandake:20352/home/ac/cela/file.cfg $(SCRATCH_DIRECOTRY)/file.cfg) (https://kandake:20352/home/ac/cela/matrix.trf $(SCRATCH_DIRECOTRY)/matrix.trf)) (file_stage_out=($(SCRATCH_DIRECOTRY)/kk.out https://kandake:20352/home/ac/cela/kk.out) (environment=(DIMEMAS_HOME/usr/local/cepba-tools)(TMPDIR $(SCRATCH_DIR)))"; err = globus_gram_client_job_request( contact, desc, GLOBUS_GRAM_PROTOCOL_JOB_STATE_DONE, callback_contact, &job_contact ); if (err != GLOBUS_SUCCESS) ERROR(); err = globus_gram_client_job_status( job_contact, &job_status, &res ); if (err != GLOBUS_SUCCESS) ERROR(); while (1) { globus_poll(); globus_poll_blocking(); } globus_gram_client_job_contact_free( job_contact ); globus_module_deactivate( GLOBUS_GRAM_CLIENT_MODULE ); }

10. GridSuperscalar • A programming paradigm which allows: • Reduce the complexity of developing Grid Applications to the minimum • Automatic task parallelization over a GRID environment • Basic idea: superscalar processors • Sequential control flow • Well defined object name space, I/0 arguments to operation • Automatic construction of precedence DAG • Renaming • Forwarding • DAG scheduling, locality management • Prediction, Speculation

11. GridSuperscalar basis • Code: • Sequential application in C/C++/Fortran90 with calls the GridSuperscalar run-time (Execute call) • Run-time performs: • Task identification (based on Execute primitive) • Data dependency analysis: files are the objects • Data dependence graph creation • Task scheduling based on the graph • File renaming to increase graph concurrency • File forwarding

12. T10 T20 T40 T30 Grid T50 T11 T21 T41 T31 T51 T12 … GridSuperscalar behavior overview Application code for (i=0; i<N; i++ { Execute(T1, “file1.txt”, “file2.txt”); Execute(T2, “file4.txt”, “file5.txt”); Execute(T3, “file2.txt”, “file5.txt”, “file6.txt”); Execute(T4, “file7.txt”, “file8.txt”); Execute(T5, “file6.txt”, “file8.txt”, “file9.txt”); }

13. GridSuperscalar: user interface • Actions to do when developing an application • Task definition: identify those subroutines/programs to be executed in the Grid • Tasks’ interface definition: input/output files and input/output generic scalars • Write the sequential program using calls to the GridSuperscalar primitives (Execute) Instruction set definition

14. GridSuperscalar: code example • Simple optimization search example: • perform Neval simulations • recalculate range of parameters • end when goal is reached • Tasks: FILTER, DIMEMAS, EXTRACT • Parameters (current syntax): <OP_NAME> <n_in_files> <n_in_generic> <n_out_files> <n_out_generic> FILTER 1 2 1 0 DIMEMAS 2 0 1 0 EXTRACT 1 0 1 0 • Sequential code

15. GridSuperscalar: code example Range = initial_range(); while (!goal_reached() && (j<MAX_ITERS)) { for (i= 0; i< Neval; i++) { L[i] = gen_rand_L_within_current_range(range); BW[i] = gen_rand_BW_within_current_range(range); Execute( FILTER, “bh.cfg”, L[i], BW[i], “bh_tmp.cfg”); Execute( DIMEM, “bh_tmp.cfg”,“trace.trf”, “dim_out.txt”); Execute( EXTRACT, “dim_out.txt”, “final_result.txt”, “final_result.txt”); } GS_Barrier(); generate_new_range(“final_result.txt”, &range); j++; }

16. GridSuperscalar: code example • Worker: switch(atoi(argv[2])) { case FILTER: res = filter(argc, argv); break; case DIMEM: res = dimemas_funct(argc, argv); break; case EXTRACT: res = extract(argc, argv); break; default: printf("Wrong operation code\n"); break; }

17. FILTER FILTER DIMEMAS DIMEMAS EXTRACT EXTRACT GridSuperscalar run-time: task graph generation Range = initial_range(); while (!goal_reached() && (j<MAX_ITERS)){ for (i=0; i< Neval; i++){ L[i] = gen_rand_L_within_current_range(range); BW[i] = gen_rand_BW_within_current_range(range); Execute( FILTER, “bh.cfg”, L[i], BW[i], “bh_tmp.cfg” ); Execute( DIMEMAS, “bh_tmp.cfg”,“trace.trf”, “dim_out.txt” ); Execute( EXTRACT, “dim_out.txt”, “final_result.txt” ); } GS_Barrier(); generate_new_range(“final_result.txt”, &range); j++; } Neval FILTER … DIMEMAS EXTRACT BARRIER …

18. GridSuperscalar: task scheduling FILTER FILTER FILTER … DIMEMAS DIMEMAS DIMEMAS EXTRACT EXTRACT EXTRACT BARRIER … CIRI Grid

19. GridSuperscalar: task scheduling FILTER FILTER FILTER … DIMEMAS DIMEMAS DIMEMAS EXTRACT EXTRACT EXTRACT BARRIER … CIRI Grid • Additional function: • GS_barrier

20. GridSuperscalar: renaming T1_1 T2_1 TN_1 “f1_2” “f1_1” “f1” “f1” “f1” … T1_2 T2_2 TN_2 T1_3 T2_3 TN_3 • Additional functions: • GS_open, GS_close

21. GridSuperscalar: file forwarding T1 T1 f1 (by socket) f1 T2 T2 • Prototype implemented with Dyninst • Allows to execute different tasks on different hardware resources • Initial tests • High overhead due to worker behavior mutation • Future tests with new Dyninst version 4.0

22. Grid superscalar: current Globus implementation • Previous prototype over Condor and MW • Current prototype over Globus 2.x, using the API • File transfer, security, … provided by Globus • Task submission • globus_gram_client_job_request • Asynchronous end of task synchronization • Asynchronous state-change callbacks mechanism provided by Globus • globus_gram_client_callback_allow • callback_func function, provided by the programmer

23. GRID Performance Measurement • NAS GRID benchmarks • Designed to provide an objective measure of GRID systems • At the present it is only a paper and pencil description • The basic executables are the NAS Parallel Benchmarks • BT, SP, LU => Solvers • MG => Post-processor • FT => Data visualization • MF => Interpolation filter • Only the data flow graphs are defined • 4 data flow graphs proposed (ED, HC, VP, MB)

24. Launch SP SP SP SP SP SP SP SP SP Report GRID Performance Measurement • Embarrassingly Distributed (ED) • Parameter studies

25. GRID Performance Measurement • Helical Chain (HC) • A set of flow calculations Launch BT MF SP MF LU MF BT MF SP MF LU MF Report BT MF SP MF LU

26. GRID Performance Measurement • Visualization Pipe (VP) • Solver + Post-processor + Visualization module Launch BT MF MG MF FT BT MF MG MF FT Report BT MF MG MF FT

27. GRID Performance Measurement • Mixed Bag (MB) • VP with asymmetry Launch LU LU LU MF MF MF MG MG MG MF MF MF FT FT FT Report

28. GRID Performance Measurement • Original script eval "(./bin/bt.$CLASS <<-EOF ) | $THROWIT /dev/null $ASCII $NAME $CLASS $ITER $WIDTH $DEPTH $PID $VERBOSE EOF"

29. GRID Performance Measurement • Present code in our programing model sprintf( genIn, "%s %s %d %d %d %d %d %d", NAME, CLASE, ascii, iter, width, depth, pid, verbose); Execute( BT, "HC_BT_IN", genIn, "HC_BT_OUT" );