1 / 28

Array Operation Synthesis to Optimize Data Parallel Programs

Array Operation Synthesis to Optimize Data Parallel Programs. Department of Computer Science, National Tsing-Hua University Student:Gwan-Hwan Hwang Advisor: Dr. Jenq Kuen Lee. Array Operation Synthesis to Optimize Data Parallel Programs. 國立清華大學 資訊工程系 Student: 黃冠寰 Advisor: 李政崑博士.

linore
Download Presentation

Array Operation Synthesis to Optimize Data Parallel Programs

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript


  1. Array Operation Synthesis to Optimize Data Parallel Programs Department of Computer Science, National Tsing-Hua University Student:Gwan-Hwan Hwang Advisor: Dr. Jenq Kuen Lee

  2. Array Operation Synthesis to Optimize Data Parallel Programs 國立清華大學 資訊工程系 Student:黃冠寰 Advisor: 李政崑博士

  3. Array Operation Synthesis on Distributed-memory Machines 國立清華大學 資訊工程學系 黃冠寰, Phd.

  4. Research Interests • Key Issues • •Array Operation Synthesis for Intrinsic Array • Functions (JPDC, ACM PPoPP’95, ICPP’96) • Compiler Optimization for • Parallel Computations on • Distributed & Shared Memory • Machines • •Automatic Alignment for Data Parallel • Languages (LCPC’97) • •Communication Code for Block-Cyclic • Distribution of HPF(IPPS’98) • Concurrent Testing • •Reachability Testing of Concurrent Program • (IJSEKE’95, APSEC’93) • Parallel Object Program Model & • Heterogeneous Computing • •Java-Based Network Computing Environment • •Transparent Parallel Computing Environment • (Ongoing)

  5. Outline of Presentation • Fortran 90 Intrinsic Array Operations • Array Operation Synthesis(AOS) • SYNTOOL • Apply AOS to Shared-Memory Machines • Apply AOS to Distributed-Memory Machines • Conclusion and Future Work

  6. Outline of Presentation • Fortran 90 Intrinsic Array Operations • Array Operation Synthesis(AOS) • SYNTOOL • Apply AOS to Shared-Memory Machines • Apply AOS to Distributed-Memory Machines • Integrate AOS with Automatic Data Alignment • Conclusion and Future Work

  7. Intrinsic Array Operations • Provided by Modern Program Languages. • E.g. Fortran 90, High Performance Fortran(HPF), HPF2, Fortran 97, APL, MATLAB, MATHEMATICA, NESL, C* • Engineering and Scientific Applications • Facilitate a Compilation Analysis for Optimization • Support Parallel Execution and Portability

  8. Intrinsic Array Operations(Cont’d) CSHIFT, TRANSPOSE, MERGE, EOSHIFT, RESHAPE SPREAD, Section Move, Where Constructs, Reductions. • Array Operations Provided by Fortran 90, HPF. • Examples: B=CSHIFT(A,1,1) C=TRANSPOSE(B)

  9. Consecutive Array Expressions • Array Expression • Consecutive Array Operations C=EOSHIFT(MERGE(RESHAPE(S,/N,N/),A+B,T),1,0,1) FXP=CSHIFT(F1,1,+1) FXM=CSHIFT(F1,1,-1) FXP=CSHIFT(F1,2,+1) FYM=CSHIFT(F1,2,-1) FDERIV=ZXP*(FXP-F1)+ZXM*(FXM-F1)+ ZYP*(FYP-F1)+ZYM*(FYM-F1)

  10. Classification of Array Operations • Model Array Operations by Data Access Functions(DAF) Type 4 Type 2 Type 1 Type 3

  11. Data Access Functions • Represent Array Operations by Mathematical Functions • Model Array Operations by Data Access Functions(DAF) • Single-Source, Multiple-Source • Single-Clause, multiple-Clause

  12. Type 1: Single-source Single-clause Data Access Function • One Source Array • One Data Access Pattern B=TRANSPOSE(A) Data Access Function is B(I,J)=A(J,I)

  13. Single-source Single-clause Data Access Function • One Source Array • One Data Access Pattern B=TRANSPOSE(A) Data Access Function is B(I,J)=A(J,I)

  14. Type 2: Multiple-source Single-clause Data Access Function • Multiple Source Arrays • One Data Access Pattern R=MERGE(T,F,M) Array T Array F Array M Array R Data Access Function is where

  15. Multiple-source Single-clause Data Access Function • Multiple Source Arrays • One Data Access Pattern R=MERGE(T,F,M) Array T Array F Array M Array R Data Access Function is where

  16. Type 3: Single-source Multiple-clause Data Access Function • Single Source Array • Multiple Data Access Patterns B=CSHIFT(A,1,1) Array A Array B Data Access Function is : a segmentation descriptor

  17. Single-source Multiple-clause Data Access Function • Single Source Array • Multiple Data Access Patterns B=CSHIFT(A,1,1) Array A Array B Data Access Function is : a segmentation descriptor

  18. Type 4: Multiple-source Multiple-clause Data Access Function • Multiple Source Arrays • Multiple Data Access Patterns • No array operation of Fortran 90 belongs to type 4 • Synthesis of multiple array operations may derive a type 4 data access function.

  19. Multiple-source Multiple-clause Data Access Function • Multiple Source Arrays • Multiple Data Access Patterns • No array operation of Fortran 90 belongs to this type • Synthesis of multiple array operations may derive a multiple-source multiple-clause data access function

  20. Straightforward Compilation B=CSHIFT((TRANSPOSE(EOSHIFT(A,1,0,1),1,1) • Translate each operation into a parallel loop FORALL (I=1:N:1; J=1:N:1) IF (1<=I<=N-1) and (1<=J<=N) THEN T1(I,J)=A(I+1,J) ELSE T1(I,J)=0 ENDFORALL EOSHIFT FORALL (I=1:N:1; J=1:N:1) T2(I,J)=T1(J,I) ENDFORALL TRANSPOSE FORALL (I=1:N:1; J=1:N:1) IF (1<=I<=N-1) and (1<=J<=N) THEN B(I,J)=T2(I+1,J) ELSE B(I,J)=T2(I-N,J) ENDFORALL CSHIFT

  21. Array Operation Synthesis B=CSHIFT((TRANSPOSE(EOSHIFT(A,1,0,1),1,1) • Construct the Parse Tree of Array Expression • Represent Array Operations by Mathematical Functions (DAF) EOSHIFT TRANSPOSE CSHIFT

  22. Array Operation Synthesis (Cont’d) Synthesis of two functions CSHIFT TRANSPOSE COSHIFT+ TRANSPOSE EOSHIFT

  23. Synthesis of two Data Access Functions • Substitution (Term Rewriting like method) • Having two Data Access Patterns: • The Synthesized Data Access Pattern is: where where

  24. Synthesis of two DAFs (Cont’d) • For example, • By the substitution rule 

  25. Synthesis of two DAFs (Cont’d) • For example,

  26. Code Generation for Synthesized Data Access Function Code Generation FORALL (I=1:N:1; J=1:N:1) IF (/I,J/,/1:N-1,1:N/) (/J,I+1/,/1:N-1,1:N/) THEN B(I,J)=A(J+1,I+1) IF (/I,J/,/1:N-1,1:N/) (/J,I+1/,/N:N ,1:N/) THEN B(I,J)=0 IF (/I,J/,/N:N ,1:N/) (/J,I+1/,/1:N-1,1:N/) THEN B(I,J)=A(J+1,I-N+1) IF (/I,J/,/N:N ,1:N/) (/J,I+1/,/N:N ,1:N/) THEN B(I,J)=0 ENDFORALL

  27. Code Generation for Synthesized Data Access Function After Optimization 1 N-1 N 1 N-1 N

  28. Optimization • Simplifying the ranges at compilation time instead of runtime • Optimization process: • Normalize: • Intersection for each dimension:

More Related