1 / 39

Instruction-level Parallelism

Instruction-level Parallelism. Compiler Perspectives on Code Movement. dependencies are a property of code, whether or not it is a HW hazard depends on the given pipeline. Compiler must respect ( True ) Data dependencies (RAW) Instruction i produces a result used by instruction j, or

iris-campos
Download Presentation

Instruction-level Parallelism

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. Instruction-level Parallelism

  2. Compiler Perspectives onCode Movement • dependencies are a property of code, whether or not it is a HW hazard depends on the given pipeline. • Compiler must respect (True) Data dependencies (RAW) • Instruction i produces a result used by instruction j, or • Instruction j is data dependent on instruction k, and instruction k is data dependent on instruction i.

  3. Compiler Perspectives onCode Movement • Other kinds of dependence also called name (false) dependence: two instructions use same name but don’t exchange data • Antidependence (WAR dependence) • Instruction j writes a register or memory location that instruction i reads from and instruction i is executed first • Output dependence (WAW dependence) • Instruction i and instruction j write the same register or memory location; ordering between instructions must be preserved.

  4. Control Dependence • Control Dependence • Example if (c1) I1; if (c2) I2; I1 is control dependent on c1 and I2 is control dependent on c2 but not on c1.

  5. A sample loop Loop: LD F0,0(R1) ;F0=array element, R1=X[] MULD F4,F0,F2 ;multiply scalar in F2 SD F4, 0(R1) ;store result ADDI R1,R1,8 ;increment pointer 8B (DW) SEQ R3, R1, R2 ;R2 = &X[1001] BNEZ R3,Loop ;branch R3!=zero NOP ;delayed branch slot Where are the dependencies and stalls? Operation Latency (stalls) FP Mult 6 (5) LD 2 (1) Int ALU 1 (0)

  6. Instruction Scheduling Loop: LD F0,0(R1) MULD F4,F0,F2 SD 0(R1),F4 ADDI R1,R1,8 SEQ R3, R1, R2 BNEZ R3,Loop NOP Number of cycle per iteration?

  7. Instruction Scheduling Loop: LD F0,0(R1) MULD F4,F0,F2 SD 0(R1),F4 ADDI R1,R1,8 SEQ R3, R1, R2 BNEZ R3,Loop NOP Loop: LD F0,0(R1) ADDI R1,R1,8 MULD F4,F0,F2 SEQ R3, R1, R2 BNEZ R3,Loop SD -8(R1),F4 Cycles/iteration?

  8. Loop Unrolling Loop: LD F0,0(R1) ADDI R1,R1,8 MULD F4,F0,F2 SEQ R3, R1, R2 BNEZ R3,Loop SD -8(R1),F4 Can extract more parallelism

  9. Loop Unrolling Loop: LD F0,0(R1) ADDI R1,R1,8 MULD F4,F0,F2 SEQ R3, R1, R2 BNEZ R3,Loop SD -8(R1),F4 Loop: LD F0,0(R1) ADDI R1,R1,8 MULD F4,F0,F2 SEQ R3, R1, R2 BNEZ R3,Loop SD -8(R1),F4 LD F0,0(R1) ADDI R1,R1,8 MULD F4,F0,F2 SEQ R3, R1, R2 BNEZ R3,Loop SD -8(R1),F4 What is the problem here?

  10. Loop Unrolling Loop: LD F0,0(R1) ADDI R1,R1,8 MULD F4,F0,F2 SEQ R3, R1, R2 BNEZ R3,Loop SD -8(R1),F4 Loop: LD F0,0(R1) ADDI R1,R1,8 MULD F4,F0,F2 SEQ R3, R1, R2 BNEZ R3,Loop SD -8(R1),F4 LD F0,0(R1) ADDI R1,R1,8 MULD F4,F0,F2 SEQ R3, R1, R2 BNEZ R3,Loop SD -8(R1),F4 Unnecessary instructions and redundant instructions

  11. Loop Unrolling Loop: LD F0,0(R1) ADDI R1,R1,8 MULD F4,F0,F2 SEQ R3, R1, R2 BNEZ R3,Loop SD -8(R1),F4 Loop: LD F0,0(R1) MULD F4,F0,F2 SD 0(R1),F4 LD F0,8(R1) ADDI R1,R1,16 MULD F4,F0,F2 SEQ R3, R1, R2 BNEZ R3,Loop SD -8(R1),F4 Hint Still problems with scheduling?

  12. Register Renaming Loop: LD F0,0(R1) ADDI R1,R1,8 MULD F4,F0,F2 SEQ R3, R1, R2 BNEZ R3,Loop SD -8(R1),F4 Loop: LD F0,0(R1) MULD F4,F0,F2 SD 0(R1),F4 LD F10,8(R1) ADDI R1,R1,16 MULD F14,F10,F2 SEQ R3, R1, R2 BNEZ R3,Loop SD -8(R1),F14 Let’s schedule now

  13. Register Renaming Loop: LD F0,0(R1) ADDI R1,R1,8 MULD F4,F0,F2 SEQ R3, R1, R2 BNEZ R3,Loop SD -8(R1),F4 Loop: LD F0,0(R1) LD F10,8(R1) MULD F4,F0,F2 MULD F14,F10,F2 ADDI R1,R1,16 SEQ R3, R1, R2 SD 0(R1),F4 BNEZ R3,Loop SD -8(R1),F14 Cycles/iteration?

  14. How easy is it to determine dependences? • Easy to determine for registers (fixed names) • Hard for memory: • Does 100(R4) = 20(R6)? • From different loop iterations, does 20(R6) = 20(R6)? • Another Example: ST R5, R6 LD R4, R3

  15. Memory Disambiguation • Problem: In many cases, it is likely but not certain that two memory instructions reference different addresses • Disambiguation is much harder in languages with pointers • Example: void annoy_compiler1(char *foo, char *bar) { foo[2] = bar[2]; bar[3] = foo[3]; } Memory references are independent unless foo = bar

  16. Disambiguation 2 • Making things worse, some programs have independent memory references some of the time • Example: void annoy_compiler2(int *a, int *b) { int I; for (I = 0; I < 256; I++){ a[I] = b[f(I)]; } } • Conventional compiler needs to assume that any references that could be to the same location are to the same location and serialize them

  17. HW Schemes: Instruction Parallelism • Why in HW at run time? • Works when can’t know dependence until run time • Variable latency • Control dependent data dependence • Can schedule differently every time through the code. • Compiler simpler • Code for one machine runs well on another • Hardware techniques to find/extract ILP • Tomasulo’s Algorithm for Out-of-order Execution

  18. Tomasulo’s Algorithm • Developed for architecture of IBM 360/91 (1967) • 360/91 system’s goal was to significantly improve performance (especially floating-point) without requiring people to change their code • Sound familiar? 16MHz 2MB Mem 50X faster Than SOA

  19. Tomasulo Organization

  20. Tomasulo Algorithm • Consider three input instructions • Common Data Bus broadcasts results to all FUs RS’s (FU’s), registers, etc. responsible for collecting own data off CDB • Load and Store Queues treated as FUs as well

  21. Reservation Station Components Op—Operation to perform in the unit (e.g., + or –) Qj, Qk—Reservation stations producing source registers Vj, Vk—Value of Source operands Rj, Rk—Flags indicating when Vj, Vk are ready Busy—Indicates reservation station is busy Register result status—Indicates which functional unit will write each register, if one exists. Blank when no pending instructions that will write that register.

  22. Three Stages of Tomasulo Algorithm 1. Issue—get instruction from FP Op Queue If reservation station free, the scoreboard issues instr & sends operands (renames registers). 2. Execution—operate on operands (EX) When both operands ready then execute; if not ready, watch CDB for result 3. Write result—finish execution (WB) Write on Common Data Bus to all waiting units; mark reservation station available.

  23. Tomasulo Example ADDD F4, F2, F0 MULD F8, F4, F2 ADDD F6, F8, F6 SUBD F8, F2, F0 ADDD F2, F8, F0 Multiply takes 10 clocks, add/sub take 4

  24. Tomasulo – cycle 0 Instruction Queue ADDD F4, F2, F0 MULD F8, F4, F2 ADDD F6, F8, F6 SUBD F8, F2, F0 ADDD F2, F8, F0 F0 F2 F4 F6 F8 0.0 ADDD F2, F8, F0 2.0 SUBD F8, F2, F0 4.0 ADDD F6, F8, F6 6.0 8.0 MULD F8, F4, F2 ADDD F4, F2, F0 1 2 3 1 2 FP adders FP mult’s

  25. Tomasulo – cycle 1 Instruction Queue ADDD F4, F2, F0 MULD F8, F4, F2 ADDD F6, F8, F6 SUBD F8, F2, F0 ADDD F2, F8, F0 F0 F2 F4 F6 F8 0.0 2.0 ADDD F2, F8, F0 4.0 add1 SUBD F8, F2, F0 6.0 8.0 ADDD F6, F8, F6 MULD F8, F4, F2 1 2 3 ADDD 2.0 0.0 1 2 FP adders FP mult’s

  26. Tomasulo – cycle 2 Op Qj Qk Vj Vk Busy MULD add1 - - 2.0 Y Instruction Queue ADDD F4, F2, F0 MULD F8, F4, F2 ADDD F6, F8, F6 SUBD F8, F2, F0 ADDD F2, F8, F0 F0 F2 F4 F6 F8 0.0 2.0 4.0 add1 ADDD F2, F8, F0 6.0 8.0 mult1 SUBD F8, F2, F0 ADDD F6, F8, F6 1 2 3 ADDD 2.0 0.0 1 2 MULD add1 2.0 FP adders FP mult’s

  27. Tomasulo – cycle 2 Instruction Queue ADDD F4, F2, F0 MULD F8, F4, F2 ADDD F6, F8, F6 SUBD F8, F2, F0 ADDD F2, F8, F0 F0 F2 F4 F6 F8 0.0 2.0 4.0 add1 ADDD F2, F8, F0 6.0 8.0 mult1 SUBD F8, F2, F0 ADDD F6, F8, F6 1 2 3 ADDD 2.0 0.0 1 2 MULD add1 2.0 FP adders FP mult’s

  28. Tomasulo – cycle 3 Instruction Queue ADDD F4, F2, F0 MULD F8, F4, F2 ADDD F6, F8, F6 SUBD F8, F2, F0 ADDD F2, F8, F0 F0 F2 F4 F6 F8 0.0 2.0 4.0 add1 6.0 add2 8.0 mult1 ADDD F2, F8, F0 SUBD F8, F2, F0 1 2 3 ADDD 2.0 0.0 1 2 ADDD mult1 6.0 MULD add1 2.0 FP adders FP mult’s

  29. Tomasulo – cycle 4 Instruction Queue ADDD F4, F2, F0 MULD F8, F4, F2 ADDD F6, F8, F6 SUBD F8, F2, F0 ADDD F2, F8, F0 F0 F2 F4 F6 F8 0.0 2.0 4.0 add1 6.0 add2 8.0 add3 ADDD F2, F8, F0 1 2 3 ADDD 2.0 0.0 1 2 ADDD mult1 6.0 MULD add1 2.0 SUBD 2.0 0.0 FP adders FP mult’s

  30. Tomasulo – cycle 5 Instruction Queue ADDD F4, F2, F0 MULD F8, F4, F2 ADDD F6, F8, F6 SUBD F8, F2, F0 ADDD F2, F8, F0 F0 F2 F4 F6 F8 0.0 2.0 2.0 - 6.0 add2 8.0 add3 ADDD F2, F8, F0 1 2 3 ADDD 2.0 0.0 1 2 ADDD mult1 6.0 MULD 2.0 2.0 SUBD 2.0 0.0 FP adders FP mult’s 2.0 (add1 result)

  31. Tomasulo – cycle 6 Instruction Queue ADDD F4, F2, F0 MULD F8, F4, F2 ADDD F6, F8, F6 SUBD F8, F2, F0 ADDD F2, F8, F0 F0 F2 F4 F6 F8 0.0 2.0 add1 2.0 - 6.0 add2 8.0 add3 1 2 3 ADDD add3 0.0 1 2 ADDD mult1 6.0 MULD 2.0 2.0 SUBD 2.0 0.0 FP adders FP mult’s

  32. Tomasulo – cycle 8 Instruction Queue ADDD F4, F2, F0 MULD F8, F4, F2 ADDD F6, F8, F6 SUBD F8, F2, F0 ADDD F2, F8, F0 F0 F2 F4 F6 F8 0.0 2.0 add1 2.0 - 6.0 add2 2.0 - 1 2 3 ADDD 2.0 0.0 1 2 ADDD mult1 6.0 MULD 2.0 2.0 SUBD 2.0 0.0 FP adders FP mult’s 2.0 (add3 result)

  33. Tomasulo – cycle 9 Instruction Queue ADDD F4, F2, F0 MULD F8, F4, F2 ADDD F6, F8, F6 SUBD F8, F2, F0 ADDD F2, F8, F0 F0 F2 F4 F6 F8 0.0 2.0 add1 2.0 6.0 add2 2.0 1 2 3 ADDD 2.0 0.0 1 2 ADDD mult1 6.0 MULD 2.0 2.0 FP adders FP mult’s

  34. Tomasulo – cycle 12 Instruction Queue ADDD F4, F2, F0 MULD F8, F4, F2 ADDD F6, F8, F6 SUBD F8, F2, F0 ADDD F2, F8, F0 F0 F2 F4 F6 F8 0.0 2.0 - 2.0 6.0 add2 2.0 1 2 3 ADDD 2.0 0.0 1 2 ADDD mult1 6.0 MULD 2.0 2.0 FP adders FP mult’s 2.0 (add1 result)

  35. Tomasulo – cycle 15 Instruction Queue ADDD F4, F2, F0 MULD F8, F4, F2 ADDD F6, F8, F6 SUBD F8, F2, F0 ADDD F2, F8, F0 F0 F2 F4 F6 F8 0.0 2.0 - 2.0 6.0 add2 2.0 1 2 3 1 2 ADDD 4.0 6.0 MULD 2.0 2.0 FP adders FP mult’s 4.0 (mult1 result)

  36. Tomasulo – cycle 16 Instruction Queue ADDD F4, F2, F0 MULD F8, F4, F2 ADDD F6, F8, F6 SUBD F8, F2, F0 ADDD F2, F8, F0 F0 F2 F4 F6 F8 0.0 2.0 - 2.0 6.0 add2 2.0 1 2 3 1 2 ADDD 4.0 6.0 FP adders FP mult’s

  37. Tomasulo – cycle 19 Instruction Queue ADDD F4, F2, F0 MULD F8, F4, F2 ADDD F6, F8, F6 SUBD F8, F2, F0 ADDD F2, F8, F0 F0 F2 F4 F6 F8 0.0 2.0 2.0 10.0 - 2.0 1 2 3 1 2 ADDD 4.0 6.0 FP adders FP mult’s 10.0 (add2 result)

  38. Tomasulo Summary • Prevents Register as bottleneck • Avoids WAR, WAW hazards • Lasting Contributions • Dynamic scheduling • Register renaming (in what way does the register name change?) • Load/store disambiguation

  39. Limitations • Exceptions/interrupts • Can’t identify a particular point in the program at which an interrupt/exception occurs • How do you know where to go back to after an interrupt handler completes? • OOO completion??? • Interaction with pipelined ALUs • Reservation station couldn’t be released until instruction completes, would need many reservation stations.

More Related