1 / 46

Advanced Microarchitecture

Advanced Microarchitecture. Lecture 10: ALUs and Bypass. This Lecture: Execution Datapath. ALUs Scheduler to Execution Unit interface Execution unit organization Bypass networks Clustering. Result. Opcode. “ALU”. Adder. Logic. Shift. Mult. Div. Operand1. Operand2. ALUs.

mave
Download Presentation

Advanced Microarchitecture

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. Advanced Microarchitecture Lecture 10: ALUs and Bypass

  2. This Lecture: Execution Datapath • ALUs • Scheduler to Execution Unit interface • Execution unit organization • Bypass networks • Clustering Lecture 10: ALUs and Bypassing

  3. Result Opcode “ALU” Adder Logic Shift Mult Div Operand1 Operand2 ALUs • ALU: Arithmetic Logic Units • FU: Functional Units • EU: Execution Units Adder ALU What’s the difference? Implementation details, algorithms, etc. of adders, multipliers, dividers not covered in this course Lecture 10: ALUs and Bypassing

  4. Interfacing ALUs to the Scheduler • Issue N instructions • Read N sets of operands, immediates, opcodes, destination tags • Route to correct functional units Fetch & Dispatch ARF PRF/ROB Physical register update Data-Capture Scheduler Bypass Functional Units Lecture 10: ALUs and Bypassing

  5. Effectively one nasty crossbar Data-Capture Payload RAM Payload RAM Issue Port 3 Select Logic opcode ValL ValR Issue Port 0 opcode ValL ValR Issue Port 2 opcode ValL ValR Select decisions, port bindings, etc. Issue Lane 0 Issue Lane 1 Issue Lane 2 Issue Lane 3 Lecture 10: ALUs and Bypassing

  6. Payload RAM select 0 select 1 select 2 select 3 Issue 0 Issue 3 Issue 2 Issue 1 “Register File” Organization Each RF read port input has a 1-to-1 correspondence with one and only one RF read port output No MUXing of outputs is required Register File “R1” “R7” “R3” “R4” val(R1) val(R7) val(R3) val(R4) Lecture 10: ALUs and Bypassing

  7. SRAM Row Decoders But how do you assign which set of data gets routed to which set of read port outputs? “Register File” Is An Overkill RS entries Payload RAM Select Select Select Select Lecture 10: ALUs and Bypassing

  8. Execution Lane ↔ Select Binding RS entries Payload RAM Select Select Select Select Payload RAM read port outputs are in the same order as the Select Blocks Lecture 10: ALUs and Bypassing

  9. Tri-State Driver Single Entry Close-Up Select Port 0 Select Port 1 Select Port 2 Select Port 3 Opcode Src L Src R Single RS Entry grant 0 bid 0 grant 1 bid 1 grant 2 bid 2 grant 3 bid 3 One RS entry can only bid on one select port, so payload never driven to more than one port Output buses connected to all payload RAM entries Each select port only gives the grant to a single RS entry, so more than one payload entry can never drive the same payload output port Lecture 10: ALUs and Bypassing

  10. Need to “Swizzle” at the End Opcode Silo Src L Silo Src R Silo Nasty tangle of wires (Src’s are 64-128 bits each!) Lecture 10: ALUs and Bypassing

  11. Non-Data-Capture Scheduler Register File Row Decoders RS entries Payload RAM Register File SRAM Array Select Select Select Select Src L tags Src R tags Lecture 10: ALUs and Bypassing

  12. Immediate Values • data-capture can store immediate values in payload bay • non-DC needs separate storage • Could add extra field to payload • could allocate a physical register and store the immediate there • Could store in a separate “immediate file” Lecture 10: ALUs and Bypassing

  13. Distributed Scheduler • Grant/Payload read lines may have to travel further horizontally (multiple RS widths) • ScheduleExecute latency less critical than ScheduleSchedule (wakeup-select) loop latency FM/D FAdd Shift Store FP-Ld FP-St ALU1 ALU2 M/D Load Select 2 Select 3 Select 0 Select 1 Payload RAM Lecture 10: ALUs and Bypassing

  14. store Fadd FMul shift mult load FDiv add div Naive ALU Organization • Besides making scheduling hard to scale, arbitrary any issue  any ALU makes operand routing a horrible mess (needs full cross bar) From Payload/RF Read Ports Lecture 10: ALUs and Bypassing

  15. FPCvt FP ld store mult load shift add add div Execution-Port-Based Layout • Just need to fan-out data to FUs within the same execution lane; no cross-bar needed • Each FU needs a “valid” input to know that the incoming data is meant for it and not another FU in the same lane • Or just let them all compute in parallel and use only the output that you want  wasted power Lane 0 Lane 1 Lane 2 Lane 3 Lecture 10: ALUs and Bypassing

  16. shift mult add div Bypass Network Organization From Payload RAM/Register File N × 2 sets of inputs f × 64 bits N=Issue Width, f=Num FUs O(f2N) area just for the bypass wiring!!! … which is cubic since f = W(N) f × 64 bits Previous slide had f=9 FUs, and that didn’t even include all of the FP units Lecture 10: ALUs and Bypassing

  17. Fadd FPCvt store Fmul load FP st shift mult Fdiv add add div FP ld ALU Stacks From Payload/RF Integer Bypass Floating Point Bypass Bypass  FU Fan-Out Bypass MUXes reduced to one pair per ALU stack (as opposed to one per FU) Lecture 10: ALUs and Bypassing

  18. FPCvt store FP ld FP st Fadd shift Fmul mult load Fdiv add add div Bypass Sharing From Payload/RF Integer Bypass Floating Point Bypass Bypass wiring reduced to one output per execution lane/ALU stack Bypass  FU Fan-Out Local FU Output Lecture 10: ALUs and Bypassing

  19. shift load add S X X E1 E2 2-cycle shift, to Lane 1 S X X E 1-cycle add, to Lane 1 Two instructions want to writeback using same bypass path! X Bypass Sharing (2) • If all FU’s in a stack have the same latency, writeback conflicts are impossible • because only one instruction can issue to each lane per cycle • But not all FU’s have the same latency: Lecture 10: ALUs and Bypassing

  20. E1 E2 X X Bypass Sharing (3) • How to resolve this structural hazard? • Obvious solution: stall • Creates scheduling headaches • Treat bypass/WB as another structural resource • Separate select logic* for bypass allocation 0 1 2 3 4 5 To Bypass S X X 2-cycle shift, to Lane 1 To Bypass S S E 1-cycle add, to Lane 1 0 1 2 3 4 5 6 *Not same as regular select logic, just a table read/write Writeback Scoreboard X Lecture 10: ALUs and Bypassing

  21. S X X E S X X E1 E2 E3 Bypass Sharing (4) 0 1 2 3 4 5 6 7 8 S X X E1 E2 A: 2-cycle shift, to Lane 1 S B: 1-cycle add, to Lane 1 S S C: 3-cycle load, to Lane 1 0 1 2 3 4 5 6 7 8 Wasted issue opportunity: B picked by select, but cannot issue due to WB conflict C could have issued, but is stalled by one cycle Select B C Lecture 10: ALUs and Bypassing

  22. FPCvt store FP ld FP st Fadd shift Fmul mult load Fdiv add add div Bypass Critical Path Total wire length is about twice the total width plus twice the total height Lecture 10: ALUs and Bypassing

  23. Each execution lane/ALU stack is self-contained FPCvt store FP ld shift mult load add add div Longest path only crosses total width once Bypass Critical Path (2) Lecture 10: ALUs and Bypassing

  24. Bypass Control Problem • We now have the datapaths to forward values between ALUs/FUs • How do we orchestrate what goes where and when? • In particular, how do we set the controls of each of the bypass MUXes on a cycle-by-cycle basis? Lecture 10: ALUs and Bypassing

  25. add mul Port 2: MUL P30 = P21 * P17 S X X E E E Scoreboarding • For each value produced, make note (in the scoreboard) of where it will be available • For each source, consult scoreboard to find out how to rendezvous 4 R 0 1 2 3 4 5 6 7 17 - 0 Port 1: ADD P21 = … S X X E 21 1 R Port 0: ADD P17 = P21 + P4 S X X E Lecture 10: ALUs and Bypassing

  26. add Scoreboarding (2) • Setting bypass controls is easy • Read where the value will come from and feed to bypass MUXes in the operand read stage P21 1 Payload (src tags) WB Score board P4 R • May add scheduleexecute stages for data-capture scheduler • why not for non-data-capture? Lecture 10: ALUs and Bypassing

  27. Value read from RF A needs to update SB this cycle for C to correctly source its operand Scoreboarding (3) • Updating can be more complicated • Depends on when SB read occurs w.r.t. operand reading • earlier reads cause more disconnect Assume SB read in 1st cycle after schedule RF A Value bypassed, WB to RF S X X E1 E2 E3 B S X X E C S X X E Lecture 10: ALUs and Bypassing

  28. Scoreboarding (4) • Scoreboard can become a critical timing bottleneck • All sources must read from scoreboard • All destinations must update scoreboard • Once at schedule to indicate bypass location • Once later to indicate value has written back to RF • ~ 4×N ports for the scoreboard! • If scoreboard becomes multi-cycle, things can get really crazy • need to bypass scoreboard reads/writes like inter-group rename bypassing Lecture 10: ALUs and Bypassing

  29. CAM-based Bypass • Extend data-capture concept to bypass network Register Value from Payload/RF Result Value Register Tag Result Tag Lane 0 Lane 1 Lane 2 Lane 3 = = = = Use Lane 0 Use Lane 1 Use Lane 2 Use Lane 3 Use PL/RF Lecture 10: ALUs and Bypassing

  30. CAM-based Bypass (2) • Must carry destination tag to execution and broadcast along with result • But you have to do this anyway; need the destination tag for RF writeback • A lot of CAM logic • Costs power and area • Control is simple: it’s basically control-less Lecture 10: ALUs and Bypassing

  31. Writeback to Data-Caputure • Looks very similar to bypass CAM Payload of DC Scheduler = = = = = = = = Exec Lane 0 Exec Lane 1 Exec Lane 2 Exec Lane 3 SrcL SrcR ValL ValR Lecture 10: ALUs and Bypassing

  32. ?? A B Problem: How does C pickup the value of P21? A B PRF Writeback Latency Physical Register File (3-cycle write latency) A: ADD P21 = … B: ADD P17 = P21 + … C: MUL P30 = P21× P17 A B C A Bypass Network Lecture 10: ALUs and Bypassing

  33. Multi-Level Bypass • Bypass network must cover the latency of the writeback operation • If WB requires N cycles, then bypass must be able to source all N cycles worth of results Physical Register File C B A But this is only for one ALU (or ALU stack) C B A 3-level Bypass From PL/RF Lecture 10: ALUs and Bypassing

  34. Superscalar, Multi-Level Bypass 3-cycle PRF WB latency ALU Stack 0 ALU Stack 1 ALU Stack 2 AL Lecture 10: ALUs and Bypassing

  35. A Bit More Hierarchical To Physical Register Writeback ALU Stack 0 ALU Stack 1 ALU Stack 2 ALU Stack 3 Lecture 10: ALUs and Bypassing

  36. Bypass Network Complexity • Parameters • N = Issue width • f = Number of functional units • b = bit width of data* (e.g., 32 bits, 64 bits) • D = Network depth (RF write latency) • Metrics • Area • Latency … Both contribute directly to power *For CAM-based bypass logic, should include tag width as well Lecture 10: ALUs and Bypassing

  37. Bypass Network Complexity (Area) • Width • 2×(N+D) + 1 inputs at b bits each • Replicated N times • Total 2N2b + Nb(D+1) • Height • N values at b bits each, times D levels • MUXes: O((D-1)×(lg N) + lg(N+D)) • Assume FUs per ALU stack is constant: f/N = O(1) • Total O(NDb) • Total Area • O(N3b2D + N2b2D2) • Cubic in N, Quadratic in D and b N stacks N values O(lg N) N values O(lg N) N values O(lg(N+D)) N+D inputs ALU Stack 0 O(f/N)-to-1 MUX for outputs: O(lg(f/N)) height 1 value Lecture 10: ALUs and Bypassing

  38. Bypass Network Complexity (Delay) • ALU output to 1st latch • O(lg(f/N)) gates for the MUX • O(N+D) wire delay horizontally • O(f/N + lg(N+D)) wire delay vertically • Last latch to ALU input • O(N+D) wire horizontally • O(lg N) gate delay for 1st MUX • O(N + lg N) wire delay vertically • O(lg(N+D)) gate delay • Gate Delay (worse of the two) • O(lg(N+D)) or O(lg(f/N)) • Wire Length (ditto) • O(N + D + f/N) • Unbuffered wire has quadratic delay N stacks N values O(lg N) N values O(lg N) N values O(lg(N+D)) N+D inputs ALU Stack 0 O(f/N)-to-1 MUX for outputs: O(lg(f/N)) height 1 value Lecture 10: ALUs and Bypassing

  39. Bypass Network Complexity* * Complexity analysis is entirely dependent on the layout assumptions. For example, hierarchical vs. non-hierarchical bypass organizations lead to different areas, wire lengths and gate delays When someone says “this circuit’s area scales quadratically with respect to X”, this really means that “this circuit’s area scales quadratically with respect to X assuming a layout style of Z” Lecture 10: ALUs and Bypassing

  40. ALU Clustering • The exact distribution of FUs to ALU stacks and/or select binding groups can affect layout • Already saw how separation of INT and FP stacks reduces unnecessary datapaths • Has additional benefits when bits(INT) != bits(FP) • Ex. x86 uses 32/64-bit integers, but internally uses 80-bit FP • SSE3 introduces 128-bit packed SIMD values, but normal GPRs are still only 64 bits wide • Certain instructions do not generate outputs (branches) • Memory instructions treated differently (outputs go to LSQ), and stores don’t generate a register result Lecture 10: ALUs and Bypassing

  41. Clustered Microarchitectures • Bypass network delays scale poorly • Scheduling delays scale poorly • RF delays scale poorly • Partition into smaller control and data domains Lecture 10: ALUs and Bypassing

  42. FUs FUs FUs FUs Clustered Scheduling RS Entries (Cluster 1) RS Entries (Cluster 2) RS Entries (Cluster 3) RS Entries (Cluster 0) Cross-Cluster Wakeup Interconnection Network Cross-cluster wakeup may take > 1 cycle Payload0 Payload1 Payload2 Payload3 Execution Cluster 0 Lecture 10: ALUs and Bypassing

  43. Cross-Cluster Wakeup A C D B  E But a different clustering algorithm only needs 3! Cross-Cluster Wakeup Delay 2 cluster, round-robin cluster assignment A C B D Cross-Cluster Wakeup A C E   Normally takes 3 cycles (assume all 1-cycle latencies) B D  E Now it takes 5 cycles Lecture 10: ALUs and Bypassing

  44. FUs FUs FUs FUs Cross-Cluster Bypass Payload0 Payload0 Payload0 Payload0 Cross-Cluster Bypass Network Similar delay issues like the case for scheduling Values may take > 1 cycle to get from cluster to cluster Lecture 10: ALUs and Bypassing

  45. X X X X E Penalties are not additive! Cross-Cluster Bypass (2) • So do we have to pay X-cluster penalties once at schedule and again at bypass? This assumes that the Wakeup Delay (CiCj) is equal to the Bypass Delay (CiCj) If true for all i and j, then bypass and wakeup delays always overlapped S X X X X E A  S B B schedules 2 cycles after A due to extra cycle of wakeup delay Lecture 10: ALUs and Bypassing

  46. Clustered RFs • Place 1/nth of the physical registers in each cluster • How to partition? • ARF/PRF: read at dispatch, extra latency may require more levels of bypassing • Unified PRF: latency may make schedexec delay intolerable (replay penalty too expensive), plus all of the bypassing • Replicate PRF • Keep a full copy of the register file in each cluster • Reduces per cluster read port requirements • Still need to write to all clusters (each cluster needs full set of write ports) Lecture 10: ALUs and Bypassing

More Related