LOAD BALANCING SWITCH

1. Final presentation for project By: Oleg Schtofenmaher Maxim Fudim Supervisor: Walter Isaschar • Winter 2007 ( Part A) LOAD BALANCING SWITCH

2. General overview • Software solutions for real-time are too slow • Power dissipation limits work frequencies • Greater computing power needed • H/W accelerators can improve S/W processes • Multi-core, multi-threaded systems are the future

3. Project Goals • Multiprocessor environment for parallel processing of vectors data stream • Maximal Throughput • Configurable hardware • Statistics report • Expandable design

4. System specifications • 1M pulse/sec data stream • Vectors of 8 ÷ 1024 pulses • 1K ÷ 125K vectors/sec • Variable number of processors • System span over multiple FPGAs

5. Problem • How to manage Data stream? • How to manage multiple parallel units? • How to achieve full and effective utilization of resources?

6. Solution • Load Balancing Switch • Converting shared resources to “personal” work space. • FCFS for input, RR for routing/output • Smart management of system • Monitoring for each unit’s load

7. System Block diagram S/W or H/W generator DDR2 Bank A Input vectors Load Balancing Switch (LBS) PCI NIOS VPU NIOS VPU S/W or H/W consumer Data and Control DDR2 Bank B Output reports Statisticsreports PROCStar II Stratix II FPGA

8. Organization of VPU’s(Vector Processing Units) • NIOS VPUs joined into the clusters • Constant number of Clusters • Various number of NIOS VPU’s in cluster • Variable configuration of NIOS • Different Priority for different clusters

9. System Top Diagram NIOScluster NIOScluster NIOScluster NIOScluster DDR2 Bank A Input vectors NIOScluster NIOScluster NIOScluster NIOScluster Load Balancing Switch (LBS) Gidel’s FIFO control IP Data flow NIOScluster NIOScluster NIOScluster NIOScluster DDR2 Bank B Output reports NIOScluster NIOScluster NIOScluster NIOScluster Stratix II FPGA PROCStar II

10. LBS Top Level View FIFO Input Port Input data bus Cluster Arbiter Cluster Arbiter Cluster Arbiter PCI Input Reader NIOS II System NIOS II System NIOS II System Control Main Controller unit Statistics Reporter Control Control and Status Output Writer Control FIFOOutputPort Muxed output data bus Stratix II FPGA

11. System Interfaces Software to Hardware Interface: • Input and Output MultiFIFO PCI data bus • MultiFIFO status • 2x32-bit general read purpose registers • 2x32-bit general write purpose registers • 8-bit information register • Software reset signal

12. Input System Interface LBS Input Interface: • 64 bit data bus from Input MultiFIFO • Read request and ack. Signals • MultiFIFO status flags • SW/HW input signals

13. Output System Interface LBS Output interface: • 64 bit data bus to Output MultiFIFO • Write request and ack. Signals • MultiFIFO status flags • SW/HW input signals

14. Data Packet Format …… Header Tail Data 1 to N of 32-bit Words Header : SW/HW Control 1-bit Unused Nios Number Data Length N Vector ID/Command Type Type 1-bit (Data/Command) Version 4-bit 8-bit 16-bit 32-bit Tail : Sync Data or Checksum(in the future)

15. NIOS Input Interface Hardware: • 64-bit input data bus – from LBS • 10 bit data slices counter – from LBS • Write request signal – from LBS • Chip select signal – from LBS • NIOS ready signal – from NIOS • Data ready signal – from LBS

16. NIOS Output Interface Hardware: • 64 bit output data bus – from NIOS • 7 bit data slices counter – from LBS • Read request signal – from LBS • Chip select signal – from LBS • Output ready signal – from NIOS • Output taken signal – from LBS

17. Twin VPU SystemInput / Output waveform

18. System Demonstration

19. FIFO Input Port Input data bus LBS Units DescriptionInput Reader Input Reader Main Controller unit Statistics Reporter Control and Status Output Writer FIFOOutputPort Muxed output data bus • Reading data from input FIFO • Writing data to selected cluster • Providing header control bits for main controller • Synchronization checks • Vector length counter Cluster Arbiter Cluster Arbiter Cluster Arbiter NIOS II System NIOS II System NIOS II System

20. Input Reader Diagram

21. LBS Units DescriptionInput Controller - FSM

22. FIFO Input Port Input data bus LBS Units DescriptionOutput Writer Input Reader Main Controller unit Statistics Reporter Control and Status Output Writer FIFOOutputPort Muxed output data bus • Reading data from selected cluster • Writing data to output FIFO • Vector length counter Cluster Arbiter Cluster Arbiter Cluster Arbiter NIOS II System NIOS II System NIOS II System

23. Output Writer Diagram

24. LBS Units DescriptionOutput Controller - FSM

25. FIFO Input Port Input data bus LBS Units DescriptionMain Controller Input Reader Main Controller unit Statistics Reporter Control and Status Output Writer FIFOOutputPort Muxed output data bus • Enabling input and output units • Selecting control source (S/W or H/W) • Monitoring clusters’ load via status buses • Selecting clusters for input/output operations • Data validity indication Cluster Arbiter Cluster Arbiter Cluster Arbiter NIOS II System NIOS II System NIOS II System

26. Main ControllerStatus Decoders

27. LBS Units DescriptionMC Status Alghoritm • Status input and output independent decoders • Dynamic port mapping • Always selecting next active neighbor • Suits “similar NIOSes” design • To be expanded in part B

28. LBS Units DescriptionMC Status Alghoritm Status input Dynamic port mapping Compare Active ports Next port 0 0 0 1 1 0 0 0 1 1 1 0 1 0 1 1 1 1 1 1 1 13|4 14|3 14|3 15|2 1|0 2|15 3|14 4|13 4|13 3|14 0|1 1|0 14 13 15 3 2 1 4 0 3 ... ... ...

29. Decoding Flow

30. FIFO Input Port Input data bus LBS Units DescriptionStatistics Reporter Input Reader Main Controller unit Statistics Reporter Control and Status Output Writer FIFOOutputPort Muxed output data bus • Monitoring system activity • Error reporting for software • Counting processed vectors • Throughput = Vectors served / Time of service • To be expanded in part B Cluster Arbiter Cluster Arbiter Cluster Arbiter NIOS II System NIOS II System NIOS II System

31. FIFO Input Port Input data bus LBS Units DescriptionCluster Entity Input Reader Main Controller unit Statistics Reporter Control and Status Output Writer FIFOOutputPort Muxed output data bus • Cluster parametric enabling • Cluster controller • Watchdog • NIOS System Cluster Arbiter Cluster Arbiter Cluster Arbiter NIOS II System NIOS II System NIOS II System

32. LBS Units DescriptionCluster Structure

33. LBS Units DescriptionCluster Controller • Input 4-phase REQ/ACK protocol with NIOS • Nios Ready • Data Ready • Output 4-phase REQ/ACK protocol with NIOS • Output Ready • Output Taken • Smart Status Reporter

34. LBS Units DescriptionCluster Controller

35. Cluster Input FSM

36. Cluster Output FSM

37. LBS Units DescriptionExample for NIOS System • SOPC components: • Input Vector • Output Vector • Nios II • On-chip memory • Timer

38. LBS Units DescriptionInput vector • Export signals from LBS • 64-bit data • Nios/Data Ready • Address , Chipselect , Write request ,Clock , Reset • On-chip memory for 1024 32-bit words • Avalon slave data port for 32-bit data to NIOS II • Avalon slave data ready port

39. LBS Units DescriptionInput vector component

40. LBS Units DescriptionOutput vector • Avalon slave 32-bit data output port from NIOS II • Avalon slave output ready port • On-chip memory for 128 32-bit words • Export signals to LBS • 64-bit data • Output Ready / Taken • Address , Chipselect , Read request ,Clock , Reset

41. LBS Units DescriptionOutput vector component

42. Resource Usage Resource usage data for 5 VPU system VPU resource usage is based on basic NIOS’s with no accelerators and will only increase when accelerators will be introduced.

43. Tasks • Study PROCStar Board – Done • Study Altera’s Stratix II FPGA – Done • Study Quartus and HDL designer– Done • Study GIDEL API – Done • Learn to use Signal Tap tool – Done • Study Altera’s NIOS II – Done • Define interface with software group –Done • Develop signal generator for testing – Done

44. Tasks (cont.) • Define interface with accelerator group – Done • Build direct connection with s/w and NIOS II – Done • Expand design for several NIOS’s – Done • Define basic algorithm for h/w switching – Done • Implementation and debugging of the switch – Done • Integration with NIOS system – Done • SW Test application for operating and integration with hardware design – Done • Integration of LBS with other groups

45. Summary • LBS implementation with SW/HW control and statistics • Up 16 NIOS’s connected to the system • Fully functional S/W – LBS , LBS – NIOS interface • Successful hardware and software integration • Working design examples for other teams

46. Conclusions • Switch concept implemented successfully • Vector transit time is queue and processing only • Two layer abstraction concept = minimize changes • Single level of mastering = minimize resources • 64-bit buses = maximize throughput

47. Tasks for Part B • Increase number of Nios’s in clusters • Improve algorithm for priority cluster selection • Expand statistic reports • Expand SW/HW communication • Add error correction/handling • Add smart vector queue management (SJT) • Spread design to several FPGAs • Multiple Stage LBS ???