1 / 33

LLRF Embedded Development

LLRF Embedded Development. Brian Chase, Joshua Einstein-Curtis Controls Modernization Group 28 September 2018. Introduction. LLRF is a high-speed, high-precision real time system. For the FNAL accelerator infrastructure, have relatively high data rates Similar to beam instrumentation

daphne
Download Presentation

LLRF Embedded Development

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. LLRF Embedded Development Brian Chase, Joshua Einstein-Curtis Controls Modernization Group 28 September 2018

  2. Introduction • LLRF is a high-speed, high-precision real time system. • For the FNAL accelerator infrastructure, have relatively high data rates • Similar to beam instrumentation • Necessitated builds of a Labview client for commissioning, debug, and diagnostics (Backdoor/custom) • Example: • CMTS has 4 ADC and 2 DAC channels per cavity. In addition to the raw data, there are also the baseband IQ and amplitude/phase signals, error, and controller intermediate stages • 2 * (4*2 + 2 + 10) * 65 MSPS * 32-bit = 82 Gbps max • 82/65 = 1.26 Gbps if 1 MSPS sampling • 1.26 Gbps / 8 * 360 = 56 TB/hour • 0.02 * 56 = 1.12 TB/hour • if only 20 ms of 1MSPS data every second LLRF Embedded Development

  3. Real-time Needs • Hard Realtime  LLRF (timescale-dependent, of course) • Failures not allowed • Bucket-resolution timestamping on data • Necessary to do extensive hardware testing or simulation (Qemu, ARM simulator) • Good debugging tools a necessity • Utilization and overloading • Need for optimizations and good compilers LLRF Embedded Development

  4. vxWorks • MVME2400, MVME5500 with Kinetic Systems backplane • VxWorks 5.5, 6.1, 6.4, 6.9 • 5.5 running on MI, FAST, 6.4 on Recycler, 6.9 dev system provided to controls • Power PC 603e, 750 or 7457 • Standardized communication between front ends enables faster development. Shared libraries, etc. • https://www-bd.fnal.gov/controls/micro_p/mooc_project/welcome.html • Protocol Compiler • Custom for each device? Possibly leads to complexity and versioning issues • Each device has own receive stack, and each group develops their own LLRF Embedded Development

  5. Current Model LLRF Frontend sys_init.cpp Spreadsheet Dennis’ Frontends Modification Scripts Python Scripts-to-dabbel Acnet and Front-end independently record and manage variables LLRF Embedded Development

  6. SoC MFC LLRF Embedded Development

  7. LLRF Control and Timing • VUCD • Developed in mid-nineties • No replacement for current systems available • Timing and data decoding using backplane triggers • Multi-timer system • No planned replacement for current/deployed systems • Bucket resolution and accurate timestamping needed for future diagnostics LLRF Embedded Development

  8. Build Environment • nova build environment (esd tools) well developed • LLRF projects are all there, including configuration files for new SoC systems (spreadsheet register maps and defaults) • Need for a well-developed upgrade plan across departments • Need for supporting both legacy systems and new systems using same tools? • Significant increases in performance with each controller, but processor-specific programming can make porting difficult LLRF Embedded Development

  9. Revision Control • Currently part of build toolchains on nova • One of the largest issues we’ve run in to is revision and versioning • Separate versions for firmware, software, libraries • Storing git commits in f/w and software works well • Xilinx allows for same bitfile from same files • routing/placement hash/seed is based on files included in project • Enforcing this across all project levels is a challenge • PLC versioning? Software versioning? HDL versioning? LLRF Embedded Development

  10. Revision Control • LBL/LCLS-II and CERN have self-describing firmware/software • Register maps in control system and front-ends match • gzip’d JSON in memory on devices • CERN’s FESA and cheby • Acnet database • TFTP/remote image loading is crucial • Limit wear on flash (limited write) devices • Remote/central logging is crucial • See above • Verification and Continuous Integration are valuable tools LLRF Embedded Development

  11. What is plan for future? • Responsibility of each department? • High-speed data and diagnostics • artDAQ? FTP? Something new? • DAQ analysis and storage • High-speed means what for each system? • Data lifetime? • Scope capabilities • Event identification • Design lifetime • 20 years? 5 years? 1 year? • Tool availability and automation • Build system for non-experts • Nova replacement? Command-line functionality? LLRF Embedded Development

  12. Looking Forward • What will be available in 2 years? 5 years? 10 years? • Need a long-term plan • Due to few resources at developer level, really need process and procedure directed from managment • Hardware - should there be a standard developed across groups? • Shared FPGA/Processor boards? • Crate standards? • Physical interfaces? LLRF Embedded Development

  13. Xilinx Zynq Ultrascale+ RFSoC Designed for cell base stations, radar, military Single-chip includes all necessary for an RF system, but might not meet performance requirements of modern accelerators. Separate application and real-time processors Shared memory makes for very powerful, high-bandwidth toolset https://www.xilinx.com/products/silicon-devices/soc/rfsoc.html LLRF Embedded Development

  14. Conclusion • Concerns for future • DAQ and timing/precision • Data retention and high-speed data management • Revision Control and Build Environment are significant concerns • Ease of integrating in to control system: shared protocols, development architectures, etc • Need a multi-year, division-wide path forward • While still supporting legacy systems • Thanks! LLRF Embedded Development

  15. Extra Slides LLRF Embedded Development

  16. Switching to Fiber • Fiber benefits outweigh slightly increased cost • Decreased power • Higher bandwidth • Low cost per bandwidth • Relative ease of implementation given available tools LLRF Embedded Development

  17. Communication Proposals • 9.027 MHz RF synchronous frame rate • Local clock source needed on each device • 1300 MHz harmonic as data rate • Need known frame size (ie 128-bit) • Previous State, Events, Beam pattern for next group • Single-source star • Send as timing/events -- Return line as MPS • Data concentrators? • Throw out proposal • 3 links – DAQ, timing/MPS, control LLRF Embedded Development

  18. LLRF Embedded Development

  19. Intel Cyclone V and Arria 10 LLRF Embedded Development

  20. SoC MFC Test DAQ System Courtesy Ed Cullerton LLRF Embedded Development

  21. Arm Product Line https://www.silabs.com/documents/public/white-papers/Which-ARM-Cortex-Core-Is-Right-for-Your-Application.pdf LLRF Embedded Development

  22. Microsemi SmartFusion2 Single Event Upset Immune design LLRF Embedded Development

  23. NXP LPC17xx Kinetis K60 i.MX LLRF Embedded Development

  24. Why not Pis? • The question is really: what does real-time mean? • Bucket-level resolution and repeatability? • TCLK decoding accuracy? Event decoding accuracy? • What runs at each layer? • Firmware/embedded RTOS • Software/application layer • Lifetime and long-term support • Open Hardware • Interconnect standards • SPI + I2C support? (1 SPI, 1 I2C, 1 UART) • Standard interconnects? • Bus/backplane protocols? LLRF Embedded Development

  25. DSP • LLRF requires higher-performance DSP and DAQ • Newer FPGAs are offering dedicated floating-point blocks • Require use of special tools or overloading standard operators • Integrated simulation and verification environment being more important • Collaboration requires the ability to transfer IP easily LLRF Embedded Development

  26. SoM Benefits https://www.intel.com/content/www/us/en/programmable/products/soc/ecosystem/system-on-modules.html LLRF Embedded Development

  27. Linux (and variants) • Linux w/ preemptRT • RTAI (Linux as task) • NI Real-Time Linux (dual-schedulers) LLRF Embedded Development

  28. RTOS • FreeRTOS -- https://www.freertos.org/ • Contiki (IoT) -- http://www.contiki-os.org/ • MicroC/OS-IIx • Mbed (IoT) • RTEMS • vxWorks • Now a Linux variant • MQX LLRF Embedded Development

  29. MPC603e LLRF Embedded Development

  30. MPC750 LLRF Embedded Development

  31. MPC7457 LLRF Embedded Development

  32. Processing Models • IOC+Epics • “Dumb” front-end with managing IOC • Subscriber model • Crate processors • Slot 0 manages all cards in crate. Advanced data processing happens ??? • NADs • Multiple models available • Acnet • Requirements on who does each portion is not defined • Controls communication (MOOC(AN), libacnet, erlang, PC) • Debug communication (Labview, Python) • Front-end intelligence LLRF Embedded Development

More Related