Control System Overview

1. Control System Overview L. R. Dalesio EPICS Collaboration March 11, 2008

2. Outline • Requirements • Standard Control System Components • Development Opportunities • Embedded Device Control • Client/Server Architecture for High Level Applications • Relational Database and tools for all configuration parameters • Near Term Plans • Scope of Work and Budget • Concluding remarks

3. Control System Requirements – 1 of 3 • Bunch Length 1-40 psecs • 2.6 usec ring revolution • Top off every 1 minute • Top off bunch train 140-300 nsec • Top off damping time 10-50 msecs (no extraction) • Synchronous reading/writing (latency?jitter?future settings?rate?) • Manual control of orbit trims, quadrupoles, sextupoles, and insertion devices are asynchronous • ~10 Hz write/read is suitable for “turning knobs” for a power supply • 5 Hz updates to operators of up to 1000 chosen parameters • Archive up to 6000 parameters at a rate of 2 Hz continually • What is done elsewhere? • Must scale to support 150,000 physical I/O connections and 400,000 computed variables • 99.99% availability 24/7

4. Control System Requirements – 2 of 3 • Transient Recording • Take coherent turn by turn orbit data for up to 800channels 1024 turns • Latch the last 10 seconds of data from all parameters in the storage ring • Others? • Beam line needs 1 msec archiving over 1 minute for temperatures and positions • Provide data for all control aspects • 5 KHz RF Feedback on beam phase • 10 kHz orbit feedback, (100 usec loop time) • 300 BPMs (10 per cell) • 2 * 120 Corrector PS in 90 I/O Controllers (IOC) • 20 msec equipment protection mitigation (too long?) • 1 Hz model based control • 10 kHz power supply read backs triggered from timing sys • 10’s of Hz Data Collection for RF loop correction. • 80 psecs pulse to pulse timing jitter. • During top off, some beamlines will need 1.1 - 1.8 psecs of timing jitter • Machine physics thinks this number should be < 2 psecs • Provide electron detector as event for beam line?

5. Control System Requirements – 3 of 3 • First need dates • 2008/03 – vibration tests on girders done using stand alone software. Some vibration sensors will be used online. • 2008/06 – support libera BPM evaluation • Where do the rest of the diagnostics fit • Where does the conventional facility demonstrator fit • Where does the power supply test stand fit • 2008/08 – X-9 upgrade for NSLS beam line • 2008/09 – low level RF lab support • 2009/06 – support water systems • 2009/09 – support thermocouple measuring • 2009/09 – support vacuum test stands • 2010/03 – support high power RF • Control Room • Must be able to be relocated • Control room in the LOB for commissioning • Control room re-located with accelerator team for operation

6. Nomenclature Standard Psy:PI-Ssy:SI-Tsy:TI<Dev:DI>Sg:SgI-SD System Device Signal

7. Control System Software Standards • EPICS based • Embedded Real-Time Operating System choices: RTEMS which is in use at LCLS, Spear, and CLS or VxWorks which is in use at APS, SNS, Diamond, and SLS. • Linux Workstations • Use of standard EPICS engineering tools: Extensible Display Manager, Channel Archiver, Striptool, Alarm Handler • Evaluate the possibility of participating in the CSS development • Visual Database Configuration Tool w/ Modifications for Table Entry • Use of physics applications: Matlab Middle Layer Toolkit (MMLT) which is in use at LBL, Spear, Diamond, SLS, CLS, and ASP, and eXtensible Accelerator Language (XAL) which is in use at SNS and Elegant which is in use at APS. • Use of Beam Line Applications: Matlab in place of SPEC, BluIce, IDL, SynApps

8. Control System Architecture Operator stations: Displays, Archiving, Alarm Management, Strip charts, Save/Restore Utility Events/ / Timing Data Shared Memory Ethernet – EPICS Channel Access Protocol Read Remote Ethernet and serial devices. High density IO, Motor Control Position Control, Image Acquisition, Non FOFB Diagnostics CPU CPU EVR CEL L EVR CEL L MTRS EVG CEL L EVR CPU I /O CPU BPM IOCs PS IOCs Timing Master Instrumentation Ethernet BPM PS PLC CNET I /O I /O ALS NET Fast Equipment Protection Signal BPM PS BPM PS BPM PS BPM ….. ….. Field I/O Field I/O Field I/O Field I/O BPM PLCS ,Slow I/O, High Reliability, Low Accuracy, High Density Vacuum, PPS, MPS, Non FOFB PS, Cryo., Facility control

9. Control Hardware Standards for Test Stands • Dell Linux development workstations • PLC Solutions • Allen Bradley Control Logix • Allen Bradley FlexIO • Building Automation • ALS Needs fast Ethernet based interface board • VME crates • Rittan 4 slot VME64x-2U4S-PS300C-SM 4,395.00 • Rittan 7 slot VME64x-4U7S-PS900C-SM 5,510.00 • Weiner 9 slot VME195xPO 6,050.00 • Weiner 21 slot VME6023/611_JL 7,271.00 • CPU Boards • Motorola MVME 5500 3,000.00 • PP410 3,500.00 • Motor Controllers • Hytec 8601 with motor driver 32 axis ~1,000/axis • Newport XPS Intelligent motion controllers

10. Areas For Technical Development • Embedded Controllers need an open standard for high speed, deterministic functions. Work with other labs and board manufacturers to develop one. • High Level Applications currently tie together functions through data or file structures. To make the components of High Level Applications modular and distributed, a client/server architecture is needed. • Relational Databases support data management through the life of a project. Development of adequate tools to enter and report this data is required early.

11. Control Hardware For Beam Synchronous Applications Physics Applications: Correlation plots, orbit calculations, beam dump data Events/ / Timing Data – Beam Trip Event, Synch Data Event 2 GB Shared Memory – Communicates Cell Data @ 20 usecs Ethernet – EPICS Channel Access Protocol CPU CPU EVR CEL L EVR CEL L EVG CEL L CPU Compute Fast Orbit Feedback BPM IOCs PS IOCs Timing Master BPM PS Fast Equipment Protection Signal BPM PS BPM PS BPM PS BPM BPM

12. Embedded Controllers - Per Cell BPM Controller BPM Controller BPM Controller BPM Controller BPM Controller BPM Controller VME I/OC EPICS RF Clock Distribution 45 MHz VME Interface +T0 + - Settling time on BPMs +0.0 usecs - BPM to Compute Controller 384 bits = 64 bits * 6 BPMs 4.9 usecs = 384 bits over 100 MBit enet +4.9 usecs - Compute Controllers to each other 10,290 bits = 30 nodes * 384 bits 10.5 usecs = 10,290 bits over 2 GB enet +15.4 usecs – Compute local matrix 0 usecs + 15.4 usecs - Communicate t Power Supply Controllers 24 = 4 PS * 4 bytes each 3.5 usecs +18.9 usecs – loop complete settling time for magnets communicate diagnostic waveforms etc… +200 usecs – start again Fiducial Distribution 1 Hz Prev Cell Core Controller VME I/OC EPICS VME Interface Core Controller PS Controller PS Controller PS Controller PS Controller VME I/OC EPICS VME Interface 100 MB ENET Next Cell Core Controller 2 GBit ENET

13. Embedded Controllers - Approach • Develop a prototype cell controller: • Redundant 2 Gbit communication paths for peer to peer communication • RF timing signals • Verify communication and timing jitter meet requirements • Develop the interface from the cell controller to a processor for integration into EPICS • In parallel: • Develop the inexpensive device controller with redundant 100 MBit controllers • Develop the 100 Mbit receive and transmit circuits for the Cell Controllers • Develop the EPICS interface to the Cell Controller

14. Embedded Controllers - Status • FY 08 • Purchase order in place with LBL (Alex Ratti and Larry Doolittle) • Develop a prototype cell controller: • Redundant 2 Gbit communication paths for peer to peer communication • RF timing signals • Verify communication and timing jitter meet requirements • Develop the interface from the cell controller to a processor for integration into EPICS • FY 09 • Develop the inexpensive device controller with redundant 100 MBit controllers • Develop the 100 Mbit receive and transmit circuits for the Cell Controllers • Begin integration of these device controllers • Libera • Power supply control • LLRF control

15. High Level Applications – Client/Sever Configuration Parameters Measured Orbit Optics Beam Resp. Matrix (S) Lattice Orbit Differences Conversions Model Data Name Mapping Optics Deviations Physics Applications Mid Level Model Client/Server Gradient Errs & Corrections Application / Family Control System / Channels Computed Data Beam Resp. Matrix Diff’s EPICS Client/Server Need to port Need to develop Existing State Data I/O Controllers

16. High Level Applications - Approach • Install an operational NSLS II simulation using Matlab Middle Layer Toolkit • Define an interface library for physics applications to communicate to the configuration parameters, calibrations, model data, and measured data. • Port some subset of the available packages (Matlab Middle Layer Toolkit, Elegant, XAL, Dimad, and SAD) onto the interface library one application at a time. • Develop a client/server implementation of the interface library. • Use an RDB to store configuration parameters and calibrations.

17. Relational Database • Support all configuration data • Provide tools as they are needed throughout the project. • Make the RDB the source of all configuration data and derive it from there for all uses.

18. Relational Database - Approach • Use the IRMIS database as the basis for our component and wiring database which is currently in use at APS, SNS, CLS, Diamond, and SLS. • Extend the database to support Optics early through a contract with Tech-X in conjunction with our physicists and control group. • This contract includes a name mapper and lattice input / report tool • Develop tools through contracts, collaboration, and the hire of a second RDB expert. • We are preparing SOWs for Naming Tool and Wiring Input • Schedule the time and put in the project rigor to use the RDB and tools as new portions of the project are needed.

19. WBS 1.03.05 Global Controls System WBS Dictionary: Hardware and software for the global controls of the linac, booster ring, transport lines, storage ring, network, relational database, data archiving, timing, fast feedback, and physics application support. ~ 10% of Accelerator

20. Near Term Plan • Start R&D • Contact Nearly in place for development of open-source embedded device controllers with LBL team (Larry Doolittle and Alex Ratti) • RDB server is purchased and we are ready to Install IRMIS • Tech-X contract ready to go out the door to extend IRMIS to support Optics • Position open for someone to Develop RDB tools for entry and reports • Installed NSLS II Simulation through IOCs • Installed Matlab Middle Layer Toolkit for Commissioning • Two positions in place to Start Development of API for High Level Applications (one starts in May, the other position should be filled by March) • Make Preliminary Hires • RDB job ad is placed. A candidate has been indentified and could start by March 15th.. • High Level Applications Engineers – Guobao Shen starts May 10, 2nd position starts by March 15th.. • 2 positions are open for project engineers • 1 position is open for an embedded systems expert • Completed preliminary design • Completed extensions to Visual DCT for Spreadsheet based input

21. Concluding Remarks • Global requirements are complete • Equipment control S/W and H/W infra-structure is selected. • A collaboration of LBL and SLS EEs along with NSLS II team are prepared to begin the open-source, embedded device controller development • RDB tables and tools have begun building on the success of IRMIS. • High Level Application environment is selected. The Diamond simulation running under EPICS is installed along with the Matlab Middle Layer Toolkit as a starting point at KEK by Guabao Shen. • The API for High Level Applications is being developed in conjunction with MMLT, SDDS, and XAL authors. • Staff Acquisition is a critical activity and we have made good progress.