1 / 26

Trigger Coordination Report

Trigger Coordination Report. Wesley H. Smith U. Wisconsin CMS Trigger Coordinator LHCC Rehearsal Meeting April 18, 2008 Outline: Level 1 Trigger DAQ & HLT Infrastructure HLT Algorithm performance -- Christos Leonidopoulos. Calorimeter Trigger Primitives. ECAL Barrel:

galvin
Download Presentation

Trigger Coordination Report

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. Trigger Coordination Report • Wesley H. Smith • U. Wisconsin CMS Trigger Coordinator LHCC Rehearsal Meeting April 18, 2008 Outline: Level 1 Trigger DAQ & HLT Infrastructure HLT Algorithm performance -- Christos Leonidopoulos

  2. Calorimeter Trigger Primitives • ECAL Barrel: • Barrel: All Trigger Concentrator Cards (TCC-68) installed • Tests with Regional Cal. Trig. Ongoing: patterns, timing • Ready for operation in CR0T • ECAL Endcap: • TCC-48 submitted for board manufacture • Present “schedule” (under negotiation!): • Qualification July 1 • 1st batch October 15 • 2nd batch November 7 • 3rd batch December 1 • HCAL: • All installed, operating in Global Runs

  3. Regional Calorimeter Trigger • Hardware • RCT Hardware is complete in USC55 and ready for CR0T • For CRAFT: Finalize timing • Monitoring: • Link Status: Efficiency Status (GREN Data):  Links off in order to test display

  4. Global Calorimeter Trigger • Status: • All 63 required Source Cards are in the system and debugged • One Wheel card with 3 Jet Leaf Cards installed in GCT main crate before Easter & corresponding jet fibres connected • Available for May run: • The Electron Trigger. • The Positive Rapidity Jet Data. • use the emulator on these data to produce trigger distributions. • As soon as the second Wheel Card goes in we will have a very good picture how the Jet-Et distributions look without beam. • All Hardware Integrated in the Trigger Supervisor • Source Cards, Electrons and Jets • Available for June run: • All Electron & Jet Triggers

  5. Drift Tube Trigger • Local Trigger + Sector Collector ready (5 wheels) • Phi DTTF connection ready • Eta connection being prepared • All DTTF Crates mounted • All 72 PHTFs will be installed • 20 spare boards to assemble and test soon • TIM and DLI are all available + spares • Input fibers • Input fibers in the final position • CSC connection • All cables in final position • Transition Boards being installed

  6. CSC Trigger ME1 ME2 ME3 local run 41356 – April 17 • Reconstructed positions from 3 CSC stations with CSC trigger (213/234 CSCs active) • Trigger configuration: LHC pattern trigger (cosmic ray rate ~ 28 Hz) • Integrated with Trigger Supervisor, Emulator Validated (excl. ME1 phi flip) • Monitoring data available, panel in development, DQM in use with plots available

  7. RPC Trigger • Link System (UX) • YB+2,+1,0 - commissioned from USC; • YB-1,-2 - in commissioning from USC now; • YE+3,+1 - LBBs installed and cabled; • YE+3,+1 - boards installed in 75%, waiting for LV; • YE-1,-3 - LBBs installed and cabled, boards not installed. • Enough modules for barrel & + endcap installation, BUT lack of modules for - endcap and spare modules - need to fix broken modules or small production! • Trigger System (US) • Trigger Boards:64 TBs (all needed for staged RPC trigger system) at CERN • Trigger Crates (12): • 6 - installed now in USC (4 with TBs, installed & cabled), • 6 - will be installed, filled with TBs and cabled in one month. • UX-US cables • YE+3:YB-2 - installed, YE-1,-3 - waiting for installation; • USC trigger cabling • TTC, GMT, DCs, Slink, DCC cables – installed/on demand, • Splitter cables delivered to CERN, cabling in April and May

  8. RPC Trigger News • Optical transmission problems between RMB and DCC • Eliminated adjusting GOL laser power (default settings too small). • Minor hardware fix needed. • All recently shipped boards have the fix in. Need retrofit on 3 crates. • DCC firmware • Updated. Improved distribution of L1 on board. Currently under test • Noise Problems • One sector has fluctuating noise -- special rate histograms implemented on LB • Software • Integrated with Trigger Supervisor • Emulator has been stable for a year • Inconsistency in CMSSW geometry: no endcap RPC triggers for the moment • L1 DQM working with initial histograms available • Integrated with Configuration Database, preparing for Conditions Database • Working on masked channels, parallelizing crate configuration

  9. GMT tuning for startup • Accept 2-station  triggers only for  where CSC coverage is missing • Quick check with ~6M min. bias  low lumi. thresh (3-10 GeV) rate increase is small • For high lumi. thresh. (>10 GeV) rate increase large - need full RPC & ME4/2 High lumi. thresholds station4 ME4/2 de-scoped station3 Low lumi. thresholds station2  station1  CSC  coverage

  10. Global Trigger News • Installed a “hot spare” crate (GT2) in USC55 • on top of “production” crate (GT1) • each crate is controlled via CAEN by a separate crate PC • TTC clock/orbit signals fanned out to serve both crates • can quickly switch between crates • can test spare modules or upgrade software without interfering with normal operation of Global Trigger • Software: • Integrated with online configuration database • New database schema including keys, menus, scales • Design finished and documented, implementation under way • XDAQ/Trigger Supervisor monitoring implemented • Panels for monitoring, control, configuration

  11. Trigger Supervisor • Developments: • Provision of progress bar • Improved startup • Provision/integration of TTCci Cell • New Release: • Error handling and tracking • Upgrade to new version of XDAQ • Reporting and monitoring • Handling of alarms • Integration with Configuration & Conditions Databases

  12. L1 emulator: bit-wise reproduction of trigger hardware in CMSSW: Current use cases: Test hardware during commissioning (e.g. pattern tests). Study triggers with MC (used in HLT exercise and CSA07). Future: Run in filter farm at P5 for prompt hardware monitoring during data taking. Stage new trigger algorithms before migration to hardware. Current work: deploy emulator online. Emulator configuration must match hardware. Emulator modules configured from DB via EventSetup. ORCON synchronized with OMDS before each run using O2O. OMDS ORCON ORCOFF EventSetup L1 O2O L1 Emulator L1 Hardware L1 Emulator Configuration L1 Trigger subsystems online XDAQ online CMSSW offline CMSSW Trigger Supervisor

  13. L1 Emulator Development • CMSSW 2_0_0: • New HF-based minbias trigger quantities (HCAL, GCT) • Feature bits (tower over threshold), ring sums • CSC beam halo trigger (CSC TF) • New L1 trigger menus for 1030, 1031 lumi (GT) • HLT seeding possible from ‘technical triggers’ (GT) • eg. RPC cosmic trigger, beam scintillator counters • No technical triggers are emulated yet. • Lightweight interface to trigger results for AOD (GT) • Firmware related updates to CSC TPG, CSC TF, DTTF • First version DB interface

  14. Level 1 Trigger DQM • GRUMM: • L1T sources for almost all trigger systems run online during GRUMM (from two SM event servers). • Created two sets of configurations to run in/out of P5. • Emulator based data validation ongoing • Plans: • Attempt semi-automated procedure to run at CAF, (replica of the online DQM but on all events). • Run L1T DQM at Tier0 and FNAL Tier1. • integrate with offline DQM workflow. • Run Emulator DQM in HLT.

  15. DAQ Capability • 100 kHz EVB (readout) installed • Use of EVB Farm as 25 kHz EVB + limited Filter Farm • With 100 ms/core (@2 GHz on EVB nodes): 14 kHz • Could be factor of 2 off, but present HLT measured 43 ms on 3 GHz • Operational Now • Drop to 7 kHz during May-June (recable) & August (integration of new PCs) • Dedicated Nodes for Filter Farm • Purchase of 900 PCs (dual quad-core @ 2.6 GHz) (720 HLT, 180 DAQ Srvcs) • Capacity 50 kHz (Phase 2 in 2009  100 kHz) • Contingency in present rack space for 112 more PCs • Operational: expect end of summer • Storage: (i.e. HLT accept rate) • Expect about 0.7 kHz peak, assuming 1.5 MB evt size (1 GB/s) • Local Storage: Now: 120 TB • 120 TB = 34 hours at 1 GB/s • Link to Tier-0 (110 Gb/s & one spare -- needs sustained throughput)

  16. Summary of DAQ Schedule

  17. DAQ Commissioning, Testing & Operating Plans • Complete: • Tested Storage Manager • Tested transfers to T0 over CDR links (120 MB/s so far) • Mar (GRUMM): • Technical run: EVB farm (640 PCs) in SCX configured as EVB + limited HLT & transfer to CDR T0 link: chain working • May & June: • Cosmic Run at B=0 • July: • Cosmic Run at B=4T (CRAFT) • Ready for Beam if available

  18. HLT SW Infrastructure • Support for reduced-content streams (for calibration etc.) • Ready and tested (since CMSSW 1_8_0) • Error event stream • Filter Unit side done & Storage Manager side underway • Exception streams • Collect all events generating (non-fatal) exception in at least one path • Underway (not ready for CR0T) • Hard HLT Timeout • Stop processing event in HLT if Tp>Tc • Considered an “extreme” measure to supplement soft timeout • HLT “rate monitor” • Scalers collected from Filter Units & summed up in HLT Scaler Gatherer (HLTSG) stored to online DB per lumi section • Proof-of principle implementation in operation • Prescale Service • Unique table of “working points” agreed between L1 and HLT

  19. HLT Configuration • Offline/development switching to ConfDB • Phase out of CVS menu will be completed with 2_1_0 • Agreed on a scheme of link to production DB & relation to online • Support offline access to HLT menus used for data-taking

  20. Validation Farm: status and plans • Currently: 20 DAQ PC (2x2GHz dual-core, 4GB) • install 2TB disk on hilton01 to serve input data • Playback MC from CASTOR • Playback cosmics from SM • HiLTon overhaul phase 1(June) • Replace current nodes with 25-30 1U PC (same as filter farm phase 1) (move current PCs back in DAQ spare pool) • 0.5-1TB SATA disk each (playback input, test samples) • HiLTon overhaul phase 2 (December) • Duplicate system with second rack • Introduce CDR relay via SM loggers for fast transfer from CASTOR

  21. HLT Online Operations • Gear up for 24/7 operation of HLT in P5 • EvF core team in charge of operations • Adding to experts team • HLT Online Operations Forum to coordinate HLT online decisions with other stakeholders • Checkpoint validation of HLT menus for data-taking • CMSSW version and related issues • Coordinate with offline reconstruction • Calibrations, configurations etc. • Coordinate with tier0/DM/WM • E.g. number/content of online streams etc. • Coordinate with sub-detector communities • Via DPG and GDA contacts

  22. 2008 HLT Status and Outlook • CMSSW_1_8_0: new HLT data model • Started switch of HLT menu development to ConfDB • StorageManager hardware: phased in for first time in GRUMM • GRUMM • Special stream operations (Calibration, Express with specific event content) • CMSSW_2_0_0: April -> for data taking • Remaining pending sw infrastructure features • CCR0T: • First operation with tracker • Tests complete table online • CRAFT: • Integrate all lessons learned for LHC startup • July-August Filter Farm Installation: DAQ @ 36 kHz

  23. HLT Code Integration • Successful transition CVS  ConfDB based HLT-configuration • iCSA08 tables developed in ConfDB • Checked that previously existing table gives identical results in both ConfDB and CVS configurations • Developments guaranteed to work in ConfDB but have kept for this release cycle the old CVS configuration synchronized • Kept HLT table for 1032 as reference • Used ConfDB / GUI features • Renamed all HLT modules/sequences/paths for consistency and to avoid crashed with RECO, etc. in the different workflows • Made consistency checks of usage of products in the HLT paths • Further improvements of configuration • Inclusion of L1 part in the table + consistency checks for seeds already during configuration • Rewriting of the prescale service to add flexibility • Simplification of non-event data part • Changes in configuration of debug information for AOD and RAW • Work with Framework & EvtFilter for optimal way of keeping debug information online • Progress on FastSim-HLT workflow based on ConfDB

  24. Version, Change Control, Review • Each new version of any menu is flagged by its date, e.g. 1E32_27MAR08. • Listed on a twiki page. • We keep a menu history file on another twiki page. • Any time any menu item is changed, an entry would be added of the form: menu version, date, person, change description • Triggers under change control • Changes sent to all affected parties for approval • Initiating rotating review of all triggers • Consultation with PAGs, POGs, DPGs when problems found

  25. Trigger Change Workflow • Goal: Understood, transparent, documented path known to all CMS • Procedure Outline: • A trigger change or introduction of a new trigger must be requested by one of the PO/PA/DP group contacts -- any CMS collaborator can contact one of them to start. • The trigger contact is considered the proponent of the trigger change and is expected to follow through the procedure till the proposal gets approved or rejected. • After notification of TC, DTC, forwards the request to the Trigger Menu development (TMD) and HLT code integration (HCI) groups. • The coordinators of the TMD and HCI groups notify the DTC in case of major problems. Otherwise, the request is forwarded to the Trigger Performance (TP) group. • TP group prepares a “Trigger Performance Impact” statement on the effect of the proposal • TP Coords notify the DTC in case of major problems. Otherwise, the request is forwarded to the Trigger Menu integration (TMI) group. • TMI group provides support for evaluating the performance of the full trigger menu after the proposed changes: total trigger rates and average CPU-processing time. • Results from all of this are presented in an open TSG meeting • DTC & TC Decide on approval/rejection/further consultation with Physics, Trigger Executive Board, etc.

  26. Trigger Coordination Summary • All trigger systems are operating & have been used to trigger CMS • Global Calorimeter trigger is on fast track for full operation • L1 Trigger software is functional and maturing • L1 Emulators are operational • DQM available for almost all trigger systems • 7 kHz DAQ operational now. 50 kHz DAQ operational by Sep. 1 • HLT infrastructure is mature and robust • HLT validation Farm is working • Plan for HLT operations • HLT code is well integrated within CMSSW and CondDB, ConfDB • Versioning, Change Control & Change Workflow being developed • Concerns: ECAL Endcap TPG & RPC Trigger • Next: Trigger Menus (talk by C. Leonidopoulos)

More Related