1 / 7

Update on ALICE software status and ideas

Update on ALICE software status and ideas. Ervin D énes Wigner Research Center Hungarian Academy of Sciences. How Does the RORC SW Work?. We have DMA engines in the RORC cards (FW)that take data over optical links ( DDL ) to store it in RAM. . PC. Pci. Detector. RORC. DIU. SIU.

torin
Download Presentation

Update on ALICE software status and ideas

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. Update on ALICE software status and ideas Ervin Dénes Wigner Research Center Hungarian Academy of Sciences

  2. How Does the RORC SW Work? • We have DMA engines in the RORC cards (FW)that take data over optical links (DDL) to store it in RAM. PC Pci Detector RORC DIU SIU DDL • We need a PCI driver for the RORC. • The driver maps the RORC’s registers into memory, so that the SW can read/write • The SW can give the addresses where to put the collected data and can assure the RORC always has addresses of free memory. Cross LHC exp. DAQ meeting, 9 July 2014

  3. Physmem • To support the DDL transfers we need a RAM area which is pinned (non-swapped), with known physical addresses, and which is accessible from user-level code. • For that at boot time we limit the amount of memory used by Linux (grub parameter mem=...) e.g.: • kernel /vmlinuz-2.6.32-358.23.2.el6.x86_64 ….. nopatmem=4G ………… Physmem area 0 Physmem area #n Linux ~2,5G 4G RAM • We have a small device driver that maps the memory outside the Linux area via remap_pfn_range() • The driver gives to the user SW the size of the mapped area and its physical address Cross LHC exp. DAQ meeting, 9 July 2014

  4. Evolution of Physmem Driver • Aroundyear 2000 we started with Bigphys • Part of the Linux kernel • Not supported by IT --> for every new kernel version we had to recompile the kernel • First Physmem from 2002 • Kernel modul: no need of kernel compilation • Modifications for SLC3 and SLC4 (2006, 2008) • Compatible 32 and 64 bit versions for SLC5 (2008) • Solved to skip the “hole” under 4G and 4G crosses • Modifications for SLC6 (2012) • /proc/iomem changed, does not report above boot parameter -> we use BIOS map • Kernel 2.6.32 uses new memory management • using PAT (Page Attribute Table) which is not correct for mapping big memory (> 3.8 GB): read and write speeds fall drastically. Solution: “nopat” in GRUB (2013) • Modifications for Ubuntu (for C-RORC tests) (2013) 1 physmem area Cross LHC exp. DAQ meeting, 9 July 2014

  5. Evolution of RORC Driver • First driver for PCI32 RORC: 2001 • 2001 – 2012: • following the evolution of SLC2, 3, 4, 5, 6 kernels • following the evolution of RORC HW: • 66MHz PCI, 100 MHz PC-X, PCI-Express Gen2 • 2013: • Common driver for D-RORC and C-RORC • modifications for Ubuntu • Ubuntu is needed for HW tests of C-RORC only • RUN 3: • Commercial 10/40 GbE or PCIe over fibre HW come with their own drivers • Performance should be checked Cross LHC exp. DAQ meeting, 9 July 2014

  6. How to follow? • Should we follow with Physmem? • Pros: • no data move between kernel and user space • no limit of memory size • several process can reach data in the same time • any part of the memory can be selected • Cons: • Linux developers do not support • What else could be used? • Scatter-Gather DMA is one option • Pros: • No need to develop memory management driver • Cons: • Not known if sgDMAhas all features what Physmem has e.g. use this Cross LHC exp. DAQ meeting, 9 July 2014

  7. Next steps in common • There is some opportunity for • Exchange of information • Common development Cross LHC exp. DAQ meeting, 9 July 2014

More Related