Our ‘nic.c’ module

1. Our ‘nic.c’ module We create a ‘character-mode’ device-driver for the 82573L NIC to use in future experiments

2. No way to communicate SOFTWARE Linux operating system kernel (no knowledge of the NIC) These components lack a way to interact HARDWARE Network interface controller (no knowledge of the OS)

3. Role for a ‘device driver’ SOFTWARE Linux operating system kernel (no knowledge of the NIC) device driver module (knows about both the OS and the NIC) HARDWARE Network interface controller (no knowledge of the OS)

4. Character devices • The concept of a ‘character mode’ device is that it provides a ‘stream’ of data bytes that an application-programs can access by using standard Linux system-calls: • ‘open()’ • ‘write()’ • ‘read()’ • ‘close()’ • Maybe other ‘methods’ (depending on the device)

5. Requirement • The character-mode devices are known to a Linux system by their ‘names’ and by an associated pair of ID-numbers (major and minor) normally set up by a ‘superuser’ root# mknod /dev/nic c 97 0 root# chmod a+rw /dev/nic

6. Driver module nic.c my_proc_rx() this module’s collection of ‘payload’ functions my_proc_tx() isr() read() write() character-mode device-driver’s required ‘file-operations’ data-structure fops required pair of module administration functions module_init module_exit

7. Overview user space kernel space Linux operating system Kernel standard runtime libraries file subsystem networking subsystem We bypass use of the TCP/IP protocol stack for our demo char driver module application program hardware

8. Transmit operation user space kernel space Linux OS kernel runtime library file subsystem nic device-driver write() my_write() application program packet buffer user data-buffer copy_from_user() DMA hardware

9. Receive operation user space kernel space Linux OS kernel runtime library file subsystem nic device-driver read() my_read() application program packet buffer user data-buffer copy_to_user() DMA hardware

10. We ‘tweak’ our packet-format • Our ‘nictx.c’ and ‘nicrx.c’ modules elected to use the Ethernet-header’s Type/Length field for storing the number of data-bytes • But this allows our driver to be misled by packet’s which don’t follow that scheme • So we added a ‘signature’ and relocated our ‘count’ field to the ‘payload’ region 0 6 12 14 actual bytes of user-data destination MAC-address source MAC-address Type/Len count

11. Using ‘echo’ and ‘cat’ • Our device-driver module ‘nic.c’ is intended to allow two programs that are running on a pair of our classroom PCs to communicate via their ‘secondary’ Ethernet interfaces Transmitting… Receiving… $ echo Hello > /dev/nic $ _ $ cat /dev/nic Hello _

12. Exploring RX Descriptor FIFO Timeout for an in-class demonstration

13. ‘push’ versus ‘pull’ Host memory Ethernet controller transmit packet buffer transmit-FIFO push to/from LAN receive packet buffer receive-FIFO pull The ‘write()’ routine in our ‘nic.c’ driver can transfer some data anytime there is at least one Tx-descriptor that is not “owned” by the network hardware, but the ‘read()’ routine in our ‘nic.c’ driver will have to wait for some data to arrive. So to avoid doing any wasteful busy-waiting, our ‘nic.c’ driver can utilize the Linux kernel’s sleep/wakeup mechanism – if it enables the NIC’s interrupts!

14. Sleep/wakeup • We will need to employ a wait-queue, we will need to enable device-interrupts, and we will need to write and install the code for an interrupt service routine (ISR) • So our ‘nic.c’ device-driver will have a few additional code and data components that were absent from our ‘nictx.c’ and ‘nicrx.c’ kernel-module demos

15. How NIC’s interrupts work • There are four interrupt-related registers which are essential for us to understand ICR Interrupt Cause Read 0x00C0 0x00C8 0x00D0 0x00D8 ICS Interrupt Cause Set IMS Interrupt Mask Set/Read IMC Interrupt Mask Clear

16. Interrupt-event types 31 30 18 17 16 15 14 10 9 8 7 6 5 4 2 1 0 reserved reserved 31: INT_ASSERTED (1=yes,0=no) 17: ACK (Rx-ACK Frame detected) 16: SRPD (Small Rx-Packet detected) 15: TXD_LOW (Tx-Descr Low Thresh hit) 9: MDAC (MDI/O Access Completed) 7: RXT0 ( Receiver Timer expired) 6: RXO (Receiver Overrun) 4: RXDMT0 (Rx-Desc Min Thresh hit) 2: LSC (Link Status Change) 1: TXQE( Transmit Queue Empty) 0: TXDW (Transmit Descriptor Written Back) 82573L

17. Interrupt Mask Set/Read • This register is used to enable a selection of the device’s interrupts which the driver will be prepared to recognize and handle • A particular interrupt becomes ‘enabled’ if software writes a ‘1’ to the corresponding bit of this Interrupt Mask Set register • Writing ‘0’ to any register-bit has no effect, so interrupts can be enabled one-at-a-time

18. Interrupt Mask Clear • Your driver can discover which interrupts have been enabled by reading IMS – but your driver cannot ‘disable’ any interrupts by writing to that register • Instead a specific interrupt can be disabled by writing a ‘1’ to the corresponding bit in the Interrupt Mask Clear register • Writing ‘0’ to a register-bit has no effect on the interrupt controller’s Interrupt Mask

19. Interrupt Cause Read • When interrupts occur, your driver’s ‘interrupt service routine’ (isr) discovers which specific conditions triggered the interruption if it reads the NIC’s Interrupt Cause Read register • In this case your driver can clear any selection of these bits (except bit #31) by writing ‘1’s to them (writing ‘0’s to this register will have no effect) • If case no interrupt has occurred, reading this register may have the side-effect of clearing it

20. Interrupt Cause Set • For testing your driver’s interrupt-handler, you can artificially trigger any particular combination of interrupts by writing ‘1’s into the corresponding register-bits of this Interrupt Cause Set register (assuming your combination of bits corresponds to interrupts that are ‘enabled’ by ‘1’s being present for them in the Interrupt Mask)

21. Our interrupt-handling • We decided to enable only a subset of the possible interrupt causes (those related to transmits, receives, and link-changes): • The ones we take action on are TXDW, RXT0, and RXDMT0 #define INTR_MASK (1<<0)|(1<<1)|(1<<15)|(1<<2)|(1<<4)|(1<<6)|(1<<7) transmit-related receive-related link-status changed

22. NIC “owns” some Rx-descriptors RDBAH/RDBAL RDH 0 1 2 3 4 5 6 7 descriptor 0 This register gets initialized to 0, then gets changed by the controller as new packets are received descriptor 1 descriptor 2 descriptor 3 RDLEN descriptor 4 RDT =0x80 descriptor 5 This register gets advanced by 4 when the NIC’s supply of descriptors drops below a programmed threshold descriptor 6 descriptor 7 Our ‘static’ variables rxhead Keeps track of where packet-data resumes pickup

23. In-class exercise • Login to a pair of classroom machines • Install our ‘nic.ko’ module on both hosts • Try transferring a textfile from one of the machines to the other, by using ‘cat’: hrn23501$ cat textfile > /dev/nic hrn23502$ cat /dev/nic > recv1000.out • How large a textfile can you successfully transfer using our Linux device-driver? • Any problems encountered doing this?