Chapter 7 UNIX and LINUX

1. Chapter 7UNIX and LINUX

2. Outline • Overview • Processes in UNIX • Memory management in UNIX • I/O in UNIX • UNIX file system

3. Overview • Why some programmers like UNIX better than Windows? • Although GUI may be easy for beginners, they provide little flexibility and no insight into how the system works! • UNIX has GUI, too! • History of UNIX • UNICS in Bell Labs • PDP-11 UNIX • Portable UNIX, portable C compiler • Berkeley UNIX (BSD) • UNIX standards: POSIX from IEEE • MINIX and Linux

4. UNIX Goals (Philosophy) • Designed by programmers, for programmers • Simple, elegant and consistent • Power and flexibility • A small number of basic elements • One program should do just one thing • An infinite combination to suit the applications • No useless redundancy • Simple interface

5. Interfaces to UNIX User interface Library interface Users Standards utility programs (shell, editors, compilers, etc) System call interface Standard library (e.g. open, close, read, write, fork, etc) User mode UNIX operating system (process management, memory management, the file system, I/O, etc) Kernel mode Hardware (CPU, memory, disks, terminals, etc)

6. The UNIX Shell: A Command Line Interface • Wait for a command line • Extract the first word from the command line, use it as program name, run the program • Pass arguments • Flags: argument controlling the operation or specify an optional value • Wild cards and magic characters • Standard input/output • Filter and pipe symbol • grep ter *.t | sort | head –20 | tail –5 > foo • Background execution and shell scripts

7. UNIX Utility Programs • File and directory manipulation commands: cp, mv • Filters: grep, head, sort, tail • Program development tools: cc • Text processing: vi, edit • System administration: ps • Miscellaneous: time

8. Structure of UNIX Kernel

9. Outline • Overview • Processes in UNIX • Memory management in UNIX • I/O in UNIX • UNIX file system

10. Basic Concepts • UNIX is a multiprogramming system • User processes • Daemons • Create new processes by system call fork() • Parent process and child process • PID, parent process gets child PID from fork() • Pipes between two processes • Shell pipelines are implemented using pipes • Signals: soft interrupt

11. Process Management System Calls in UNIX

12. Thread Management • Not every UNIX has threads package • Threads package becomes popular • Threads can be implemented in either user space or kernel • POSIX does not specify

13. Thread Management System Calls

14. Implementation of Processes • Two key data structures for processes • Process table: always in main memory • Scheduling parameters, memory image, signals, miscellaneous • A process has one entry in the table • User structure: in memory only when the process is in memory • Machine registers, system call state, file descriptor table, accounting info, kernel stack • A block per process, adjacent to the stack segment

15. Executing “ls” Typed to the Shell PID=501 PID=748 PID=748 sh sh sh New process Same process 3. Exec call 1. Fork call 4. Sh overlaid with ls 2. New sh created Fork code Exec code • Allocate child’s process table entry • Fill child’s entry from parent • Allocate child’s stack and user area • Fill child’s user area from parent • Allocate PID for child • Set up child to share parent’s text • Copy page tables for data and stack • Set up sharing of open files • Copy parent’s registers to child • Find the executable program • Verify the execute permission • Read and verify the header • Copy arguments, environ to kernel • Free the old address space • Allocate new address space • Copy arguments, environ to stack • Reset signals • Initialize registers

16. Threads in UNIX • Threads in user space: a user space library • Threads in kernel: may lead to many problems • How to maintain the correct traditional UNIX semantics? • File I/O • Signal handling • All solutions to these problems cause something to break somewhere

17. Threads in Linux • System call clone • pid=clone(func, stack_ptr, sharing_flgs, arg) • Sharing_flgs determine whether the thread is in current process or in a new process • In the same process: changes visible to other threads • The new thread executes func • The new thread has its own stack

18. Scheduling in UNIX • A two level algorithm • Low-level algorithm: pick the process to run • High-level algorithm: move processes between memory and disk • Low-level algorithm uses multiple queues • Priority=CPU_usage+nice+base

19. Multilevel Queue Structure

20. Scheduling in Linux • Scheduling is based on threads • Three classes of threads • Real-item FIFO, non-preemptive, highest priority • Real-time round robin, preemptive, high priority • Timesharing • Scheduling based on priority and quantum

21. Booting UNIX • Read & run first sector of the boot disk • MBR has a small program (up to 512 bytes) • Load program boot • Boot reads the root directory of boot device • Boot reads OS kernel, boot ends • Kernel starts, set parameters & environment • Initialization, build kernel data structure • System auto-configuration, detect devices • Load device drivers • Start process 0

22. Process 0 • Continue initialization • Program the real-time clock • Mount the root file system • Create init (process 1) and page daemon (process 2)

23. Process 1 • Check flags of single-user/multi-user mode • Single user mode • Fork off a process running shell • Wait for that process to exit • Multi-user mode • Fork off a process • Run system initialization shell script /etc/rc • Read /etc/ttys, fork off processes for terminals • Run gtty

24. Sequence of Processes in Booting Process 0 Page daemon Process 1 init Process 2 Terminal 1 Terminal 2 Terminal 0 sh login getty Login: Password: % cp f1 f2 cp

25. Loading Drivers • Dynamically loading • One binary code can be used everywhere • Drivers are loaded dynamically, even through a network • Non-dynamically loading • Only the system administrator can make kernel binary code • No one can insert any component into the kernel

26. Outline • Overview • Processes in UNIX • Memory management in UNIX • I/O in UNIX • UNIX file system

27. Three Segments in UNIX Processes • Text segment: program’s executable code • Read-only, size fixed after process is created • Data segment: program’s variables, strings, and other data • Initialized data and un-initialized data (BSS) • Un-initialized global variables are in BSS, save space • Size can change • Stack segment

28. Sharing Between Processes • Text segment & mapped file can be shared

29. Memory Management System Calls • POSIX does not specify • Common system calls

30. Swapping • High level scheduling: swapper • What cause swapping? • Fork() needs memory for a child process • Brk() needs to expand a data segment • Stack needs to be expanded • Which process will be swapped out? • The ones blocked/waiting for I/O • The ones with high (priority + residence time) • The ones consuming much CPU/staying long in mem

31. Swapping Back Processes • Check processes on disk every few seconds • Find processes are ready • Select the one staying on disk longest • If have enough free memory (easy swap), bring that process in • Otherwise, swap out some processes in main memory • No process is swapped out until it stays in memory for 2 seconds

32. Paging in UNIX • Do not use the working set model • Done by kernel & page daemon (process 2) • Never page out kernel and core map • Core map records info about contents of the page frames

33. Page Replacement Algorithm • Run every 250 msec by page daemon • If insufficient free page frames, page daemon transfers pages to disk • A modified version of the clock algorithm

34. Memory Management in Linux • A three-level paging scheme • Directory, middle, page + offset • Buddy algorithm

35. Outline • Overview • Processes in UNIX • Memory management in UNIX • I/O in UNIX • UNIX file system

36. Input/Output in UNIX • Integrate devices into file system as special files • Each I/O device is assigned a path name • Block special files and character special files

37. Networking and Sockets • Socket: interface to network • Types of networking supported by sockets • Reliable connection-oriented byte stream • Reliable connection-oriented packet stream • Unreliable packet transmission Receiving process Sending process User space kernel space Socket

38. I/O System Calls in UNIX • POSIX specifies function calls for terminal • Function call ioctl is used in many UNIX

39. Outline • Overview • Processes in UNIX • Memory management in UNIX • I/O in UNIX • UNIX file system

40. UNIX File • A sequence of bytes • No distinction between ASCII, binary, or any other kinds of files • Directories are stored as files • Important directories in UNIX systems

41. Mounting and Locking • Only one file system (tree) • One disk is mounted in another disk’s file tree • UNIX allows user to lock files • From single byte to an entire file • Shared locks and exclusive locks • A process specifies whether it wants to block if the lock cannot be placed • Shared locked regions may overlap

42. System Calls About Files

43. System Calls About Directories

44. Disk Layout of Classical UNIX • Boot block: contain code to boot computer • Superblock: contain critical info about the layout of the file system • # of i-nodes, # of disk blocks, the start of the list of free disk blocks, … • I-nodes • Data blocks

45. Reading A File • System call n=read(fd, buffer, nbytes) • Start with file descriptor fd, find i-node • Put pointer to the i-node in file descriptor? • Only one current position for one file, recorded in i-node • Two processes open one file with different current position • Put current position info in file descriptor? • One file can have multiple current positions • P1 and P2 open f at the same time, P2 want to write after P1 finishes • P2 cannot get the correct current position

46. Open File description Table

47. Berkeley Fast File System • Allow file names up to 255 characters • Divide the disk up into cylinder groups • Each with its own superblock, i-nodes and data blocks • Whenever possible, blocks are allocated in the cylinder group containing the i-node • Two block sizes • Large files use large block size

48. Summary • Three interfaces to UNIX: • Shell, C library, and system calls • Key concepts in UNIX • Process, memory model, I/O, and file system • Process management in UNIX • Process table and user structure • Memory model: text, data and stack • I/O and file system