CS134a: Filesystems and I/O

1. CS134a: Filesystems and I/O • Overall outline • Filesystems • Device Management Computing Systems http://mojave.caltech.edu/cs134/a/ September 20, 2014

2. Input/Output (I0) • Any computer system must be equipped with devices that permit the user to interact with secondary storage units and keep information for indefinite periods of time • Input/Output (IO) processing manages • Input devices: terminal, keyboard, network • Output devices: secondary storage, printers, disk Computing Systems http://mojave.caltech.edu/cs134/a/ September 20, 2014

3. Filesystems • Most systems manage data on secondary storage (like a disk) as a set of files • A file is a collection of data elements grouped together for access control, retrieval, and modification • Each data element is a record • Many modern OS (e.g. Unix) view a file as a sequence or stream of bytes • Files contain programs, application data, file directories, accounting and performance data, etc. • A filesystem is responsible for creating, destroying, organizing, reading, writing, … files Computing Systems http://mojave.caltech.edu/cs134/a/ September 20, 2014

4. Basic issues in device management • The filesystem must: • present a logical view of the files by hiding the details of IO devices • facilitate sharing of IO devices, and optimize their use • provide protection mechanisms for data being managed by the IO devices Computing Systems http://mojave.caltech.edu/cs134/a/ September 20, 2014

5. Logical file view • I/O devices can be very complex • To write a word to a disk: • Move the read/write arm to the track containing the word • Wait until the word rotates under the write head • Check for error conditions (on error, may want to try again) • To write a word to a network • Wait for room in the buffer, then add to the buffer • Network interface executes complex protocol for flow control, retransmission, etc. Computing Systems http://mojave.caltech.edu/cs134/a/ September 20, 2014

6. Device independence • Communication devices are sources and sinks of sequential streams of data—much like a file restricted to sequential access • Generic device interfaces look a lot like the file interface—keeping the same interface for files and devices simplifies the user’s design model • The system may schedule device operations to maximize utility (for example, re-ordering disk operations to minimize head travel) Computing Systems http://mojave.caltech.edu/cs134/a/ September 20, 2014

7. Abstract file interface • Six main operations: open/close, read/write • Open(F, mode): • Open the file named F for the operation specified by mode (read, write, create, …) • F may also specify a device (so open also reserves the device) • Close(F): • Close the previously-opened file • If F is a device, this may also release the device Computing Systems http://mojave.caltech.edu/cs134/a/ September 20, 2014

8. Absolute read/write • Read(F, n, buf): • Copy record n from file F to the buffer • If files are just sequences of bytes, n would be the starting byte number • Write(F, n, buf): • Copy the contents of the buffer into record n of file F • Absolute Read/Write functions may not be available for some devices (such a network interfaces) Computing Systems http://mojave.caltech.edu/cs134/a/ September 20, 2014

9. Sequential read/write • The systems keeps an internal pointer that represents the “current” position in the file • Read(F, buf): • Copy the next record into the buffer • Update the pointer to refer to the next logical record • Write(F, buf): • Copy the buffer to the next record • Update the pointer to refer to the next logical record Computing Systems http://mojave.caltech.edu/cs134/a/ September 20, 2014

10. Unix • int open(char *name, int mode, int perm):open the file with the given name; read/write mode, and default permissions (if the file needs to be created). Returns a file descriptor • void close(int fd):closes the file identified by the file descriptor • int lseek(int fd, int off, int whence):change the position referred to by the file descriptor for the “current” byte Computing Systems http://mojave.caltech.edu/cs134/a/ September 20, 2014

11. Unix read/write • int read(int fd, char *buf, int amount):copy at least one, and no more than amount bytes from the file into the buffer; returns the number copied • int write(int fd, char *buf, int amount):copy at least one, and no more than amount bytes from the buffer into the file; returns the number copied Computing Systems http://mojave.caltech.edu/cs134/a/ September 20, 2014

12. Filesystem layers • L5: Directory retrieval • File names are symbolic • The directory converts symbolic names to identifiers, which point the precise location of the file • L4: Basic file system • Activates and deactivates files by invoking open/close routines • Checks the access rights of the user • Produces a file descriptor for the file when it is opened Computing Systems http://mojave.caltech.edu/cs134/a/ September 20, 2014

13. Layers • L3: Physical organization methods • Translates logical file addresses (for instance, an offset within a file) to physical addresses (like a sector on a disk) • Allocates/releases blocks of secondary storage • L2: Device IO • Requested operations are converted to sequences of IO instructions • Additional buffering schemes • Optimizations (like minimizing rotational delay of a disk) Computing Systems http://mojave.caltech.edu/cs134/a/ September 20, 2014

14. Layers • L1: IO scheduling and control • Queuing, scheduling, initiating, controlling IO requests • Basic IO interrupt servicing • Controller and channel status monitoring Computing Systems http://mojave.caltech.edu/cs134/a/ September 20, 2014

15. Unix example • To open a file: FILE *fp = fopen(name, mode); • Locates the file called name and saves its status in the systems tables • Unix maintains two open file tables (OFT): • The per-user OFT (the stdio buffers) contain file descriptors (an int) • The file descriptor is an index into the system-wide table, which contains indexes into the inode table • An inode contains a file’s physical location info Computing Systems http://mojave.caltech.edu/cs134/a/ September 20, 2014

16. File system tables in Unix Computing Systems http://mojave.caltech.edu/cs134/a/ September 20, 2014

17. Levels L5 and L4 Files are identified by inode The inode contains a table of the file’s disk blocks To find and open a file: Search the directory for the inode-number Allocate an entry in the system OFT (identified by a “file descriptor” fd) Initialize stdio FILE struct with fd Return a file-pointer Unix open Computing Systems http://mojave.caltech.edu/cs134/a/ September 20, 2014

18. (Get a character from the file) Allocate the FILE buffer (level L3) Fill the buffer using the syscall:read(fd, buf, n) translates the file location to a physical address (using the inode) transform read request into a packet (levels L3, L2), and queue the packet on the device Device driver (level L1) eventually executes the packet Packets may be reordered in the queue Unix getc(fp) Computing Systems http://mojave.caltech.edu/cs134/a/ September 20, 2014

19. Directory management Basic file system Physical organization IO subsystem Outline for the next lectures Computing Systems http://mojave.caltech.edu/cs134/a/ September 20, 2014

20. Directories identify files that are accessible to processes A directory entry contains a file name and either the address of the file or its descriptor (inode) the entire descriptor Directories are commonly treated as special files Directory management Computing Systems http://mojave.caltech.edu/cs134/a/ September 20, 2014

21. There is a root directory Branches point to either directories or files A pathname is the concatenation of filenames along a path Directory tree structure Computing Systems http://mojave.caltech.edu/cs134/a/ September 20, 2014

22. Directory tree Computing Systems http://mojave.caltech.edu/cs134/a/ September 20, 2014

23. To shorten path names, each process has a working directory Links can be used to form arbitrary graphs A link points to a directory entry When a file is deleted, the link may become invalid In Unix A symbolic link refers to a directory entry A hard link is formed when two directory entries refer to the same inode Directory tree Computing Systems http://mojave.caltech.edu/cs134/a/ September 20, 2014

24. Each file has a file descriptor (in Unix, an inode) that describes the file info Often, the file descriptor is saved in a separate area of the storage device The file info contains: File identification: the name or number of the file Physical address information: location, extent, and storage blocks for the file Access control information:who can access the file Other: dates, number of links, physical organization, etc. Basic Filesystem Computing Systems http://mojave.caltech.edu/cs134/a/ September 20, 2014

25. For files that already exist Locate file and make it available Check that the user has the authority to access the file For new files, allocate storage and device for the file Allocate memory needed for IO buffers Load routines for accessing the file (as well as any memory-map info) Possibly generate IO command skeletons Insert an entry into the OFTs Basic filesystem: open routine Computing Systems http://mojave.caltech.edu/cs134/a/ September 20, 2014

26. Delete the file if it is temporary Add the entry into the directory if the file is permanent For permanent files: Update the descriptor (inode) with accounting information Possibly add EOF markers at the end of the file For all files: Possibly rewind the file (if it is a tape drive) Remove the entry in the OFTs Release buffer space and IO device Basic filesystem: close routine Computing Systems http://mojave.caltech.edu/cs134/a/ September 20, 2014