1 / 58

CHAPTER 12: FILE SYSTEM IMPLEMENTATION

CHAPTER 12: FILE SYSTEM IMPLEMENTATION. File System Structure File System Implementation Directory Implementation Allocation Methods Free-Space Management Efficiency and Performance Recovery Log-Structured File Systems NFS. FILE-SYSTEM STRUCTURE.

caia
Download Presentation

CHAPTER 12: FILE SYSTEM IMPLEMENTATION

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. CHAPTER 12: FILE SYSTEM IMPLEMENTATION • File System Structure • File System Implementation • Directory Implementation • Allocation Methods • Free-Space Management • Efficiency and Performance • Recovery • Log-Structured File Systems • NFS

  2. FILE-SYSTEM STRUCTURE • Disks provide the bulk of secondary storage on which a file system is maintained • can be written in place • Sequential access and direct access • I/O in blocks not in bytes • File system allows the data on disks to be stored, located, and retrieved easily • To define how the FS should look to the user • To create algorithms and data structures to map the logical FS onto the physical secondary-storage devices.

  3. File-System Structure: Layered file system

  4. File-System Structure: Layered file system • I/O control • consists of device drivers and interrupt handlers to transfer information between the main memory and the disk system. • Input: retrieve drive 1, cylinder 73, track 2, sector 10 • Output: low-level, hardware-specific instructions that are used by the hardware controller • Basic file system • issues generic commands to the appropriate device driver to read and write physical blocks on the disk • Input: retrieve block 123 • Output: retrieve drive 1, cylinder 73, track 2, sector 10

  5. File-System Structure: Layered file system • The file-organization module • knows about the files and their logical blocks, as well as physical blocks. • To translate a file’s logical block addresses to its physical block addresses. • Each file’s logical block addresses are numbered from 0 (or 1) through N. • Each file’s physical block addresses are different, are unique within a partition. • Free-space manager: • Tracks unallocated blocks • And provides these blocks when requested.

  6. File-System Structure: Layered file system • The logical file system • manages metadata information (metadata v.s. actual data) • To manage the directory structure • To manage the file structures via FCB (file control blocks) • Contains information such as ownership, permissions, location of the file contents • Responsible for protection and security.

  7. File-System Structure: Example FSes • Windows: • FAT (File Allocation Table) (12, 16, 32) • NTFS (Windows NT File System) • UNIX: • UFS (Unix File system)  ext2  ext3 • more /proc/filesystems • cd /usr/src/linux-2.4/fs # to explore for more.

  8. FILE-SYSTEM IMPLEMENTATION • Overview: Data structures for implementing a FS • On-disk structures • In-memory structures • Partitions and mounting • VFS

  9. File-System Implementation: Overview • On-Disk structures • Boot control block (boot block, partition boot sector) • Used to boot an OS from that partition. • Partition control block (super block, Master File Table) • Block numbers, block size, free-block count, free-block pointers, free FCB count and FCB pointers • Directory structure used to organized the files • Linear list / hash tables • FCB (inode, vnode, Master File Table record) • File permissions, ownership, size, location of the data blocks

  10. File-System Implementation: Overview • In-memory structures used for both file-system management and performance improvement via caching • In-memory partition table containing information about each mounted partition. • In-memory directory structures containing the directory information of recently accessed directories. • System-wide open-file table containing a copy of the FCB of each open file as well as other information. • Per-process open file-tables containing a pointer to the appropriate entry in the system-wide open-file table, as well as other information.

  11. File-System Implementation: Overview • To create a new file • An application program calls the logical file system • The logical file system • To allocate a new FCB (see the next slide for FCB) • To read in the appropriate directory • UNIX treats a directory exactly as a file. • Windows NT treats a directory as a record inside the MFT. • To add a new entry • To fill in the new entry with the filename and the new FCB • To write it back to the disk

  12. File-System Implementation: Overview: FCB

  13. File-System Implementation: Overview • To open a file • To pass a file name to the logical file system • To search the directory for the given file name • To read in the file’s FCB • To put the file’s FCB to the system-wide open-file table • To add a new entry in the per-process open-file table, with a pointer to the entry in the system-wide open-table and some other fields • To return a pointer to the appropriate entry in the per-process file system table. (file descriptor or file handle)

  14. File-System Implementation: Overview • To close a file • To remove the entry in the per-process open-file table • To decrement the system-wide open-file entry’s open count. • If open count is 0, copy the updated file information to the disk-based directory structure and delete this entry.

  15. File-System Implementation: Overview

  16. File-System Implementation: Overview • To use the file system as interface to other system aspects, such as networking. • To use caches to speed up the file operations • BSD UNIX • To use the file system for other purpose such as networking interface.

  17. File-System Implementation: partitions and mounting • Partitions vs disks • A disk can be sliced into multiple partitions • A partition can span multiple disks • Partitions can either be “raw” or “cooked” • Raw partition: • No file system • swap space, database • Cooked partition: • Has file system • Boot block can be used for selective booting. • Super block

  18. File-System Implementation:partitions and mounting • To mount a partition before using its FS • Manual mounting vs automatic mounting • How to mount a partition • To read in the super block via its device driver • To verify its consistency • To repair it if necessary (fsck) • To add an entry in the in-memory mount table structure. • How to mount a partition for Windows • To mount at boot • To mount manually

  19. File-System Implementation:partitions and mounting • How to mount a partition for UNIX • To mount a partition at a directory • To add a entry at the mount-table • To let one field of the mount-table entry point to the super block of the FS on that device • To set a flag in the in-memory copy of the inode for that directory, indicating this directory is a mount point • To set a field in the in-memory copy of the inode for that directory point to an entry in the mount table, indicating which device is mounted there.

  20. File-System Implementation: VFS • How to support multiple FSes？ • How to integrate many Fses into a directory structure? • How to seamlessly move among various FSes?  • To write directory and file routines for each types. • To use VFS (VFS uses oo techniques to simplify, organize, and modularize the implementation) • Contributed by SUN Microsystems.

  21. File-System Implementation: VFS • Top layer: file-system interface • Open, read, write, and close and file descriptors • The middle layer: VFS • To separate FS generic operations from their implementation by defining a clean VFS interface • The VFS is based on a file-representation, called a vnode, that contains a numerical designator for a network-wide unique file. (UNIX inodes are unique within only a single file system) • The bottom layer • Various FS implementation • Ext3 • NFS

  22. File-System Implementation: VFS

  23. DIRECTORY IMPLEMENTATION Linear list • To use a linear list of file names with pointers to the data blocks. • To find a file • To require a linear search. • To create a file • To search the directory to make sure no existing file has the same file name. • To add a new entry at the end of the directory. • To delete a file • To search the directory for the file. • To remove the entry. • To free the space allocated to this file.

  24. Directory Implementation Linear list • To reuse a directory entry • To mark the entry as used. • To attach it to a list of free directory entries. • To decrease the length of the directory. • Discussion: • Simple to program, time-consuming to execute • The searching is expensive • Binary search • To keep it sorted list • To use B-tree

  25. Directory Implementation Hash Table • To use a linear list to store the directory entries and to use a hash table to quickly find out the directory entry given a file name. • No collisions allowed (Each hash entry has a single value) • Use a hash function to map file name to a hash value. • Hash function should be dynamically changed. • Fastest. • Collisions allowed (Each hash entry has a list of multiple values) • Use a hash function to map file name to a hash value and use this value to index the hash table and then search the list to find out the directory entry. • Faster.

  26. ALLOCATION METHODS • An allocation method refers to how disk blocks are allocated for files: • Contiguous allocation • Linked allocation • Indexed allocation

  27. Allocation Methods: Contiguous Allocation • The contiguous-allocation methods requires each file to occupy a set of contiguous blocks on the disk • The directory for a file consists only its starting location (block #) and length (number of blocks) (See the next slide) • Discussions: • Supports both sequential access and direct access. • Simple to implement. • External fragmentation (dynamic storage-allocation problem). • How to specify an initial size for a file • Under estimating its size • Over estimating its size

  28. Allocation Methods: Contiguous Allocation

  29. Allocation Methods: Contiguous Allocation • Extent-based file system • Extent-based file systems allocate disk blocks in extents. • An extent is a contiguous block of disks. Extents are allocated for file allocation. A file consists of one or more extents. • Many newer file systems (I.e. Veritas File System) use a modified contiguous allocation scheme. • Contiguous allocation can be combined with other allocation methods. • Contiguous allocation for small files • Other allocations for large files.

  30. block = pointer Allocation Methods: Linked Allocation • With linked allocation, • Each file is a linked list of disk blocks; • The disk blocks may be scattered anywhere on the disk. • The directory contains a pointer to the first and last blocks of the file. (See the next slide) • Disk block • An example linked file (see the next slide)

  31. Allocation Methods: Linked Allocation

  32. Allocation Methods: Linked Allocation • Simple – need only starting address • Free-space management system – no waste of space • No random access • Pointers waste space: to use clusters rather than sectors • To improve usage • To speed up • Poor reliability • Imagine the pointer is messed up.

  33. Allocation Methods: Linked Allocation • FAT FS • FAT (File Allocation Table) duplicated • Supports direct access • Cached • Poor disk utilization

  34. Allocation Methods: Linked Allocation

  35. Allocation Methods: Indexed Allocation • Problems: • External fragmentation and size-declaration for contiguous allocation • Direct access for linked allocation • Indexed allocation • Bringing all the pointers together into one location: index block (See the next slide) • Logical view index table

  36. Allocation Methods: Indexed Allocation

  37. Allocation Methods: Indexed Allocation • Need index block • Access methods: sequential access and direct access. • Mapping from logical to physical in a file of maximum size of 256K words (or 1024KB) and block size of 512 words. We need only 1 block for index table. (512x2KB=1024KB) • Index blocks waste space • How large should the index block be? • Linked scheme • Multilevel index • Combined scheme

  38. Allocation Methods: Indexed Allocation • Linked scheme • For a small file, one index block • For a large file, more index blocks can be linked together. • Multilevel index • 1-level index block: 1024*4KB • 2-level index block: 1024x1024*4KB • 3-level index block: 1024x1024*1024*4KB • (similar to paging)

  39. Allocation Methods: Indexed Allocation  outer-index file index table

  40. Allocation Methods: Indexed Allocation • Combined scheme • 12 for direct pointers • 1 single-indirect block • 1 double-indirect block • 1 triple-indirect block

  41. Allocation Methods: Indexed Allocation

  42. Allocation Methods: Performance • Two criteria: • Storage utilization efficiency • Data block access time • Contiguous allocation: Good for known-size file • Linked allocation: Good for storage utilization • Indexed allocation: Access time depends on index structure, file size, block position • Conclusion: • Combining contiguous allocation and linked allocation (Some OS) • Combining contiguous allocation and index allocation (SUN)

  43. FREE-SPACE MANAGEMENT • Bit vector • Linked Lists • Grouping • Counting

  44. Free-Space Management: Bit vector • The free-space list is implemented as a bit map or bit vector. Each block is represented by 1 bit. • If the block is free, the bit is 1; • if the block is allocated, the bit is 0. • An example • 0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 • 0 0 1 1 1 1 0 0 1 1 1 1 1 1 0 0

  45. Free-Space Management: Bit vector • Discussion: • Simple to implement • Efficient to find the first free block • Easy to get contiguous files • Block number calculation: (number of bits per word) *(number of 0-value words) +offset of first 1 bit • Bit map requires extra space. Example: block size = 212 bytes disk size = 230 bytes (1 gigabyte) n = 230/212 = 218 bits (or 32K bytes)

  46. Free-Space Management: Linked Lists • Linked list (free list) • Cannot get contiguous space easily • No waste of space

  47. Free-Space Management: Others • Grouping • To linked blocks to store the addresses • To group n-1 addresses into an address block • To use the last address in an address block to point to next address block. • Easier to find a large number of free blocks. • Counting • Every entry is a pair of (starting address, contiguous block number) rather than just a address • The total list is smaller.

  48. EFFICIENCY AND PERFORMANCE • Efficiency • Preallocating the i-node on a partition (UNIX FS) • Different cluster size (BSD UNIX) • File size • To fix the parameters or dynamically change the parameters.

  49. Efficiency and Performance • Page Cache • I/O Without a Unified Buffer Cache • I/O Using a Unified Buffer Cache • Reads/Writes • Synchronous reads (initially) /Aynchronous reads (later on) • Aynchronous writes (normally) / Synchronous write/Aynchronous (sometimes) • RAM Disks vs OS caching.

  50. Efficiency and PerformanceI/O Without a Unified Buffer Cache

More Related