1 / 25

Operating Systems CMPSCI 377 Lecture 23: Advanced File Systems

Operating Systems CMPSCI 377 Lecture 23: Advanced File Systems. Emery Berger University of Massachusetts Amherst. Advanced File Systems. Motivation & Brief review Journaling Log-structured file system. New Challenges. Servers have special needs! Large hard-disk partitions

alize
Download Presentation

Operating Systems CMPSCI 377 Lecture 23: Advanced File Systems

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript


  1. Operating SystemsCMPSCI 377Lecture 23: Advanced File Systems Emery Berger University of Massachusetts Amherst

  2. Advanced File Systems • Motivation & Brief review • Journaling • Log-structured file system

  3. New Challenges • Servers have special needs! • Large hard-disk partitions • Quick crash recovery • High-performance I/O • Storing thousands of files, TB of data • None supported well by ext2

  4. Review: Blocks & Fragmentation • Logical block – smallest unit of storage that can be allocated by file system • Internal fragmentation – occurs when file does not fill block completely • Example: file = 10K, block = 8K: wastes 6K • External fragmentation – occurs when logical blocks that make up file are scattered over the disk • Causes poor performance

  5. Review: Organization • Extent – contiguous blocks: • Triple: (file offset, starting block number, length) • file offset – offset of extent's first block from file start • starting block number – first block in extent • length –number of blocks in extent • Meta data – file system's internal data structures • Time stamps, ownership, size & locations on disk, etc.

  6. Extents • Triple:(offset, start, length) • Contiguous blocks

  7. Review: Inodes • inode – stores information about file • e.g., permissions, types, # of links to file • pointers to file data blocks (direct pointers) • pointers to direct pointers (indirect pointers) • each has unique inode number • directory = special file: contains pointers to other files

  8. Metadata Updates • Lots of meta data being updated • Scattered on disk = slow, and non-atomic • Example: Creating a new file • Modifies inodes, free lists, directory entries • What happens if interrupted? • e.g., power outage, crash • Interrupted meta-data write) file system in inconsistent state

  9. Repairing File System Inconsistency • UNIX solution: fsck =“file-system check” • Analogous utilities for Windows, etc. • Detect and repair structural integrity problems • Scan file system’s entire meta data • Looks for inconsistencies & fixes them • Problems? • S…l…o…w… • Might not succeed

  10. Journaling File Systems • Solve problem by maintaining journal • Log all transactions to disk • Updates to the disk committed atomically • Example: Creating a new file • Log modifications to inodes • Log changes to free lists, directory entries • Now begin update • When finished, write “committed” into journal

  11. Recovery in JFS • Power outage: • Some updates fully committed to file system • No problem • Not yet fully committed • File system reads journal, replays transaction • Advantages over fsck: • Much quicker than scan of disk • Guarantees file system always consistent • But: does not guarantee zero data loss

  12. Advanced File Systems • Brief review • Journaling • Log-structured file system

  13. LFS: Beyond JFS • Insight:While logging updates, why not logthe data too? • “Log-structured file system” • Preserves data integrity • Provides improved performance, too!

  14. Sprite LFS[Rosenblum & Ousterhout] • Outperforms then-current Unix file systems by an order of magnitude for small-file writes • Matches or exceeds Unix performance for reads and large writes • Even with overhead • Can use 70% of disk bandwidth for writing • Unix file systems typically can use only 5-10%

  15. Technology Trends • Disk technology improving rapidly • But: capacity rather than performance • Increasing RAM ) cache more effective • Disk traffic dominated by writes • Logs: write new information sequentially • Increase write performance, eliminating seeks

  16. Existing File Systems • Exercise worst-case given trends! • Information spread around disk • Many small accesses • Synchronous writes • Application performance can be bottlenecked by disk performance

  17. Log-Structured File Systems • Approach: improve write performance by combining into single sequential writes • Converts small random writes into large sequential transfers • Use file cache as write buffer • But where does meta data go?

  18. Structure of LFS (1/3) • Some index structures used to retrieve information • Inodes not fixed: inode map used to maintain location of each inode • Fixed checkpoint region on each disk: identifies locations of inode map blocks

  19. dir1 dir2 Log Disk Sprite LFS file1 file2 file1 file2 Disk Unix FFS dir1 dir2 Inode Directory Data Inode map Structure of LFS (2/3) • Disk layout (compared with Unix FFS)

  20. Structure of LFS (3/3) • Segments • Important to maintain large free extents for writing new data • Divide disk into large fixed-size extents called segments • Segments always written sequentially • Before segment is rewritten,all live data must be copied out (“cleaned”)

  21. Crash Recovery (1/2) • Checkpoints • Position in log at which all of file system structures are consistent and complete • Checkpoint region written on special fixed position: • Periodic intervals • When the file system unmounted or shut down

  22. Crash Recovery (2/2) • Roll-forward • After crashes, LFS scans through log segments that were written after last checkpoint • Use information in segment summary blocks • When summary block indicates presence of new inode, LFS updates inode map

  23. LFS Wrap-Up • Extends journaling to data • Sequential writes in segments • Fast! • Periodic clean-up • Periodic checkpoints

  24. Summary • Traditional filesystems – integrity problems • Solved by journaling • Log-structured further improves speed by optimizing write performance

More Related