1 / 123

Introduction to the Microsoft exFAT File System

NYC4SEC Meet-up Group ‎John Jay College-Criminal Justice 899 10th Avenue, New York, New York Wednesday June 11 th , 2014 06:30pm. Introduction to the Microsoft exFAT File System. Robert Shullich CPP, CISSP, CRISC, GSEC, GCFA, CEH, CHFI, CCFP-US. But First, are you D4CS?. Agenda.

shyla
Download Presentation

Introduction to the Microsoft exFAT File System

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. NYC4SEC Meet-up Group‎John Jay College-Criminal Justice899 10th Avenue, New York, New YorkWednesday June 11th, 2014 06:30pm Introduction to the Microsoft exFAT File System Robert Shullich CPP, CISSP, CRISC, GSEC, GCFA, CEH, CHFI, CCFP-US NYC4SEC Meet-up Group – June 11th, 2014

  2. But First, are you D4CS? NYC4SEC Meet-up Group – June 11th, 2014

  3. NYC4SEC Meet-up Group – June 11th, 2014

  4. Agenda • About me, the paper and the presentation • The need for a new generation of FAT • Digital Forensics Relevance • Exponents and Standards • exFAT Overview • Linux Development • Memory Cards & Flash Memory • exFAT File System Internals • Closing NYC4SEC Meet-up Group – June 11th, 2014

  5. About me, the paper and the presentation About Me About the Presentation About the SANS Paper A Gold Standard Another Paper Reference Disclaimer exfat NYC4SEC Meet-up Group – June 11th, 2014

  6. About Me I have been in the IT field for 40+ Years, and in InfoSec for over 20 Years I carry many IT and InfoSec certifications This research was originally for a class term project towards my D4CS MS degree I then expanded that term paper into a practical paper for my SANS “Gold” GCFA certification Links to the SANS paper and my blog are provided at the end of this presentation NYC4SEC Meet-up Group – June 11th, 2014

  7. About the Presentation What I call the exFAT Road Show • The New York Forensics Computer Show 4/20/2010 • Techno Security and Digital Investigations 6/7/2010 • SANS What Works in Forensics and IR Summit 7/8/2010 • HTCIA International Training Conference & Expo 9/20/2010 • The New York Forensics Computer Show 4/19/2011 • http://techchannel.att.com/play-video.cfm/2011/8/16/Conference-TV-Computer-Forensics-Show:-Introduction-to-exFAT • NYC4SEC 6/11/2014 • HTCIA International Training Conference & Expo 8/26/2014 NYC4SEC Meet-up Group – June 11th, 2014

  8. About the SANS Paper • Consider it “exFAT – the missing manual” • Very little published about exFAT today • Two current forensics books mention exFAT: • Wiley - Mastering Windows Network Forensics and Investigation • Sybex - EnCase Computer Forensics - The Official EnCE: EnCase Certified Examiner • “For those seeking an in-depth understanding of the exFAT file system, you should read the SANS paper entitled “Reverse Engineering the Microsoft Extended FAT File System (exFAT) by Robert Shullich” NYC4SEC Meet-up Group – June 11th, 2014

  9. A Gold Standard • 2005 Book considered the authority on different file systems • The book’s Author developed the open-source TSK forensics tools (The Sleuth Kit) & Autopsy • This year adding exFAT to TSK NYC4SEC Meet-up Group – June 11th, 2014

  10. Another Paper Reference NYC4SEC Meet-up Group – June 11th, 2014

  11. Disclaimer • The released specification and implementation is Release 1.00 of exFAT • The specification mentions additional features that were not implemented yet, but may at a future time/ Some of these are Windows CE holdovers • Both may be presented today • Some directory entries will be skipped • Focus is Microsoft Desktop/Server implementation • Will talk about Flash/Solid State, but high level • For exFAT, tried to stay with the patent terminology NYC4SEC Meet-up Group – June 11th, 2014

  12. The need for a new generation Legacy FAT Why do we need a new file system? Why do we need Faster I/O and Higher Capacity? Hi-definition movie recording MPEG-4. H.264 exfat NYC4SEC Meet-up Group – June 11th, 2014

  13. Legacy FAT • FAT 8 • 1977 Bill Gates and Marc McDonald • Floppy based • FAT 12 • 1980 • FAT 16 • 1984 with release of PC/AT & MS DOS 3 • FAT 16B • 1987 Compaq DOS 3.31 • FAT 16X • 1995 PC DOS 7.0/Win 95 – LBA Addressing • FAT 32 • 1996 Windows 95 OSR2, 98, ME, MS DOS 7.1 – CHS Addressing • FAT 32X • LBA Addressing NYC4SEC Meet-up Group – June 11th, 2014

  14. Why do we need a new file system? • Current Limits Exhausted (Ran Out of Bits!) • Larger volumes (>2TB) (Scale to Larger Capacity) • Larger files sizes (>4GB) • Faster I/O • (UHS-I: 104 MB/s - UHS-II: 312MB/s) • Removable Media • Flash/Solid State Media • Flexibility • Extensibility (Difficult to add new features) • NTFS Features without the overhead • Easier to implement FS in firmware NYC4SEC Meet-up Group – June 11th, 2014

  15. Why do we need Faster I/O and Higher Capacity? http://www.cnet.com/news/what-is-4k-uhd-next-generation-resolution-explained/ NYC4SEC Meet-up Group – June 11th, 2014

  16. Hi-def movie recording MPEG-4. H.264 NYC4SEC Meet-up Group – June 11th, 2014

  17. Digital Forensics Relevance Relevance to Forensics Study What happens when you have exFAT formatted media and no exFAT support? Forensics Challenges in 2009 Forensics Challenges Today exfat NYC4SEC Meet-up Group – June 11th, 2014

  18. Relevance to Forensics Study • Digital Evidence Extraction • Finding the evidence • Including the hiding places • Validation • Completeness • Daubert Expert Testimony • Need to know and understand file org • Establish Credibility • New Media (SD Cards) will drive exFAT adoption, and the potential for CP investigations. • Larger Media Capacity also driving exFAT adoption NYC4SEC Meet-up Group – June 11th, 2014

  19. Trust but Verify NYC4SEC Meet-up Group – June 11th, 2014

  20. What happens when you have exFAT formatted media and no exFAT support? NYC4SEC Meet-up Group – June 11th, 2014

  21. Forensics Challenges 2009 • In 2009, in regards to exFAT: • No tools (RAW) • No documentation or Training • No expertise • Evidence backlog NYC4SEC Meet-up Group – June 11th, 2014

  22. Forensics Challenges Today • Today • exFAT Misunderstood • Linux OS Support • Tuxera drivers may help • FUSE and No-FUSE hacks • Most Distributions – No native support • Mac OS Support (Nov 2010) OS/X 10.6.5+ • Implementation Deviations, No Standards • Open Source Tools • Commercial Tools • Encase (6.14.3 Dec 2009) • Encase (6.18.0.59) NIST Test March 2014 • FTK (3.2 Oct 2010) • FTK (3.3) NIST Test April 2014 • Cross Vendor Compatibility NYC4SEC Meet-up Group – June 11th, 2014

  23. NIST Computer Forensics Tool Testing • Cyber Fetch • AAFS-2013 Conference 02/21/2013 • Deleted File Recovery Tool • Testing Results • One Summary Item: Support for ExFAT, ext3 & ext4 is sometimes lacking. NYC4SEC Meet-up Group – June 11th, 2014

  24. Test Results for Deleted File Recovery and Active File Listing • 17 Basic Tests • March 12, 2014 – Encase 6.18.0.59 • MAC differed by 9 hours • April 3, 2014 – FTK 3.3.0.33124 • MAC differed by 4 hours • The exFAT partition and HFS+ created on OS/X 10.6 • exFAT: ctime meta-data replaced with the time of file deletion [I was unable to recreate] • Vendor Tool or Apple Implementation? • Who Validates the Test? NYC4SEC Meet-up Group – June 11th, 2014

  25. Who Validates the Validator? Superman: Easy, Miss, I've got you Lois Lane: You...you've got me, who's got you? NYC4SEC Meet-up Group – June 11th, 2014

  26. Exponents and Standards Base 2 or 10? Exponents International System of Units (SI) Table IEC 60027-2 Reference Standards Endian Microsoft Math More Math – exFAT WinCE exfat NYC4SEC Meet-up Group – June 11th, 2014

  27. Base 2 or 10? NYC4SEC Meet-up Group – June 11th, 2014

  28. Exponents 102 = 10 times 10 = 100 103 = 10 times 10 times 10 = 1000 (1K) 22 = 2 times 2 = 4 29 = 2*2*2*2*2*2*2*2*2 = 512 210 = 2*2*2*2*2*2*2*2*2*2 = 1024 (1K) 212 = 2*2*2*2*2*2*2*2*2*2*2*2 = 4096 NYC4SEC Meet-up Group – June 11th, 2014

  29. International System of Units (SI) Table • File System in powers of 2 • Device characteristics in power of 10 NYC4SEC Meet-up Group – June 11th, 2014

  30. IEC 60027-2 http://physics.nist.gov/cuu/Units/binary.html NYC4SEC Meet-up Group – June 11th, 2014

  31. How far off are we? http://cnx.org/content/m13081/1.1/ NYC4SEC Meet-up Group – June 11th, 2014

  32. Reference Standards • Bits are numbered right to left • 76543210 • Decimal Offsets (zero based) • Little-Endian numbers • Unsigned numbers • Sectors vs. Clusters • Strings are 16 bit Unicode • Strings not Terminated NYC4SEC Meet-up Group – June 11th, 2014

  33. Endian • Numbering order may vary based on processor type, is determined by the order the data bytes are read from the register. • A 32 bit number is read as 4 8-bit bytes • If I have the number 0x11 22 33 44 • Big-Endian will store it as: • 0x 11 22 33 44 • Little-Endian will store it as: • 0x 44 33 22 11 NYC4SEC Meet-up Group – June 11th, 2014

  34. Microsoft Math KB184006 Limitations of FAT32 File System The maximum possible number of clusters on a volume using the FAT32 file system is 268,435,445. With a maximum of 32 KB per cluster with space for the file allocation table (FAT), this equates to a maximum disk size of approximately 8 terabytes (TB). 512B Sectors in a 32 KB cluster = 64 228 (268,435,445) * 26 (64) * 29 (512) = 243 = 8,796,093,022,208 Size of FAT32 FS specified in BPB as sectors (32 bit number) NYC4SEC Meet-up Group – June 11th, 2014

  35. More Math, exFAT KB955704 Description of the exFAT file system driver update package • Support for volumes that are larger than 32 GB, the theoretical maximum volume size for FAT32 in Windows XP • The theoretical maximum volume size is 64 ZB. • The recommended maximum volume size is 512 TB. • Support for files that are larger than 4 GB, the theoretical maximum file size for FAT32 in Windows XP • The theoretical maximum file size is 64 ZB. • The recommended maximum file size is 512 TB. NYC4SEC Meet-up Group – June 11th, 2014

  36. WinCE NYC4SEC Meet-up Group – June 11th, 2014

  37. Overview Features of exFAT 1.00 4K (4096) Sector Size Supported Cluster Sizes Features of exFAT 1.00 (cont’d) Future Features of exFAT MBR Partition Limitations Advantages of exFAT Disadvantages of exFAT OS Support for exFAT Key Dates for exFAT exfat NYC4SEC Meet-up Group – June 11th, 2014

  38. Features of exFAT 1.00 • Maximum Volume Size (Increased Capacity) • Architectural ≈ 128 PiB (223-11 * 225) • Implementation = 512 TiB • Sector sizes from 512 [SF] to 4096 bytes [AF] • Clusters sizes to 32MiB (225) • Subdirectories to 256MiB (Root not restricted) • Maximum files on volume ≈ 232 • Maximum File Size 16 EiB-1 • Built for speed, less overhead than NTFS • Catches up with some NTFS features • Template-based metadata structures • On-disk storage of file Valid Data Length (VDL) • Speeds up storage allocation processes NYC4SEC Meet-up Group – June 11th, 2014

  39. 4K (4096) Sector Size NYC4SEC Meet-up Group – June 11th, 2014

  40. Supported Cluster Sizes NYC4SEC Meet-up Group – June 11th, 2014

  41. Features of exFAT 1.00 (cont’d) • OEM Parameters Sector for device dependent parameters • 12 sector VBR, support of larger boot program • Up to 2,796,202 files per sub-subdirectory • File Names max to 255 Characters • 16-Bit Unicode File Names and Volume Labels • Optimized for Flash Memory • Device Boundary Alignment • No FAT32 minimum cluster (65,525) restriction • No 8.3 file name support (only LFN) • UTC Timestamp Support • Vista/Server 2008 SP2+, XP/Server 2003 with KB • Native in Windows 7, 8, 8.1, Server 2008 R2, 2012 NYC4SEC Meet-up Group – June 11th, 2014

  42. Future Features of exFAT • TexFAT (To be released later) • Exists in Windows CE • Transaction Safe exFAT • ACL (To be released later) • Exists in Windows CE • Compression & Encryption Support? • Not announced, but would be easy to add NYC4SEC Meet-up Group – June 11th, 2014

  43. MBR Partition Limitations • Microsoft File Systems are limited when stored in a MBR partition • A partition is defined by a Master Boot Record • A MBR uses a 4 byte value for number of sectors • LBA as 32 bit # times 512 Sector limits to 2TiB • To get the maximum volume size, exFAT cannot be created within a MBR partition, Need GPT GUID Partition, or Super floppy Mode • ExFAT on GPT works on Mac NYC4SEC Meet-up Group – June 11th, 2014

  44. Advantages of exFAT • Large volume, file and directory sizes • Handle growing capacities in media, increasing capacity to >32 GB. • > 1000 files in a single directory. • Speeds up storage allocation processes. • Breaks file size 4 GB barrier. • Supports interoperability with future desktop OSs. • Provides an extensible format. • Large cluster sizes • Metadata integrity with checksums NYC4SEC Meet-up Group – June 11th, 2014

  45. Disadvantages of exFAT • Not all Windows CE features implemented • No direct conversion to or from other FS • Cannot use CONVERT command to NTFS • No Floppy Support • Mostly a Microsoft Desktop and Server World • No Support for Older MS systems (Pre-XP) • Support for other devices, surfacing • No Information Sector “Hint” • Like all FAT – Finding Stuff is via brute force NYC4SEC Meet-up Group – June 11th, 2014

  46. OS Support for exFAT • Windows XP & Server 2003 • KB955704 (requires SP2 or SP3) • Vista & Server 2008 SP1 • Vista & Server 2008 SP2 • (Adds UTC timestamp support) • Windows 7/Server 2008 R2 and later: • RTM • Mac OS/X 10.6.5 and later NYC4SEC Meet-up Group – June 11th, 2014

  47. Key Dates for exFAT • September 2006 – Windows CE 6.0 • March 2008 – Windows Vista Service Pack 1 • January 2009 – Announcement at CES of SDXC specification • January 2009 – Windows XP Drivers Available • May 2009 – Windows Vista Service Pack 2 • August 2009 – Tuxera Signs File System IP Agreement with Microsoft • March 2009 – Pretec Releases first SDXC Cards • December 2009 – Microsoft (re)announces exFAT license program for third-parties • December 2009 – SDXC laptops due soon • December 2009 – Diskinternals releases exFAT recovery utility • December 2009 – Encase support NYC4SEC Meet-up Group – June 11th, 2014

  48. More Key Dates for exFAT • December 2009 Sony, Canon & Sanyo License • January 2010 Funai License (LCD TV) • February 2010 Panasonic License • February 2010 Panasonic 64/48GB SDXC • February 2010 Sony Memory Stick XC • February 2010 SanDisk Ultra SDXC 64GB Card 3.0 Spec $350 • April 26, 2010 DCF Version 2.0 (Edition 2010) • June 1st 2010 Tuxera Releases Linux & Android exFAT drivers • June 3rd 2010 Kingston Releases Class 10 SDXC 64GB Card 60 MB/s read, 35 MB/s write. • October 11th, 2010 FTK 3.2 with exFAT support announced NYC4SEC Meet-up Group – June 11th, 2014

  49. More Key Dates • Mar 16th 2011 Lexar Releases SDXC 128GB • May 3rd, 2011 e.solutions (Volkswagen) • Aug 8, 2012 Sharp for Android Smart Phones • Sep 18, 2012 RIM (Blackberry) Smartphones • Nov 7, 2012 Sharp, Sigma, NextoDi, Black Magic and Atomos Global • Jan 16, 2013 BMW • April 30, 2014 PS4 V1.7 update – hidden new feature: exFAT NYC4SEC Meet-up Group – June 11th, 2014

  50. Linux Development FUSE Project Samsung (No-FUSE) exfat NYC4SEC Meet-up Group – June 11th, 2014

More Related