DFS Design and Implementation

1. DFS Design and Implementation Vijay Neelakandan vneelakandan1@student.gsu.edu

2. What is DFS? • In computing, a distributed file system is a network file system where a single file system can be distributed across several physical computers[3]. • DFS allows administrators to group shared folders located on different servers by transparently connecting them to one or more DFS namespaces.

3. Characteristics of a DFS[3] • Network transparency: same access operation as local files • Location transparency: file name should not reveal its location • Location independence: file name should not be changed when its physical location changes • User mobility: access to file from anywhere • Fault tolerance • Scalability • File mobility: move files from one place to another in a running system

4. What and Why DFS ? • Distributed file systems generally include facilities for transparent replication and fault tolerance. That is, when a limited number of nodes in a file system go offline, the system continues to work without any data loss. • Transparency • Name Service, Directory Service, Caching and replication, access control and protection

5. Files and File Systems

6. Files and File systems • Files are named data objects. Files hold structured data that are used by programs but that are not part of the programs themselves[1]. • File system is responsible for the naming, creation, deletion, retrieval, modification, and protection of a file in the system. • Logical components of a file for users. File Name File Attributes Data units

7. Files and File systems[1] • File name: symbolic name • When accessing a file, its symbolic name is mapped to a unique file id (ufid or file handle) that can locate the physical file • Mapping is the primary function of the Directory Service • File Attributes: Name, Size, Location, Time, Type etc. • Data units: Organization • Flat structure of a stream of bytes of sequence of blocks • Hierarchical structure of indexed records

8. Files and File systems • File Access • Sequential access mode File position pointer to indicate the position of the next data unit to be accessed. • Direct access Explicitly reference fixed-size data units by their block numbers. • Indexed sequential access Use an index to locate the block in which the key/object pair resides,and then accessing the data in the block until the is found.

9. Example • UNIX • Files are streams of characters for application programs and sequences of logical fixed size blocks for file system. • Both sequential and direct access methods are supported. other access methods can be built on top of the flat file structures.

10. Major Components in a file system

11. Directory Service • Directories are files that contain names and addresses of other files and subdirectories. • Mapping and locating • Search for a file • Create a file • Delete a file • List a directory • Rename a file • Traverse the file system

12. Authorization Service • File access must be regulated to ensure security • Types of access • Read • Write • Execute • Append • Delete • List

13. File Service – Basic Operations • Create • Allocate space • Make an entry in the directory • Write • Search the directory • Write is to take place at the location of the write pointer • Read • Search the directory • Read is to take place at the location of the read pointer • Reposition within file – file seek • Set the current file pointer to a given value • Delete • Search the directory • Release all file space • Truncate • Reset the file to length zero • Open(Fi) • Search the directory structure • Move the content of the directory entry to memory • Close(Fi) • move the content in memory to directory structure on disk • Get/set file attributes

14. System Service • System services are a FS’s interface to the hardware and are transparent to users of FS • Mapping of logical to physical block addresses • Interfacing to services at the device level for file space allocation/de-allocation • Actual read/write file operations • Caching for performance enhancement • Replicating for reliability improvement

15. Interaction among services in a DFS Directory Services Authorization services Clients File services System service

16. Organization of data files in a file system

17. File Mounting and Server Registration

18. File Mounting and Server Registration • Attach a remote named file system to the client’s file system hierarchy at the position pointed to by a path name (mounting point) • A mounting point is usually a leaf of the directory tree that contains only an empty subdirectory • Once files are mounted, they are accessed by using the concatenated logical path names without referencing either the remote hosts or local devices • Location transparency • The linked information (mount table) is kept until they are unmounted

19. DFS DSM /chow/book/DSM File Mounting Example root root Export chow OS Mount paper book DFS DSM /OS/DSM Local Client Remote Server

20. File mounting and Server Registration • Mounting Strategy • Explicit mounting: clients make explicit mounting system calls whenever one is desired • Boot mounting: a set of file servers is prescribed and all mountings are performed the client’s boot time • Auto-mounting: mounting of the servers is implicitly done on demand when a file is first opened by a client

21. A Simple Automounter for NFS

22. Server Registration • The mounting protocol is not transparent – the initial mounting requires knowledge of the location of file servers • Server registration • File servers register their services, and clients consult with the registration server before mounting • Clients broadcast mounting requests, and file servers respond to client’s requests

23. Stateful and Stateless File Servers

24. Stateful and stateless File Servers • State information Opened files and their clients File descriptors and file handles Current file position pointers Mounting info Lock status Session keys Cache or buffer

25. Stateful and stateless File Servers • A file server is called stateful if it maintains internally some of the state information and stateless if it maintains none at all. • Stateless file server – when a client sends a request to a server, the server carries out the request, sends the reply, and then remove from its internal tables all information about the request • Between requests, no client-specific information is kept on the server • Each request must be self-contained: full file name and offset… • Stateful file server – file servers maintain state information about clients between requests

26. Comparing

27. Research

28. Integrated High performance DFS Scientific computing applications running in the cluster environment require high performance distributed file system to store and share data[6]. 2. A new approach, the IncFS∗ , of building a high performance distributed file system by integrating many NFS servers . 3. The IncFS is aimed at providing a simple and convenient way to achieve high aggregate I/O bandwidth for scientific computing applications that require intensive concurrent file access[6].

29. INFS continued.. 4. The InFcS uses a hyper structure to integrate multiple NFS file systems. And it provides multiple data layouts to effectively distribute file data among those NFS servers.

30. INFS Continued..

31. Achieving High Availability[2] Achieving high availability is a premium goal for many distributed file systems.File replication is a well-known technique that is used to achieve this goal. It generally offers reduced client latencies and increases files availability. 2. To achieve this goal two algorithms are proposed, namely a primary replica assignment algorithm and an intelligent replica placement algorithms.

32. References • Distributed File System: Efficiency Experiments for Data Access and CommunicationUpadhyaya, B.; Azimov, F.; Doan, T.T.; EunmiChoi; SangBum Kim; Pilsung Kim;Networked Computing and Advanced Information Management, 2008. NCM '08. Fourth International Conference onVolume 2,  2-4 Sept. 2008 • Towards Achieving a Highly Available Distributed File SystemAbdalla, S.; Ahmad, I.; Ewe Hong Tat; GimAikTeh; Yong Lee Kee;Advanced Communication Technology, The 9th International Conference onVolume 3,  12-14 Feb. 2007 • Randy Chow, Theodore Johnson, Distributed Operating Systems and Algorithms, Addison-Wesley, 1997.

33. References 4. Distributed File SystemsPierre Boulet Masters Informatique TIIR et IAGL September 28,2006. 5 . Glagoleva,2000.A load balancing tool based on mining access patterns for DistributedFileSystem servers. System Sciences. HICSS. Proceedings of the 35th Annual Hawaii International Conference 6 . IncFS: an integrated high-performance distributed file system based on NFSYi Zhao; Rongfeng Tang; Jin Xiong; Jie Ma;Networking, Architecture, and Storages, 2006. IWNAS '06. International Workshop on1-3 Aug. 2006

34. Thankyou