Windows 7: Design Principles & Components

1. Chapter 19: Windows 7

2. Chapter 19: Windows 7 • History • Design Principles • System Components • Environmental Subsystems • File system • Programmer Interface

3. Objectives • To explore the principles upon which Windows 7 is designed and the specific components involved in the system • To understand how Windows 7 can run programs designed for other operating systems • To provide a detailed explanation of the Windows 7 file system • To cover the interface available to system and application programmers

4. History • 1985: Microsoft and IBM begin joint development of OS/2 Windows 1.0 released (first OS with Win16 API) Intel releases 80386 (32-bit microprocessor) • 1987: OS/2 released Windows 2.0 released • 1988: IEEE releases the first version of the POSIX standards Microsoft begins work on “New Technology”: a portable OS that supports OS/2 and POSIX API • 1990: Windows 3.0 released, meets huge popularity NT native API switched from OS/2 to Win32 • 1992: Windows 3.1 released • 1993: Windows NT 3.1 (actually the first version of NT) released (first OS with Win32) Microsoft and IBM part ways • 2000: Windows 2000 (NT 5.0) released • 2001: Windows XP (NT 5.1) released (first 64-bit Windows) OS/2 4.5 released (final version) • 2006: End of support for OS/2 • 2007: Intel stops manufacturing the 80386 • 2009: Windows 7 released • 2012: Windows 8 released • 2015: End of support for Windows 7 Windows 10 released

5. Windows 7 • 64-bit preemptive multitasking operating system for Intel and AMD microprocessors • Modular layered micro-kernel architecture • Compatibility • Backward-compatible with MS-DOS and MS-Windows applications • Complies with POSIX and OS/2 API • Can run 32-bit and 64-bit applications

6. Design Objectives • Security • Design time: • Extensive code review and testing + use of automatic analysis tools • Run time: • Access control lists • Integrity levels of objects (an object cannot edit one at a higher level; prevents web browsers being tricked into editing software) • Address-space layout randomization (prevents malicious code injection into software) • Read-only memory stacks and heaps (prevents attacks by inserting and executing code in data memory) • Reliability • Design time • Code maturity (reusing functional code from past OS) • Stress testing • Run-time: automatic testing • Analysis of source code • Providing invalid/unexpected input parameters to functions • Fault-tolerant memory heap that automatically mitigates future executions of code that crashed

7. Design Objectives • Compatibility • Backward-compatible through “compatibility layer” between user-application layer and Win32 API layer • POSIX-compatible through Interix, a POSIX/UNIX environment subsystem • Virtual XP machine available • Performance • NT subsystems communicate together via high-performance LPC (local procedure call) message passing • User-Mode Scheduling (UMS) allows programs to be decomposed and run in parallel on multiple CPUs by user-mode scheduler • Extensibility • Executive layer (protected mode) provides the basic system services • User layer (user mode) provides several modules for different environmental subsystems (emulates different OS) • Easy to extend by adding environmental subsystems without affecting executive layer

8. Design Objectives • Portability • Written in C and C++ • Processor-dependent code is isolated in dynamic link libraries (DLL) that constitute the hardware abstraction layer (HAL) • International support • National language support (NLS) API provides special routines to format date, time and currency • Specialized string comparison functions to account for different character sets • System text strings are in resource files that can be replaced easily • Energy efficiency • Decrease CPU usage to reduce energy use • Allow CPU to skip clock ticks, merge multiple software timers together, stop entire CPUs (in multi-CPU systems) when workload drops

9. Windows 7 System Components • Layered system of modules • Protected mode: HAL, kernel, executive layer • User mode: all other subsystems • Environmental subsystems emulate different operating systems • Protection subsystems provide security functions

10. Windows 7 System Components

11. Components – HAL • Provides a virtual machine interface to the kernel • Administrative functions of OS (memory management, CPU dispatching, I/O bus configuration, DMA, etc.) handled by HAL • It’s what makes Windows XP portable

12. Components — Kernel • Foundation for the executive and the subsystems • Never paged out of memory; execution is never preempted • Four main responsibilities: • Thread scheduling • Interrupt and exception handling • Low-level processor synchronization • Recovery after a power failure • Kernel is object-oriented • Object type is a system-defined data type that has a set of attributes (values) and a set of methods (functions) • Kernel objects store kernel data and perform kernel activities • Dispatcher objects control dispatching and synchronization (events, mutants, mutexes, semaphores, threads and timers) • Control objects(asynchronous procedure calls, interrupts, power notify, power status, process and profile objects)

13. Kernel — Process and Threads • A process has a virtual memory address space, information (such as a base priority), and an affinity for one or more processors • A process has one or more threads • Threads are the unit of execution scheduled by the kernel’s dispatcher • Each thread has its own state, priority, processor affinity, and accounting information

14. Kernel — Thread Scheduling Transition Ready Standby Running Waiting Terminated All needed resources become available Exit Interrupt Selected as next to run Scheduler dispatch Block on dispatcher object Dispatcher object signalled

15. Kernel — Thread Scheduling • The dispatcher uses a 32-level priority scheme divided in two classes • The real-time class contains threads with priorities ranging from 16 to 31 • The variable class contains threads having priorities from 1 to 15 • Special “idle thread” run when no other thread is ready, has priority 0 • One queue per priority; dispatcher checks each queue in order to find the next ready thread with correct processor affinity (or the idle thread) • Variable-Class thread’s priority changes • When time-quantum runs out, its priority decreases • When it wakes from waiting for a dispatcher object, its priority increases • Increase depends on the object it waited for: mouse/keyboard get largest increase • Threads associated with user’s active window get a boost Class Priority

16. Kernel — Thread Scheduling • Tends to give very good response times to interactive threads that are using the mouse and windows • Enables I/O-bound threads to keep the I/O devices busy • Prevents CPU-bound threads from monopolizing the CPU • Real-time threads are given preferential access to the CPU; but Windows does not guarantee that a real-time thread will start to execute within any particular time limit • This is known as soft realtime • Uses timestamp counter instead of clock sampling to measure process usage of CPU (more accurate) • User thread can yield to another user thread using user-mode scheduler, without switching to kernel mode

17. Kernel — Synchronization • Dispatcher objects control dispatching and synchronization • Event object: Record the occurrence of an event and synchronize it with some actions • Mutant: Kernel-mode or user-mode mutual exclusion locks • Mutex: Kernel-mode-only deadlock-free mutual exclusion lock • Defsemafore: Counting object to control the number of threads accessing a resource • Thread object: Is the object actually scheduled by the dispatcher • Timer object: Keeps track of time and signals timeouts when an operation takes too long to complete

18. Kernel — Software Interrupts • Software interrupts are implemented as two kinds of procedure calls • Asynchronous Procedure Calls must be executed immediately • Interrupt current process • Used to start and terminate threads and processes, or notify of asynchronous I/O task completion • Deferred Procedure Calls can wait • Queued and postponed after device interrupts, so they do not block other ISR

19. Kernel — Exception Handling • Architecture-independent exceptions • Memory-access violation, integer overflow, floating-point overflow or underflow, integer divide-by-zero, floating-point divide-by-zero, illegal instruction, data misalignment, privileged instruction, page-read error, access violation, paging file quota exceeded, debugger breakpoint, debugger single step • Exception dispatcher saves current state and finds an exception handler • Kernel-mode • Execute exception handler • If none is found: blue-screen-of-death • User-mode • Environment subsystem sets up a debugger port and exception port for every process • Exception handler sends exception to debugger port • If debugger port not found or does not handle it, exception handler looks for a handler routine • If none is found: try debugging again • If no debugger is running: give the environment subsystem a shot • If that still doesn’t work, terminate the process

20. Kernel — Interrupt Handling • Interrupt is represented by an interrupt object • Contains all required data • Associated with an ISR • Avoids the need for kernel to access hardware directly • Interrupt dispatcher calls the appropriate ISR • But different processors have different lists and number of interrupts • Interrupt dispatcher maps hardware interrupts to a standard Windows XP set of 32 Interrupt Request Levels (IRQL) • Handled in order of priority • Kernel binds IRQL to ISR using aninterrupt-dispatchtable • Independent tablefor each processor

21. Components — Executive Layer • Provides services for environmental subsystems

22. Executive — Object Manager • User-mode programs manipulate objects that serve as generic APIs to kernel-mode entities • Dispatcher objects, file objects, port objects, I/O device objects, etc. • Each object accessed using a handle, a standardized interface • Each process has an object table of handles to objects it currently uses • Objects can have a name or not • Object names are structured like file path names in MS-DOS and UNIX • Directory objects contain the names of all objects in directory • Objects can be accessed by creating a handled to a named object or by duplicating a handle to an unnamed object • Name can be permanent (such as a disk drive object) or temporary (the object name is deleted when last handle is closed)

23. Executive — Object Manager • Object manger manages the use of all the objects • Providing interfaces to define object types and instances • Creating and deleting objects • Generating handles to objects and returning them to processes • Enforcing security • Checking that the security ID of a thread opening a handle is in the object’s access-control list • Translating names to objects • Maintaining the name space • Maintaining the reference count of protected-mode references to the object and the handle count of object table handles to the object

24. Executive — Object Manager • Objects are manipulated by a standard virtual functions • create, open, close, delete, query name, parse and security • Different implementations for each object type • Create: creates a new object and return a handle to it • Open: returns a handle to an existing object • Close: remove a handle to an object the process owns • Decrements the object’s handle count and reference count • When an object is not used by anyone, the manager calls its delete procedure • The manager can make an object permanent by adding an extra reference to it • Delete: type-specific clean-up operations, then remove from memory • Query name: return the name of an object a thread has a reference to • Parse: search for an object given its name • Security: perform a security check on an operation, to make sure the thread has the right to perform it on that object

25. Executive — Virtual Memory Manager • In charge of virtual address space, physical memory management, paging • Page size of 4 KB or 2MB on AMD64 and IA32, 8 KB on IA64 • Each process has a virtual memory space • 4 GB for IA32, 8 TB for AMD64 • Space mapping: • CPU page table maps pages to frames • Page table self-map maps page table content using virtual addresses • Hyperspace maps working set to kernel-mode addresses • Session space maps drivers across different processes • Two-step process to allocate physical memory to user process • Reserve a portion of the process’ virtual address space • Commit the allocation by assigning physical space • Shared memory implemented using section object • Process maps a portion of it (view) into its own memory space • Section protection settings: read-only, read-write, read-write-execute, execute-only, copy-on-write, no access (used to detect illegal access)

26. Executive — Virtual Memory Manager • Three-level hierarchical page table • Virtual-to-physical address translation is done like we learned…

27. Executive — Virtual Memory Manager • A frame can be in one of seven states: • Valid: in use by a process • Free: not referenced by a PTE • Zeroed: Free frame that has been zeroed out for certain applications • Modified: page that has been edited since being put in memory, must be written to disk before being reallocated • Standby: frame that has not been edited since being put in memory. Could be a page that was not edited, a page that was modified and written, or a page that was pre-fetched for locality • Bad: a hardware error was detected on the frame; do not use • Transition: the frame has been allocated and the page is being loaded from the disk

28. Executive — Virtual Memory Manager • Per-working-set LRU page replacement algorithm • Process created with default minimum number of frames • Allowed to grow until physical memory runs low, then page replacement algorithm kicks in • Process can also have a hard limit on number of pages (even if plenty of memory is available) • Process that grows too quickly gets trimmed automatically (even if plenty of memory is available) • VMM tracks both user and kernel process, but trims each differently • Locality: VMM prefetches pages adjacent to loaded page

29. Executive — Process Manager • Provides services for • Creating, deleting, and using threads and processes • Queuing and delivering asynchronous procedure calls to threads • Debugger support to suspend and resume threads • Thread impersonation: allowing a thread to run with the security ID of another thread • Does not deal with • Parent-child relationships or process hierarchies (that’s the responsibility of each environment subsystem) • Thread scheduling (handled by the kernel dispatcher) • Processes represented by job objects • Contains values and limits on CPU usage, working-set size, processor affinity

30. Executive — I/O Manager • The I/O manager is responsible for • file systems • device drivers • network drivers • Works with VMM to provide memory-mapped file I/O • Controls the XP cache manager, which handles caching for the entire I/O system • Keeps track of which drivers and file systems are loaded • Manages buffers for I/O requests • It does asynchronous I/O operations, but can simulate synchronous I/O by explicitly waiting for an operation to terminate

31. Executive — I/O Manager • Drivers represented in Windows by driver object • Device represented by device object which links to driver object • Allows the same driver to handle multiple objects • Drivers for each device are arranged in a driver stack • Various drivers can be inserted in the stack as filter drivers that modify the I/O operation, to add high-level functionalities or utilities • I/O manager converts a request to an I/O request packet (IRP) and passes it to the top driver on the stack • Driver processes the IRP, then calls the manager to pass it to the next driver in the stack (to continue processing) or to complete the operation (if all processing is done)

32. Executive — Cache Manager • Works with the VM Manager to provide services to the I/O Manager • Cache blocks are 256KB, hold a view, represented by a virtual address control block that store VM address, file offset of view, process count • Cache manager also predicts and prefetches data by looking for simple patterns in recent request history (such as sequential requests)

33. Executive — Security Reference Monitor • Responsible for run-time access validation and audits of all user-accessible entities in computer system • Whenever a process opens a handle to an object, the security reference monitor checks the process’s security token and the object’s access control list to see whether the process has the necessary rights

34. Executive – Plug-and-Play & Power Manager • Plug-and-Play (PnP) manager is used to recognize and adapt to dynamic changes in the hardware configuration • When a new device is added the PnP manager loads the appropriate driver, determines the optimal resource assignment, and sends a start-device request to the driver with the resource assignment • If a device needs to be reconfigured (to change resource assignment for example) , the manager stops it, reconfigures it, and starts it again • Power manager reduces the hardware power consumption (useful to preserve laptop battery) • Individual devices or entire system can be put in low-power (standby or sleep) mode, where only physical memory uses power • Hibernation mode copies physical memory to disk and shuts down system • Reduce clock speed of underutilized CPU

35. Executive — Registry • Windows stores configuration information in an internal database called the registry separated in hives • Separate hives for system information (including boot information), user preferences, software installation, and security • When the system boots successfully, a copy of the registry is backed up as the system restore point • If the system cannot boot (because the user screwed something up) Windows can boot using the last known good configuration

36. Executive — Boot Sequence • BIOS loads bootloader, which loads BOOTMGR program • Runs WINRESUME if system was hibernating to restore • Runs WINLOAD is system is booting • WINLOAD loads: • HAL (hal.dll) • Kernel (NTOSKRNL.exe) • System hive • Passes execution to kernel • Kernel starts two processes • System process for kernel threads • Session Manager Subsystem (SMSS) • Initializes VMM, device drivers, and background processes: • WININIT to run Session 0 (all other sessions are user sessions) • LSASS, the security subsystem • SERVICES, the background services manager • CSRSS, the Win32 environment subsystem • WINLOGON to run user sessions

37. Components — Environmental Subsystems • User-mode processes layered over the executive services to enable Windows to run programs for other OS • Win32 subsystem is the main operating environment

38. Components — Environmental Subsystems • When a new application is executed • Win32 subsystem calls VMM to load application into memory • VMM returns the type of executable to Win32 subsystem • If it is a Win32 executable, the subsystem runs it • If not, Win32 looks for the appropriate environmental subsystem • Loads it if it is not already running and gives control of the application to it

39. Components — Environmental Subsystems • MS-DOS environment is provided by a Win32 application called the virtual DOS machine (VDM), based on MS-DOS 5.0 • Cannot run all MS-DOS applications: forbids direct access to hardware and CPU hogging • WIN16 Environment provided by a VDM called Windows on Windows, based on Windows 3.1 kernel routines and sub routines • WIN32 shim layermodifies WIN32 behaviour to approximate Windows 95/98 for backwards compatibility • Windows XP Virtual Machine available for “bug-for-bug” reproduction of that OS • The POSIX subsystem is designed to run POSIX applications following the POSIX standard based on the UNIX model

40. NT File System (NTFS) • Fundamental structure is the volume • A logical disk partition • May occupy a portion of a disk, an entire disk, or span several disks • Volume information is stored in a set of metadata files • Basic unit of disk allocation is the cluster • Power-of-two number of sectors • Larger cluster sizes for larger volumes • Physical disk address is cluster number × cluster size

41. NTFS — Files and Directories • A file is a structured object consisting of attributes • Each attribute is an independent byte stream, can be created, edited, deleted • Standard attributes: name, creation time, security attributes, etc. • User data stored in an unnamed data attribute • Named data attributes can be used to store special data or attributed • A file has one or more records in the Master File Table (MFT) • One of the volume metadata files • Between 1KB and 4KB (set at file system creation time) • Small file attributes stored in MFT (resident attributes) • Large file attributes are extents, pointed to by pointers in MFT (nonresident attributes) • Very large files with lots of pointers to extents have a base file recordin MFT that points to overflow record that contains pointers to extents • A directory structure contains information about each file in that directory • Directory structured as B+ Tree • Duplicates file name, update time, size from resident attributes • Eliminates need to retrieve common info from MFT each time

42. NTFS — Recovery • All file system data structure updates are performed inside transactions • Windows log-file service logs all transactions • Log file is one of the NTFS metadata files • Before transaction, service writes a log that contains redo and undo information • After successful transaction, writes a “commit” log • After a crash, the file system data structures can be restored to a consistent state by processing the log records

43. WIN32 API • WIN32 API is the native interface to Windows • Gives interface for accessing kernel, sharing, process management, communication, and memory management • User process can access kernel services by opening a handle to a kernel object • Processes can share objects together: • A child process can inherit a handle to the object from the parent • One process names the object and the second process opens that name • Drawback: names are global, no way to check if the name corresponds to the object you’re expecting • One process gives a handle to the object to the second process, the second one duplicate the handle • Drawback: processes need to communicate

44. WIN32 API • Process is started via the CreateProcess() routine • Loads any DLL the process needs and creates a primary thread • Additional threads can be created by the CreateThread() function • Every DLL or executable file loaded in the process’ address space is identified by an instance handle • Scheduling priority • A process has a priority (idle, normal, high, realtime) • A thread can have a priority bonus or penalty (lowest, below normal, normal, above normal, highest) • A fiber is user-mode code that gets scheduled according to a user-defined scheduling algorithm • Only one fiber at a time is permitted to execute (unlike threads that can be executed concurrently) • Thread pool gives user-mode programs services • Queue thread work requests • Wait for thread completion with or without timeout

45. WIN32 API • Two modes of interprocess communication • Shared kernel objects • Windows message service • Post message (asynchronous, returns immediately, sender not notified of receipt) • Send message (synchronous, blocking until message received and processed) • Process can allocate and deallocate its virtual memory • Enables the application to specify the virtual address at which the memory is allocated • Process can lock a limited number of pages in physical memory • Windows also supports memory-mapped files, heaps, and thread-local storage

46. Review • What are the layers of the Windows 7 architecture? Describe them briefly. • How does NTFS recover from a system crash? • Does Windows 7 provide complete backward compatibility for MS-DOS applications?

47. Exercises • Read everything but skip 19.6 (Networking) • If you have the “with Java” textbook, skip the Java sections and subtract 1 to the following section numbers • 19.2 • 19.3 • 19.4 • 19.5 • 19.6 • 19.8 • 19.11 • 19.12 • 19.13

48. End of Chapter 19