Windows Graphics Overview

1. Windows Graphics Overview David Blythe ArchitectWindows Graphics & Gaming Technologiesdblythe @ microsoft.com Microsoft Corporation

2. Session Outline • Windows Graphics Big Picture • Windows codenamed "Longhorn" Display Driver Model • Basic, Advanced driver models • Graphics APIs • Legacy, Direct3D9, • Windows Graphics Foundation, OpenGL • Desktop Window Manager • Content Protection & Protected Video Path • Glitch-Free Media Support • Windows Graphics Foundation 2.0

3. Windows Graphics in Longhorn User Mode Kernel Mode Microsoft- written code IHV-written code Avalon/DWM Legacy D3D APIs DX VA D3D9 WGF 1.0 WGF 2.0 OpenGL32 OpenGL ICD Common pipeline OGL→D3D User-Mode Driver DXGKernel Kernel-Mode Driver

4. Longhorn Display Driver Model • Graphics Just Works • Fundamental principle • New driver model delivers: • Stability • Security • Availability (virtualization) • Performance • Staged introduction of driver model • Basic model supports current hardware (planned for Longhorn) • e.g., pre-Longhorn hardware • Advanced model utilizes new hardware features (post-Longhorn) • More efficient virtualization • Old XP driver model still supported for compatibility • No new features

5. Stability & Security • Application graphics state isolated from other applications • Surface data (textures, render targets, etc) • Virtual address spaces • Current rendering state • Shaders, etc • Command stream is validated • Prevents access to unauthorized resources • e.g., another application’s surfaces • Prevents illegal command streams • Avoid sending bad commands to hardware • But, still have a watchdog timer • Time out if not making forward progress • Basic driver model does sw validation in kernel • Advanced driver model does hw validation

6. Virtualization • Allow multiple applications to “time slice” the graphics hardware • Scheduler determines which commandstream to run • Basic driver model schedules batches • Batch = sequence of commands submitted by application • Batch can’t be interrupted once started • Advanced driver model schedules contexts • Batches accumulated in application context • Can interrupt a context and run a different one • More responsive to input changes (new work, priority changes) • Better user experience

7. Virtualization • Memory manager determines which memory resources need to be resident • Unused resources can be paged out • Application resources can be larger than video memory • In Window Graphics Foundation APIs only • No duplication of data • Either in graphics memory, system memory, or page file • Basic driver model • All referenced surfaces must be present for application to run • Can fail if too many surfaces (WGF 2.0 can be demanding) • Pages for inactive surfaces can be paged out • Advanced driver model • Only referenced pages need to be resident • Can fault pages in on demand • More efficient utilization of physical memory

8. Device Lost and Device Removed • Virtualization eliminates “device lost” • Eliminate notion of exclusive access to device • However, device can be removed • Low frequency, but possible • Hot-plug • “Device hang” followed by reset equivalent to: • Removal + insertion • Best to not have device hang at all (hint hint) • Use to avoid reboot on driver install • Remove device, add device with new driver • Applications should be written to handle“device removed” events

9. Performance • New model designed with performance in mind • Hardware command streams built directly in user mode • New DDIs better match for modern hardware • Support for renaming and other pipelining operations • Both Basic and Advanced driver models • Advanced model enables maximum performance • Eliminates CPU processing • Validation • Handle mapping, patching

10. Advanced Driver Model • Advanced Driver Model is the Future • More responsive scheduling • More efficient memory management • All future content protection, glitch free, etc. plans build on the Advanced Driver Model

11. Legacy APIs Microsoft- written code IHV-written code Avalon/DWM Legacy D3D APIs DX VA D3D9 WGF 1.0 OpenGL32 WGF 2.0 OpenGL ICD Common pipeline OGL→D3D User-Mode Driver User Mode Kernel Mode DXG Kernel Kernel-Mode Driver

12. Legacy APIs • Older Direct3D immediate mode APIs supported • D3DRM (retained mode) API retired • Map older runtimes onto Direct3D9 DDIs • e.g., runtimes ≤ Direct3D8 • Fixed-function onto shaders, etc • Direct3D9 interface matches DirectX 9.0c • Device removed maps to device lost • Memory virtualization “partially hidden” from Direct3D9 • Application limited to size of video memory • Avoids breaking existing applications • Port to WGF 1.0 for full virtualization

13. Windows Graphics Foundation 1.0 Microsoft- written code IHV-written code Avalon/DWM Legacy D3D APIs DX VA D3D9 WGF 1.0 WGF 2.0 OpenGL32 OpenGL ICD Common pipeline OGL→D3D User-Mode Driver User Mode Kernel Mode DXG Kernel Kernel-Mode Driver

14. Windows Graphics Foundation 1.0 • Enhanced version of Direct3D9 • Also known as “DX9.L” • Cross-process shared surfaces • “Unlimited memory” • All memory resources are “managed” • Creation fails when run out of pagefile • Resource management controls • Prioritization of resources • Antialiased text rendering support • Monochrome texture filter with large kernel size • More accurate gamma control • Extended, higher resolution gamma tables • Still support legacy 256x3 table for GDI • No more “device lost”

15. OpenGL Microsoft- written code IHV-written code Avalon/DWM Legacy D3D APIs DX VA D3D9 WGF 1.0 WGF 2.0 OpenGL32 OpenGL ICD Common pipeline OGL→D3D User-Mode Driver User Mode Kernel Mode DXG Kernel Kernel-Mode Driver

16. OpenGL • Continue OpenGL support • Important component of the Windows platform, particularly for high-end workstation applications • In-box hardware accelerated version of OpenGL • Uses WGF 1.0 as common graphics pipeline • Upgraded API support compared to XP • OpenGL 1.4 • Continue to support OpenGL ICDs • Can be shipped by OEMs • New ICDs required for LDDM • Old XP ICDs work with XP driver model in Longhorn

17. Desktop Window Manager • New desktop presentation built usingAvalon and DirectX • Uses new WGF 1.0 interfaces • Desktop Window Manager (DWM) • Composited desktop • Window movement without repainting underlying windows • No more dragging garbage • Client areas from applications • GDI, Direct3D, Windows Graphics Foundation, Avalon • Video • Non-client areas from the Window Manager • Allows different views of application windows

18. Desktop Window Manager • Supports high-dpi displays • Composition with magnification • Supports video playback • Takes advantage of Direct3D9 shaders • For non-client areas, transitions • Animated transitions, etc. • New WGF 1.0 cross-process shared surfaces • Allows DWM to access application back buffers • Turns off when certain application types running • Overlay planes, front buffer rendering, …

19. Desktop Window Manager Avalon Application WGF/DX Application GDI Application surface surface surface Desktop Window Manager

20. Full-Screen Exclusive, Multimon, and DWM • So, what happens to full-screen exclusive? • Replaced with exclusive output control • Full-screen application owns the display output • Video mode, gamma, etc. • Desktop window manager disappears • Other applications can continue to run • Doesn’t trigger device lost, etc. • May bump the priorities of the output-exclusive app • Applications should recognize case where clientarea is occluded • Background processing on the GPU • Multiple monitors independent • Composited desktop image per monitor • Exclusive output independent for each monitor

21. Content Protection Microsoft- written code IHV-written code Avalon/DWM Legacy D3D APIs DX VA D3D9 WGF 1.0 WGF 2.0 OpenGL32 OpenGL ICD Common pipeline OGL→D3D User-Mode Driver User Mode Kernel Mode DXG Kernel Kernel-Mode Driver

22. Content Protection • Preparing the PC for HD-DVD • Prevent “high-value” content from escaping • Work with content providers on protection rules • Two areas of protection • While content being processed on CPU • While content being transmitted to GPU and processed on GPU • Supported by Protected Media Path (PMP) and Protected Video Path (PVP)

23. Protected Video Path • Protection requirements • Ensure that graphics device is trusted • Driver authenticates graphics chip, OS authenticates driver • Protect content transmitted to graphics card • Special case for data crossing a “user-accessible bus” (UAB) • Encrypt content crossing UABs with an approved cipher • E.g., AES128 in counter mode • Encryption requires initialization using Diffie-Hellman algorithm • Protect content transmitted from graphics card to display device • Use “standard” output protection mechanisms (HDCP, etc) • Control using Output Protection Manager (OPM)

24. Protected Video Path Roadmap • Non-UAB systems planned for Longhorn: • Small set of driver DDIs • Driver authentication • OPM • Updates to DirectX Video Acceleration (DXVA) • Authentication of cross-process shared surfaces • Basic driver model • UAB systems require Advanced driver model • Support for paging with encrypted transfers • New DDIs • Initialization, encrypted transfers, key management, paging

25. Glitch-Free Media Support Microsoft- written code IHV-written code Avalon/DWM Legacy D3D APIs DX VA D3D9 WGF 1.0 WGF 2.0 OpenGL32 OpenGL ICD Common pipeline OGL→D3D User-Mode Driver User Mode Kernel Mode DXG Kernel Kernel-Mode Driver

26. Glitch-Free Media Support • Multi-faceted plans to address causes of glitches • Several graphics-related initiatives • Memory, scheduler controls • Statistics & feedback • Video sync & flip counts with timestamps • Method to calibrate GPU timestamps against CPU timestamps • Compute current frame being displayed • Hardware “flip” buffering • Hardware-supported “flip queue” • Allow playback to get several frames ahead • Provide elasticity to cover unexpected delays • Hardware support eliminates all software delays • Comes at “modest” increase in latency • Comes at a cost in additional memory

27. Windows Graphics Foundation 2.0 Microsoft- written code IHV-written code Avalon/DWM Legacy D3D APIs DX VA D3D9 WGF 1.0 WGF 2.0 OpenGL32 OpenGL ICD Common pipeline OGL→D3D User-Mode Driver User Mode Kernel Mode DXG Kernel Kernel-Mode Driver

28. Windows Graphics Foundation 2.0 • Next Version of Direct3D • Supported in Longhorn • Proposed Gold Logo requirement • Major improvements in • Hardware consistency • Clearest, most detailed specification • Elimination of capability bits • Programmer expressiveness • Unified shader programming model • More capable shader unit • Flexible memory model • New pipeline stages • LDDM + WGF 2.0 are fundamental improvement to the Windows Gaming Platform

29. Windows Graphics Foundation 2.0 • Performance improvements • Applications limited by state change performance • LDDM helps by eliminating extra processing • WGF 2.0 refactors state for efficient management • Enable GPU processing without CPU intervention • More flexible memory model • Stream output from middle of pipeline • Feedback to front of pipeline, with predication • Efficiency improvements • Multiple samples from textures, integer addressing • Compact surface formats (scenario-specific) • HDR, normal map compression, … • Arrayed resources

30. Windows Graphics Foundation 2.0 • Geometry shader stage • “Sees” entire primitive (3 vertices of triangle) • Can have adjacent vertices too (6 vertices total) • Limited amplification • Extrude edges, expand points, generate shells, … • Per-primitive processing • Generate extra per-primitive constant data for pixel shaders • Constant colors, normals, etc • Compute plane equations, barycentric parameters, etc. • Combine with stream output or arrayed resources • Render to cube map • Render multiple shadow maps

31. Windows Graphics Foundation 2.0 fixed programmable memory Constant Constant Constant Input Assembler Vertex Shader Geometry Shader Setup Rasterizer Pixel Shader Output Merger Sampler Sampler Stream out Sampler Vertex Buffer Index Buffer Texture Texture Stream Buffer Texture Depth Stencil Render Target Memory

32. Call To Action • Longhorn Display Driver • Basic model for Longhorn launch • Future Advanced model • Support Windows Graphics Foundation 2.0 • Comprehensive hardware spec for nextgeneration content • Support Protected Video Path features • Output protection management, for Longhorn launch • Protected transfers (discrete cards) forvalidation in 2006 • Support Glitch Free features • Counters, timestamps, flip queue ASAP • Advanced driver model

33. Community Resources • Windows Hardware & Driver Central (WHDC) • www.microsoft.com/whdc/default.mspx • Technical Communities • www.microsoft.com/communities/products/default.mspx • Non-Microsoft Community Sites • www.microsoft.com/communities/related/default.mspx • Microsoft Public Newsgroups • www.microsoft.com/communities/newsgroups • Technical Chats and Webcasts • www.microsoft.com/communities/chats/default.mspx • www.microsoft.com/webcasts • Microsoft Blogs • www.microsoft.com/communities/blogs

34. Additional Resources • Email • directx @ microsoft.com • Related Sessions • Avalon Technologies • “Advances in Display and Composition Architecture for Windows” • PMP and PVP • “Protected Media Path & Driver Interoperability Requirements” • “Longhorn Output Content Protection” • WAVE • “Windows Audio Video Excellence”