PCI Express Update for Windows Longhorn

1. PCI Express Update forWindows Longhorn Tony Pierce Technical EvangelistWindows Hardware Evangelismtonypi @ microsoft.com Microsoft Corporation

2. Session Outline • PCI Express Update • PCI Express features supportedin Windows codenamed “Longhorn” • PCI Express features not supported in Longhorn • Resource considerations during design • PCI Express 1.1 • Overview of Delta from PCI Express1.0a • Functionality that should be considered in Windows Platform design

3. Session Goals • Attendees should leave this session with the following: • Know which PCI Express features are supported by Longhorn: • How feature is supported by Longhorn • Any hardware, firmware, or driver considerations to work well with Longhorn • Know which PCI Express features are not supported by Longhorn: • Why features are not supported • Any work to address feature for future support • Which of the new functionality in PCI Express 1.1 is important when designing for Longhorn

4. Enhanced Configuration Space: Support • Memory mapped access to enhanced configuration space • MCFG table indicates memory base to OS • No new interfaces • Existing APIs updated to support 4K enhanced config space: BUS_INTERFACE_STANDARD IRP_MN_READ_CONFIG, IRP_MN_WRITE_CONFIG • _OSC indicates support by OS • ASL config access PCIConfig operation region extended to 4K • Segments support • SAL support for Itanium

5. Enhanced Configuration Space: Considerations • Firmware considerations • Use MCFG table to communicate memory mapped config base to Longhorn • _OSC method indicates segment and ASL supportby OS • See PCI Firmware Specification V3.0 at http://www.pcisig.com/specifications/conventional/pci_bios • Driver considerations: • Drivers must NOT use CFC/CF8 • Driver must use documented interfaces for configuration space access

6. PCI Express Registers • PCI Express Capability registers • Longhorn will comprehend and program registers • Devices with capability version not understood by OS (1 is the only currently defined value) will be treated like a PCI device • Save/Restore of PCI Express Configuration Settings • Save and restore PCIe capability settings and registers across power events • BIOS Configurations • BIOS configurations should be maintained on Longhorn • Not always possible

7. Device Serial Numbers • Recommended for removable devices • Must be unique per device, same for functions okay • Allow device setting migration from port to port • Identifying new device in same port with a different serial number needs a remove/reinstall • Required for combo PCIe/USB ExpressCards

8. PCI Express Hierarchy • Hierarchies must follow PCIe specification – device type tells what it is • Violation will assert and not have PCIe features enabled • Violations involve root complex and switches • No switches or endpoints in root complex • Virtual PCI-to-PCI bridges must be implemented in switches • No integrated devices appearing on internal bus of switches

9. Implementation Considerations • Firmware considerations: • Some base features, such as Max Payload Size/Max Read Request Size, should be initially set by firmware • OS may override firmware settings • Driver considerations: • Drivers will not be allowed to modify base feature registers • Hardware considerations: • Pay attention to switch implementation and PCIe hierarchy • Be sure to properly support TransactionsPending • Serial numbers required for certain classes of devices

10. Message Signaled Interrupt (MSI) • Kernel updates to improve interrupt handling and support • Both MSI/MSI-X fully supported: • Reduces problems associated with interrupt sharing • MSI/MSI-X preferred over INTx emulation for PCIe • Class driver support: • VideoPort • StorPort • NDIS • Others in future • Windows Driver Foundation fully supports MSI/MSI-X

11. MSI: Considerations • Driver considerations: • Driver changes required to use MSI/MSI-X • Whitepaper: http://www.microsoft.com/whdc/system/bus/pci/MSI.mspx • Hardware considerations: • Single vs. multiple messages: • Most devices support single message • Still requires a read to interrupt status register • Multiple messages • Different interrupt status use different messages • MSI-X does not require contiguous IDT entries or identical target processor sets across messages (very flexible)

12. Hot Plug • System interrupt signals Hot Plug event • Root/switch ports must implement MSI asthe system interrupt • Legacy INTx support is allowed as well • State machine supports all combinations of hardware elements • Supports orderly or surprise removal • ExpressCard can be surprise removed • Edge-style cards will issue OS request for removal

13. Hot Plug: Considerations • Firmware considerations: • Firmware implements _OSC for OS to take native control • See PCI Firmware Specification V3.0 • Driver considerations: • No additional driver requirements but drivers must be able to start and stop properly – test thoroughly! • Hardware considerations: • Special HW design considerations for compatibility with legacy OSs

14. PME • System interrupt signals PME event • Root/switch ports must implement MSI as the system interrupt • Legacy INTx support is allowed as well • PME event handling • OS interrupt handler reads Requester ID • Notifies driver of waking device of the wakeup event • Driver interface remains the same – Wait/Wake DDK rules don’t change • Fully supported by Windows Driver Foundation • ACPI PME notification highly discouraged • Increases firmware support and development costs • Firmware-directed PME doesn’t work in many cases and causes crashes

15. PME: Considerations • Firmware considerations: • Firmware implements _OSC for OS to take native control • See PCI Firmware Specification V3.0 • Hardware considerations: • Don’t share PME between PCI Express and PCI/PCI-X • Special HW design considerations for PME# routing for PCI Express to PCI/PCI-X bridges

16. Active State Power Management (ASPM) • Enables ASPM based on user power scheme and performance preference • Plans are still under consideration • Policy • System wide on/off • Per device on/off • Fixed per OS or user defined power scheme • Supports setting of ASPM on hot plugged devices

17. ASPM: Considerations • Firmware considerations: • Firmware can enable at POST if desired • OS may override firmware settings • Driver considerations: • Drivers will not be allowed to modify ASPM • Hardware considerations: • Devices must correctly indicate L0s/L1 exit and acceptable latencies

18. Advanced Error Logging and Reporting • Reporting and logging of correct and uncorrectable errors • I/O errors are big issues for server systems • Error logging and reporting a big part of diagnosis • Hardware vendors strongly encouraged to support this feature • Both chipsets and endpoint devices • Windows Longhorn Server will support AER via the Windows Hardware Error Architecture (WHEA) • Devices will be expected to implement support for AER • Windows expects to be able to “own” AER • PCI Express (via the PCI bus driver) will be a WHEA error source

19. Resource Considerations • The BIOS provides resource information for each PCI root bus. • Windows attempts to respect BIOS configuration of devices from these resources. When not possible, Windows chooses an acceptable location for these devices. • There is a growing trend towards 64-bit systems with large memory configurations and large number of PCI devices on every system. • This puts constraints on the lower region of the physical address space. Configure PCI Devices with memory resources above 4 GB

20. Memory Resource Assignment • Windows Server 2003 • A PCI device with BIOS configuration above 4GB, is always assigned resources from below the 4GB region • If no range below 4GB region is available, then the device is assigned a range above the 4GB boundary • This holds good even if the Windows OS cannot physically access the address range above 4GB • Longhorn • A memory address range above 4GB is available for PCI devices only if that range is physically accessible by the OS • Within this constraint, Windows will always attempt to respect the BIOS configuration on a PCI device. • A PCI device with 64-bit BARs and no BIOS configuration is still assigned a memory address range above 4GB as available on the parent

21. Hardware Design Recommendations • For Longhorn to assign memory resources to a PCI device above 4GB: • Entire device path from the PCI root bus to the device must support 64-bit memory BARs • A PCI device behind a PCI-PCI bridge can only have a Prefetchable memory window above the 4GB boundary • This constraint is imposed by the PCI-PCI bridge specification • PCI devices on root bus can have any memory type resources above the 4GB boundary Include Prefetchable BARs on PCI devices behind a bridge

22. BIOS Design Recommendations • The ACPI BIOS should describe the memory range above 4GB in the _CRS and/or _PRS of the PCI root bus • This is described using the QWord Address Space descriptor as defined in the ACPI spec (Section 6.4.3.5.1). • Windows will evaluate the _SRS method with a buffer in the same format as the _CRS/_PRS • The memory range for the PCI root bus should not overlap with the physical RAM or some other range • The memory range is required to be physically accessible by the processor/chipset • The ACPI BIOS should configure the resources on the PCI devices after evaluating the _OSI method to account for the Server 2003 behavior • The ACPI BIOS should return an appropriate buffer in the evaluation of the _CRS/_PRS on a PCI root bus to account for the Server 2003 behavior

23. PCI Express Features Not Supported in Longhorn • Extended VC/Isochronous • Not supported and also a potentiallogo failure issue • Slot Power Budgeting

24. Extended VC/Isochronous Is Not Supported: Potential Logo Failure! • Optional feature of PCI Express • Chipsets may or may not implement it • Devices cannot rely on chipset Isochronous support, and must be capable of using VC0! • No usage rules and latency guarantees • VCs are a shared resource • Different device assumptions may be incompatible • Designed For Windows Logo requirements in place: WLP 2.2 and 3.0 requirements disallow usage of Extended VC/Isochronous

25. Slot Power Budgeting • Why Not Supported: • No standard mechanism to convey overall system power budget to OS • Firmware can set slot power budget before booting OS • Future Direction: • Currently being evaluated

26. PCI Express 1.1 • PCI Express 1.1 contains • All PCI Express ECNs for 1.0a • All errata against 1.0a • Learnings of Design community for past 2 years • PCI Express 1.1 scheduled to be released 1H05

27. PCI Express 1.0a ECNs—Base Software Transaction Data Link Physical Mechanical

28. PCI Express 1.0a ECNs—Base Software Transaction Data Link Physical Mechanical

29. ECNs that are Important to Windows • Integrated Devices - Event Collector • Allows chipset integrated devices that are Express-like but sit on an internal bus to take advantage of Express features • Reset Limit Adjustment • Clarifications impact hot plug • MMCONFIG ECN • Critical for correct system operation • Windows will rely on systems and firmware being built to this spec

30. ECNs that are Important to Windows • MSI-X ECN for PCI Express • MSI-X offers great flexibility and benefits for many devices • Longhorn is planned to have full support • CRS Software Visibility • Windows my use this capability. More relevant to high end systems • Longhorn plan under investigation. • PR Acceptance Limit • Not software visible but very important clarifications • PCI Express Hot Plug • Longhorn hot plug implementation will be built to these updates

31. ECNs that are Important to Windows • PCI Express Base PCI Bus Power Management • Longhorn will utilize the no device reset • Platform reference clock power management • Not software visible but very important for ExpressCard implementations • Surprise Down Error • Critical piece of hot plug • All hot plug ports need to implement • Error Reporting • Important to systems that implement Advanced Error Reporting capability

32. Call to Action • Ensure you are up to date on the PCI Express 1.1 and PCI Firmware 3.0 specifications • Implement _OSC in your firmware for proper handoff of PCI Express Features to Windows Longhorn • Ensure your devices meet Designed for Windows Logo requirements • Ensure you are signed up for the Windows Longhorn Beta program • Regularly test • Provide Feedback • Send your hardware to our test labs

33. Community Resources • Windows Hardware & Driver Central (WHDC) • www.microsoft.com/whdc/default.mspx • Technical Communities • www.microsoft.com/communities/products/default.mspx • Non-Microsoft Community Sites • www.microsoft.com/communities/related/default.mspx • Microsoft Public Newsgroups • www.microsoft.com/communities/newsgroups • Technical Chats and Webcasts • www.microsoft.com/communities/chats/default.mspx • www.microsoft.com/webcasts • Microsoft Blogs • www.microsoft.com/communities/blogs

34. Additional Resources • Community Sites • http://www.microsoft.com/communities/default.mspx • List of Newsgroups • http://communities2.microsoft.com/communities/newsgroups/en-us/default.aspx • Attend a free chat or webcast • http://www.microsoft.com/communities/chats/default.mspx • http://www.microsoft.com/seminar/events/webcasts/default.mspx • Locate a local user group(s) • http://www.microsoft.com/communities/usergroups/default.mspx • Non-Microsoft Community Sites • http://www.microsoft.com/communities/related/default.mspx

35. Additional Resources • Email • Pciesup @ microsoft.com • Specs • PCI Express Base Specification v1.0a http://www.pcisig.com • PCI Firmware Specification v3.0 http://www.pcisig.com • Whitepapers • http://www.microsoft.com/whdc • PCI Express Overview • PCI Express Legacy Transition • PCI Express Longhorn Implementation • PCI-PCI Bridge Resource Issues • PCI Hot-Plug • PCI I/O Reduction • Related Sessions • Windows Hardware Error Architecture • Error Management Solutions Synergy with WHEA • PCI Express Trusted Configuration Space

37. © 2005 Microsoft Corporation. All rights reserved. This presentation is for informational purposes only. Microsoft makes no warranties, express or implied, in this summary.