Modern Game Development

1. Modern Game Development Oliver Franzke & Javier BlazquezLucasArts Stanford University February 6th, 2009 ofranzke@lucasarts.com jblazquez@lucasarts.com

2. Agenda • Introduction • Facts • ‘Next-gen’ graphics • Challenges • Realistic Rendering • Asset pipeline • Multithreaded games • Parallel game loop approaches

3. Introduction • Disclaimer This talk is not specifically about LucasArts! It is an attempt to present typical challenges games companies have to deal with as well as common processes and techniques used in the industry. Please be aware that every company is unique and will therefore handle things differently.

4. Introduction • Modern Game Development Facts • Timeframe: 2 – 4 years • Team: 100+ (peak) • Budget: 15+ million $ • Number of source code lines: ~1 million • Code build times: up to 30 minutes • Size of assets: 100+ GB • Units sold: 2 – 3 million [VGChartz]

5. Art Lead Concept Character Environment Animation Effects Introduction • Team structure Project Lead Engineering Design Sound/Music Lead Lead Quality Assurance Gameplay System AI Level PR HR Core Narrative Graphics Finance Tools Network Technical Artist Production Producer Product Manager

6. Introduction • Timeline (3 year production) • Concept phase (1 – 2 month) • Develop ideas and prototypes • Pre-production (18 month) • Prove all game-play features • Prepare engine and tools for production • Production (12 month) • Create final code and content • Finaling (4 - 5 month) • Remove bugs and polish • Support • Customer support, patch development and updates (DLC)

7. ‘Next-gen’ graphics • Introduction • Realistic rendering • Data pipeline Image courtesy of Crytek GmbH

8. Challenges • Consumer expectations • High resolution models • Detailed textures • Huge environments • Rendering speed • HD resolution: up to 1920 x 1080 • At least 30Hz (goal is 60Hz)

9. Challenges • Platform diversity • Next-gen consoles (Xbox 360, Playstation 3) • Handhelds (Nintendo DS, PSP, iPhone) • High-end PCs • Assets, assets, assets… • Increased development complexity • Specialized code and art for every platform

10. Comparison • TFU character complexity • Main character (Apprentice) • ~15.000 triangles • 3 x 1024² textures • Other characters: 7k – 10k • Comparison • Quake3: 800 – 900 (1999, id Software) • Half Life 2: 4k – 8k (2004, Valve) [Stirling07] • Gears of War: 10k – 15k (2006, Epic) [Stirling07]

11. Comparison • TFU character complexity • Main character (Apprentice) • ~15.000 triangles • 3 x 1024² textures • Other characters: 7k – 10k • Comparison • Wolfenstein 3D: 2 (1992, id Software) • Quake3: 800 – 900 (1999, id Software) • Half Life 2: 4k – 8k (2004, Valve) [Stirling07] • Gears of War: 10k – 15k (2006, Epic) [Stirling07]

12. Comparison • TFU level complexity • 500k triangles per frame in opaque pass • Memory limitation • Next sector is streamed in • Comparison [Stirling07] • Virtua Fighter 5: 100k – 300k (2006,Sega) • Lost Planet: ~500k (2007,Capcom)

13. Realistic Rendering • Goals • Imitation of light transport (kind of) • Create convincing images • Increased immersion • Solution • Solve the rendering equation • High quality approximation

14. Realistic Rendering • Rendering Equation [Kajiya86] [Wikipedia1]

15. Realistic Rendering • Rendering Equation [Kajiya86] [Wikipedia1]

16. Realistic Rendering • Rendering Equation [Kajiya86] [Wikipedia1]

17. Realistic Rendering • How hard can it be to solve? • Recursive integral equation • Spatial, temporal and spectral domain • Very hard!

18. Realistic Rendering • Constraints / Typical Simplifications • Must be controllable by artists!!! • Limited changes in the lighting condition: • BRDF is time independent: • Spatial relationships are static

19. Realistic Rendering • Solution • Distinguish between static / dynamic lighting and geometry • Pre-calculate static lighting information offline • No expensive calculations during run-time • Quick access • Low frequency lighting only • Memory intensive • Add dynamic lighting during run-time • No global lighting effects (local model) • Simplified lighting model

20. Realistic Rendering • Recap: Lightmaps

21. Realistic Rendering • ‘Radiosity Normal Mapping’ [McTaggart04] [Mitchell06] • Source™ Engine shading model • Developed by Valve for Half Life 2 Image courtesy of Valve

22. y z x Realistic Rendering • ‘Radiosity Normal Mapping’ • Use of basis function projection for lighting • Basis defined in tangent space Tangent space:

23. Realistic Rendering • ‘Radiosity Normal Mapping’ (Offline) • Project lighting into basis • Assumptions • Surface is smooth • BRDF is view independent (diffuse)

24. Realistic Rendering • ‘Radiosity Normal Mapping’ (Runtime) • Project perturbed normal into basis • Reconstruct incident lighting Normal map #1: Normal map #2:

25. Realistic Rendering • Lightmaps vs. ‘Radiosity Normal Mapping’

26. Realistic Rendering • Lightmaps vs. ‘Radiosity Normal Mapping’

27. Realistic Rendering • ‘Radiosity Normal Mapping’ • Low resolution sampling (Vertex, Lightmaps) • Easy to composite with dynamic lights • Works with other bump techniques • Only low-frequency diffuse lighting captured • More memory required (basis matrix vs. color)

28. Realistic Rendering • XNA demo is available (including source)!

29. +y -x -z +z +x -y Realistic Rendering • ‘Ambient Cube’ [McTaggart04] [Mitchell06] • Directional ambient lighting for objects • Simplified irradiance volume [Greger98]

30. Realistic Rendering • ‘Ambient Cube’ (Example) Ambient cube #2 Ambient cube #1

31. Realistic Rendering • ‘Ambient Cube’ (Example) Image courtesy of Valve

32. Asset Pipeline • ‘Next-gen’ challenge: Amount of assets • Asset-to-data transformation • Pre-calculation (lighting, PVS, nav mesh…) • Adaption for console hardware • Packing • Compression • Transformations depend on the game • Unique pipeline for every game

33. Asset Pipeline • Problem: Very time consuming • Solution: Build farm • Distribute asset transformation • Collection of dedicated machines • Server-client architecture • Pros and Cons • Speed-up • Incremental builds (fragment caching) • Scalable • Previewing assets can be complicated • Extra development cost

34. Conclusion • Complexity is increasing • Basis function projection used to improve image quality • HL2 2 basis, Spherical Harmonics, Wavelet • Amount of data is a big problem • Looking ahead • Sub-surface scattering • Facial animation techniques • GPU used for general purpose computations

35. Multithreading • Why a multithreaded game is important • Frame time on a typical game is very short • 16ms or 33ms max for everything • Can’t drop frames • Very noticeable, both visually and audibly • Might disrupt the player’s actions • Increasing amount of computation per frame • More precise physics • Realistic enemy behavior • Larger and more complex worlds

36. Multithreading One thread is no longer enough The console CPU’s single-threaded performance is not very good. You would be wasting ~80% of the CPU processing capacity. Your game wouldn’t live to expectations. Future trend is to add more cores, not to have faster ones.

37. CPU Comparison No out-of-order execution kills single-threaded performance

38. Multithreading Challenges What makes parallelizing games difficult: Many different kinds of processes: Graphics, Physics, Audio AI, Gameplay Lots of shared state: World geometry, game objects, navigation graphs, player state, etc. Game loop has traditionally been very sequential in nature.

39. Typical Game Loop Process Input Update Player Gameplay AI Animation Audio Physics Render

40. Parallelizing the Game Loop Several approaches: Coarse-grained multithreading Long processes on their own thread Task-based multithreading Short jobs scheduled dynamically Well-defined dependencies Hybrid

41. Coarse-grained Approach Move certain systems to their own thread Good candidates: Rendering, Physics, Audio Provide sync points with the main thread Retrieve computed data Update state Kick off next frame’s work Relatively straightforward to set up

42. Coarse-grained Loop Render Render frame 0 Render frame 1 Sync Main Input Player Logic AI Anim Audio Input Player Frame 1 Frame 2 Sync Physics Physics for frame 2 Physics for frame 3

43. Coarse-grained Approach Problems: Main thread modifying state mid-frame Defer state-changing operations Use a command queue for the system Main thread expecting results mid-frame Return last frame’s data Use a cache / double-buffering scheme Expose a deferred query API Might introduce off-by-one-frame issues

44. Coarse-grained Approach Problems (cont.): Worker thread calling back into game E.g. collision response Can happen at any time Queue callbacks for later processing Or make systems thread-safe Uneven hardware thread usage Difficult to schedule work properly E.g. audio thread usually idle Not the best suited for PS3

45. Task-based Approach Code is partitioned into small tasks Tasks operate on streams of regular data Well-defined inputs and outputs Limited access to other resources Tasks can be scheduled independently Ordering is defined by data dependencies

46. Task-based Approach Custom scheduler for running tasks Build / traverse dependency graph Schedule independent tasks in parallel Opportunity for additional simulations Cloth Fluids Best suited for PS3 / Xbox 360 architecture

47. Task-based Approach Picture from [Turner07]

48. Task-based Approach Problems: Typical game code is very difficult to partition in tasks Lots of dependencies, very order-sensitive Object-oriented designs do not help Many small objects Required data is in disparate memory locations Not very suitable for stream processing Focus is usually on maintainability, not performance

49. Conclusions Game must be parallelized as much as possible Including systems that were traditionally sequential Usually requires hand-tailored solutions OOP sometimes gets in the way

50. Q&A