Graphics Processing Unit (GPU)- An Introduction

1. Graphics Processing Unit (GPU)- An Introduction Pankaj Nathani Ayusman Sarangi Adobe Systems Inc.

2. About Us • Pankaj Nathani • Computer Science, BITS Pilani, 1998-2002 • Software Engineer, 2002-2005 (2 years at Adobe India) • MS Computer Science, University of Utah, 2003-2005 • Recently joined Adobe After Effects group in Seattle, US • Ayusman Sarangi • Biological Science with Computer Science BITS Pilani 1997-2003 • Software Engineer at Adobe India from Oct 2003 • Worked on Adobe Products like Premiere, After Effects

3. Outline • GPU for Image Processing • Introduction to GPU Programming • Few Active Research Areas • Scientific Visualization, GPGPU, Image Processing on GPU, Raytracing • Apping • Top Univs in Graphics, Important Points • Resources/References • Q n A

4. GPU for Image Processing • CPU vs GPU: The Big Fight • Processing Power leading to Real-time Performance • CPU Annual Growth ~ 1.5x (Moore’s law) • GPU Annual Growth > 2.0x • General Purpose vs. Specialized Programming.

5. GPU for Image Processing -Where all we See? • Video Games • GPU Accelerated ImageProcessing in Most Image and Video Processing Softwares like AfterEffects/ Premiere/Apple Motion. • Apple’s CoreImage • Vista taking GPU to next level on Windows.

6. Introduction to GPU Programming Graphics Pipeline TO CPU GPU CPU Application Transform (Vertex Processor) Rasterizer Color (Pixel Processor) VideoMemory(Textures) Display Finalpixels(Color, Depth) Vertices (3D) Fragments(pre-pixels) Transformed Vertices (2D) Render-to-texture • Programmable Graphics Pipeline • Programmable Vertex Processor • Programmable Pixel Processor uniform sampler2D input_texture; void main( void ) { vec4 in_color; in_color = texture2D(input_texture, gl_TexCoord[0].st ); // Process the Pixel Here gl_FragColor = in_color; }

7. Introduction to GPU Programming • Programming Environment • CPU <-> GPU Communication • DirectX • OpenGL • Adding Programmability for Vertex/Fragment • GLSL • HLSL • Cg • Sh/Brooks

8. Introduction to GPU Programming-Texturing Output – No Lights Input (3D Model) + Texture Output – With Lights

9. Introduction to GPGPU • GPGPU stands for General-Purpose computation on GPUs . • GPU beyond Games/3D Effects • GPGPU as capable CoProcessor • Image Processing • Audio/Signal Processing • Database • Bioinformatics

10. Scientific Visualization • It is the graphical representation of data as a means of gain understanding and insight into the data. • Challenge: Interactive display and analysis of data. • Applications: Fluid Dynamics, Medical Imaging, Data Mining, Simulation • Interesting Fact: Boeing used flow visualization to understand aerodynamics for their 777 line of planes.

11. Courtesy: CSAFE @ Utah

12. GPGPU • The power and flexibility of GPUs makes them an attractive platform for general-purpose computation. • Researchers have found that exploiting the GPU can accelerate some problems by over an order of magnitude over the CPU.

13. GPGPU (Contd) • Applications: a. Game Physics - Havok FX b. Protein Folding - Folding@Home (Stanford project) • Multiple vendor GPGPU initiatives: a. CUDA By NVIDIA (Compute Unified Device Architecture) b. CTM By AMD (Close To Metal)

14. Image Processing on GPU • With the advent of GPUs, IP community has been focusing on utilizing parallel processing power of GPUs to run complex IP algorithms faster. (Ex – Gaussian Blur, Image segmentation) • Adobe Labs will soon publish an SDK to write Image Processing algorithms which can run on both GPUs and multi-cores.

15. Ray Tracing • Ray Tracing is a global illumination based rendering method. It traces rays of light from eye towards the scene and those rays are tested for intersection with objects in the scene. Positive: Generates Realistic Images Effects like Reflection, Refraction, Shadow are easy to generate. Negative: Compute Intensive

16. Boeing 777 Engine: 350 million triangles Interactively rendered using Manta Interactive Ray Tracer @Utah

17. Study Table rendered by my Raytracer

18. Interesting Fact: Pixar used Raytracing to render Cars in their movie ‘Cars’

19. Top Univs in Graphics (US News 2002) • University of North Carolina - Chapel Hill • Stanford University Massachusetts Institute of Technology • Georgia Institute of Technology • University of Utah • Brown University • Carnegie Mellon University • U of California - Berkeley • University of Washington • Cornell University

20. Apping – Important Points • Project your knowledge in imaging & graphics areas through courses and research projects done. • Relevant work in PS-I will help. • Try to publish IP/graphics paper(s) in good Conferences/Journals. • Industry experience in imaging/graphics areas can be helpful in apping. • PS-II: Pick companies where you can work in your areas of interest.

21. Resources • IEEE and ACM websites. • Important Journals: a. IEEE Transactions on Image Processing b. IEEE Transactions on Visualization & Computer Graphics c. ACM Transactions on Graphics (TOG)

22. References • http://labs.adobe.com • Adobe Labs: New & emerging technologies from Adobe. • www.opengl.org • Official Homepage for OpenGL. • http://developer.nvidia.com • NVIDIA Developer’s website: Product information, samples, tools, and white papers for GPU developers. • http://ati.amd.com/developer • ATI Developer’s website • www.codesampler.com • Lots of free code for OpenGL/DirectX explaining basic concepts like Transformation/Texturing etc • www.gpgpu.org • Reference for GPGPU work.

23. Graphics clients • Visual Effects – Pirates of the Caribbean (ILM), SpiderMan (Sony) • Animated movies – Ratatouille (Pixar), Happy Feet (Animal Logic) • Games – Xbox 360, Sony Playstation 3

24. Graphics everywhere • Graphics is becoming a part of our life. Be it animated movies, television, games… • It is a result of rapid innovations happening in this field. • Bridging the gap between fantasy and reality. • Helps in building an interactive environment.