TimeGraph : GPU Scheduling for Real-Time Multi-Tasking Environments

1. TimeGraph:GPU Scheduling for Real-Time Multi-Tasking Environments XiaozhenLi

2. Agenda The Project Introduction System Model TimeGraph System Architecture TimeGraph GPU Scheduling TimeGraph GPU Reservation Evaluation

3. The Project

4. What is TimeGraph • TimeGraph is a GPU command scheduler, running as part of GPU device driver, to protect GPU applications from performance interference. • The main contributions: • Priority-driven scheduling of GPU commands issued from user-space runtime • GPU resource reservation

5. Project • Main Contributors: • ShinpeiKato:Carnegie Mellon University • KarthikLakshmanan:Carnegie Mellon University • Ragunathan (Raj) Rajkumar:Carnegie Mellon University • Project Main Page: • http://rtml.ece.cmu.edu/projects/timegraph/

6. Related Publications • TimeGraph: GPU Scheduling for Real-Time Multi-Tasking Environments (USENIX ATC'11) • Resource Sharing in GPU-accelerated Windowing Systems (RTAS'11) • Operating Systems Challenges for GPU Resource Management(OSPERT'11) • RGEM: A Responsive GPGPU Execution Model for Runtime Engines(RTSS‘11)

7. Introduction

8. Burgeoning of GPU Applications Real-time multi-tasking requirement

9. Problem Real-time aplications should be prioritized Important applications must be well-isolated from others

10. GPU Software Stack Important

11. What Happens in Multi-Tasking?

12. What Happens in Multi-Tasking?

13. Contributions Design, and implement a GPU scheduler to provide prioritization and isolation capabilities for GPU applications Adopts an event-driven scheduler model that synchronizes the GPU with the CPU in a responsive manner Two scheduling policies and two resource reservation policies

14. System Model

15. Scope and Limitations A generic multi-core CPU An on-board GPU Independent of libraries,compilers, and runtime engines Been designed and implemented for Nouveau Been ported to the PSCNV open-source driver

16. Driver Model • Some Important Words • Direct Rendering Infrastructure(DRI) • X Window System • Fermi • Tesla • GPU channels

18. TimeGraph System Architecture

19. TimeGraph System Architecture

20. Components • Scheduling policies • Predictable-Response-Time (PRT) • High-Throughput (HT) • reservation policies • Posterior Enforcement (PE) • Apriori Enforcement (AE)

22. Specification TimeGraph reads a apecification (/etc/timegraph.spec) every time a new GPU channel is allocated to a task. The format is <name:sched:resv:prio:C:T> Priority Assignment Admission Control

23. TimeGraph GPU Scheduling

24. Scheduling Policies---ThePredictable-Response-Time (PTR) When GPU is not idle, GPU commands are queued When GPU gets idle, GPU commands are dispatched

25. Scheduling Policies--- The High-Throughput (HT)policy When GPU is not idle, GPU commands are queued, only if priority is lower than current GPU context When GPU gets idle, GPU commands are dispatched

26. Interrupt Setting and Handling • Completion Notifier • Add NOTIFY command to generate an interrupt from the GPU to the CPU • Add the NOP command after the NOTIFY command • Add the SERIALIZE command right before the NOTIFY command

27. Interrupt Association: • Add the SET_REF command before the SERIALIZE command • Add the SERIALIZE command before the SET_REF command Finally GPU command group: SERIALIZE,SET_REF,SERIALIZE, NOTIFY, NOP. • Task Wake-Up

28. Conclusion PRT policy make the GPU command groups scheduled based on task priorities to make high-priority tasks responsive on the GPU. HT policy is suitable for such tasks that should execute as fast as possible.

29. TimeGraph GPU Reservation

30. Basic Idea • Each task is assigned a reserve that is represented by capacity C and period T. • Budget e is the amount of time that a task is entitled for execution. • Uses a popular rule for budget consumption • the budget is decreased by the amount of time consumed on the GPU, • Replenished by at most capacity C once every period T

31. Reservation Support– Posterior Enforcement (PE) Policy The command groups to be submitted to the GPU, if their budget is greater than zero The over-run penalty is imposed on the next budget replenishment. The budget for the next period is given by e= min(C, e+C)

32. Reservation Support–Apriori Enforcement (AE) Policy First predicts a GPU execution. The GPU command group can be submitted to the GPU, only if the predicted cost is no greater than the budget. The next replenishment amount depends on the predicted cost x and the currently-remaining budget e

33. Conclusion PE policy is aimed for light-weight reservation, allowing tasks to overrun out of their reserves to an extent. Should be Used when isolation is required, AE policy reduces reserve overruns by predicting GPU execution costs a priori at the expense of additional over-head. Should be used if extremely time-critical applications are concurrently executed on the GPU

34. Command Profiling • Prerequisite • The GPU applications Repeatedly create GPU command groups with the same methods and data sizes. • GPU execution costs depend highly on methods and data sizes, but not on data values

35. Command Profiling Average GPU execution cost Method

36. Command Profiling • Preemption Impact • Same GPU command group may consume very different GPU execution times • When the GPU contexts (channels) are switched, GPU execution times can vary • If the context is switched, TimeGraph will not update the average GPU execution cost. Saves the difference between the actual GPU execution time and the average GPU execution cost as the preemption impact. keeps updating it and consider it when measure the predicted cost

37. Evaluation

38. Experimental Conditions • Conditions: • NVIDIA GeForce 9800 GT; 1 GB of video mem-ory; Linux 2.6.35 kernel; Intel Xeon E5504 CPU; 4 GB of main memory • GPU-intensive workloads: OpenGL 3-D games • periodic workload: Mplayer • regularly-behaved workload: the Gallium3D Engine demo program • Misbehaving workload: GPU commandbomb • Using SPECviewperf to evaluate the throughput

39. Coarse-grained Performance

40. Coarse-grained Performance

41. Extreme Workloads

42. Performance Regulation

43. Fine-grained Performance

44. Interference Issues

45. Periodic Behavior

46. GPU Execution Cost Prediction

47. Overhead and Throughput

48. Thank You