Queuing Theory For Dummies

1. Queuing TheoryFor Dummies Jean-Yves Le Boudec 1

2. All You Need to Know About Queuing Theory • Queuing is essential to understand the behaviour of complex computer and communication systems • In depth analysis of queuing systems is hard • Fortunately, the most important results are easy • We will study only simple concepts 2

3. 1. Deterministic Queuing • Easy but powerful • Applies to deterministic and transient analysis • Example: playback buffer sizing 3

4. Use of Cumulative Functions 4

5. Solution of Playback Delay Pb bits A(t) A’(t) D(t) d(t) (D2): r (t - d(0) - D) (D1): r(t - d(0) + D) time d(0) - D d(0) d(0) + D A. 5

6. 2. Operational Laws • Intuition: • Say every customer pays one Fr per minute present • Payoff per customer = R • Rate at which we receive money = N • In average λ customers per minute, N = λ R 6

7. Little Again • Consider a simulation where you measure R and N. You use two counters responseTimeCtr and queueLengthCtr. At end of simulation, estimate R =responseTimeCtr / NbCust N = queueLengthCtr/ Twhere NbCust = number of customers served and T=simulation duration • Both responseTimeCtr=0 and queueLengthCtr=0 initially • Q: When an arrival or departure event occurs, how are both counters updated ?A: queueLengthCtr += (tnew - told) . q(told) where q(told) is the number of customers in queue just before the event.responseTimeCtr += (tnew - told) . q(told)thus responseTimeCtr == queueLengthCtr and thusN = R x NbCust/T ; now NbCust/T is our estimator of  7

8. Other Operational Laws 8

9. The Interactive User Model 9 9

10. Network Laws 10

11. Bottleneck Analysis • Example • Apply the following two bounds (1) (2) 17 11

12. Throughput Bounds 12

13. Bottlenecks A 13

14. DASSA • Intuition: within one busy period: to every departure we can associate one arrival with same number of customers left behind 14

15. 3. Single Server Queue 15

16. i.e. which are event averages (vs time averages ?) 16

17. 17

18. 18

19. 19

20. Non Linearity of Response Time 20

21. Impact of Variability 21

22. Optimal Sharing • Compare the two in terms of • Response time • Capacity 22

23. The Processor Sharing Queue • Models: processors, network links • Insensitivity: whatever the service requirements: • Egalitarian 23

24. PS versus FIFO • PS • FIFO 24

25. 4. A Case Study • Impact of capacity increase ? • Optimal Capacity ? 25

26. Methodology 26

27. 4.1. Deterministic Analysis 27

28. Deterministic Analysis 28

29. 4.2 Single Queue Analysis Assume no feedback loop: 29

30. 4.3 Operational Analysis • A refined model, with circulating users • Apply Bottleneck Analysis ( = Operational Analysis ) waiting time 1/c Z/(N-1) -Z 30

31. 31

32. 32

33. 5. Networks of QueuesStability • Queuing networks are frequently used models • The stability issue may, in general, be a hard one • Necessary condition for stability (Natural Condition) server utilization < 1 at every queue 33

34. Instability Examples • Poisson arrivals ; jobs go through stations 1,2,1,2,1 thenleave • A job arrives as type 1, thenbecomes 2, then 3 etc • Exponential, independent service times withmeanmi • Priorityscheduling • Station 1 : 5 > 3 >1 • Station 2: 2 > 4 • Q: Whatis the naturalstability condition ? • A: λ (m1 + m3+ m5) < 1 λ (m2 + m4) < 1 34

35. λ = 1m1 = m3 = m4 = 0.1 m2 = m5 = 0.6 • Utilization factors • Station 1: 0.8 • Station 2: 0.7 • Network is unstable ! • If λ (m1 + … + m5 ) < 1 network is stable; why? 35

36. Bramson’s Example 1: A Simple FIFO Network • Poisson arrivals; jobs go through stations A, B,B…,B, A then leave • Exponential, independent service times • Steps 2 and last: mean is L • Other steps: mean is S • Q: What is the natural stability condition ? • A: λ ( L + S ) < 1λ ( (J-1)S + L ) < 1 • Bramson showed: may be unstable whereas natural stability condition holds

37. Bramson’s Example 2A FIFO Network with Arbitrarily Small Utilization Factor • m queues • 2 types of customers • λ = 0.5 each type • routing as shown, … = 7 visits • FIFO • Exponential service times, with mean as shown S S L S S L S S L S S L • Utilization factor atevery station ≤ 4 λS • Network isunstable for S ≤ 0.01L≤ S8m = floor(-2 (log L )/L) 37

38. Take Home Message • The naturalstability condition isnecessary but may not besufficient • There is a class of networks wherethisneverhappens. Product Form Queuing Networks 38

39. Product Form Networks • Customers have a class attribute • Customers visit stations according to Markov Routing • External arrivals, if any, are Poisson 2 Stations Class = step, J+3 classes Can youreduce the number of classes ? 39

40. Chains • Customers can switch class, but remain in the same chain ν 40

41. Chains may be open or closed • Open chain = with Poisson arrivals. Customers must eventually leave • Closed chain: no arrival, no departure; number of customers is constant • Closed network has only closed chains • Open network has only open chains • Mixed network may have both 41

42. 3 Stations 4 classes 1 open chain 1 closedchain ν 42

43. Bramson’s Example 2A FIFO Network with Arbitrarily Small Utilization Factor S S L S S L S S L S S L 2 Stations Many classes 2 open chains Network is open 43

44. Visit Rates 44

45. 2 Stations 5 classes 1 chain Network is open Visit ratesθ11 = θ13 =θ15 =θ22 =θ24 =λ θsc = 0 otherwise 45

46. ν 46

47. Constraints on Stations • Stations must belong to a restrictedcatalog of stations • See Section 8.4 for full description • Wewillgivecommonlyusedexamples • Example 1: Global Processor Sharing • One server • Rate of server issharedequallyamong all customerspresent • Service requirements for customers of class c are drawniidfrom a distribution whichdepends on the class (and the station) • Example 2: Delay • Infinitenumber of servers • Service requirements for customers of class c are drawniidfrom a distribution whichdepends on the class (and the station) • No queuing, service time = service requirement = residence time 47

48. Example 3 : FIFO with B servers • B servers • FIFO queueing • Service requirements for customers of class c are drawniidfrom an exponential distribution, independent of the class (but maydepend on the station) • Example of Category 2 (MSCCC station): MSCCC with B servers • B servers • FIFO queueingwithconstraints Atmost one customer of each class isallowed in service • Service requirements for customers of class c are drawniidfrom an exponentialdistribution, independent of the class (but maydepend on the station) • Examples 1 and 2 are insensitive (service time canbeanything)Examples 3 and 4 are not (service time must beexponential, same for all) 48

49. A • Say which network satisfies the hypotheses for product form B (FIFO, Exp) C (Prio, Exp) 49

50. The Product FormTheorem • If a network satisfies the « Product Form » conditions givenearlier • The stationarydistrib of numbers of customerscanbewrittenexplicitly • It is a product of terms, whereeachtermdependsonly on the station • Efficient algorithmsexist to compute performance metrics for evenvery large networks • For PS and Delay stations, service time distribution does not matterotherthanthroughitsmean(insensitivity) • The naturalstability condition holds 50