Performance Analysis for VoIP System

1. Performance Analysis for VoIP System Members R94922009 周宜穎 R94922020 吳鴻鑫 R94922064 張嘉輔

2. Outline • What is Performance • Performance Bound • How to analyze Performance • Some Performance Analysis Exmaple

3. What is Performance ? [10] • There are numerous factors that affect the performance assessments. • Human factors • Device factors • Network factors

4. Human factors -Audiovisual Quality Assessment Metrics • Subjective quality assessment - MOS • Objective quality assessment • Signal-to-Noise Ratio (SNR) • Mean Square Error (MSE) • Perceptual Analysis Measurement System (PAMS) • Perceptual Evaluation of Speech Quality (PESQ) • E -model • E-model • R-scale ( 0 to 100 ) <=> MOS rankings and User Satisfaction

5. Human factors Voice Quality Classes

6. Device factors • Essential devices such as • VoIP endpoints • Gateways • MCUs (Multipoint Control Units ) • Routers • Firewalls • NATs (Network Address Translators ) • Modems • Operating System • Processor • memory

7. Network factors • Network congestion • Link failures • Routing instabilities • Competing traffic • General Measruement : • Delay • Jitter • Packet loss

8. The Performance Standard • Delay • Good (0ms-150ms) • Acceptable (150ms-300ms) • Poor (> 300ms) • Jitter • Good (0ms-20ms) • Acceptable (20ms-50ms) • Poor (> 50ms). • Loss • Good (0%-0.5%) • Acceptable (0.5%-1.5%) • Poor (> 1:5%)

9. E-model • ITU-T recommendation • Well established computational model • Using Transmission parameters to predict the quality • We can get the basic Performance Standard by through the model

10. Basic formula for the E-model • R-value = Ro - Is - Id - Ie + A • Ro • the basic signal-to-noise ratio based on sender and receiver loudness ratings and the circuit and room noise • Is • the sum of real-time or simultaneous speech transmission impairments,e.g. loudness levels, sidetone and PCM quantizing distortion • Id • the sum of delay impairments relative to the speech signal, e.g., talker echo, listener echo and absolute delay • Ie • the equipment impairment factor for special equipment, e.g., low bit-rate coding (determined subjectively for each codec and for each % packet loss and documented in ITU-T Recommendation G.113) • A • the advantage factor adds to the total and improves the R-value for new services.

11. Estimating the R • R = (Ro − Is) − Id − Ie + A • Ro , Is • do not depend on network environment • Id • This the Argument of Delay • Ie • It mostly affect by codec and packet loss • A • Additional adjust argument ,not considered in general

12. Estimating Id and Ie • Id = Idte + Idle + Idd • Idte -Talker echo delay • Idle - Listener echo delay • Idd - Long delay • Ie • It base on codec, but packet loss affect can be emulated as a function

13. Estimating Id and Ie • The distortion as a function of packet loss also depends on whether or not PLC (Packet Loss Concealment) • increases 4 units for codecs with PLC (in the R scale per 1% packet loss) • 10 units for codecs without PLC

14. Curve Diagram

15. Test Setup • Using 9 scenarios to test 27 possibilities • Using NISTnet network emulator • (http://snad.ncsl.nist.gov/itg/nistnet/) • create the various network health scenarios

16. MOS Vs Delay

17. MOS Vs Jitter

18. MOS Vs Loss

19. Normalized Each unit in the normalized scale corresponds to delay : 150ms jitter : 20ms loss : 0.5%.

20. The Conclusion about Performance bounds • We show that end-user perception of audiovisual quality is more sensitive to the variations in end-to-end jitter than to variations in delay or loss • We get a simple standard about the Performance to estimate Performance

21. How to Analyze Performance • Thinking about two topic • Measurement • Network Condition • Measurement mean the analysis model that estimate key parameters • Of course, it is the way to compute delay, jitter ,packet loss

22. Two Measurement [7],[8],[9] • There are two methods in performance measurement • passive measurement • records and analyzes existing traffic. • active measurement • Inject sample packets into the network.

23. Introduce a simple Measure • Measurement Method in LAN • sends sequences of UDP packets to unlikely values of destination port numbers (larger than 30,000) • This causes the destination host’s UDP module to generate an ICMP port unreachable error when the datagram arrives

24. ICMP • TCP/UDP/IP 協定若有錯誤情形發生時，會利用 Internet Control Message Protocol（ICMP）協定來送錯誤訊息 。 • 在 ICMP 的 type 中，目前約有 15 種 • The ICMP echo mechanism should be installed in host in the measurement

25. RTT of one sent packet

26. How to Compute? • Ti = Bi / v + Di /v + CL + C • Ti − CL =(Bi + Di) / v + C.

27. Keep estimating • one-way delay (T i ) • T i = (Ri − Si) −Di / v −CL/2− C/2 • This calculation assumes that all delay happens on the sending path. • J i,i+1 = (T i+1 − T i ) • Packet loss = packetslost / packetssent

28. How about more complicated? • Precision timestamping • Queuing Model • Special Model for Protocol or device • Seem to Traffic Analysis!?

29. Ex: SIP Traffic Model [11] • A model for SIP Traffic • Two Sub Model • IP Path Model • SIP Finite State Machine

30. FSH Notation • Q = State set • M = fixed number of sessions • C = the bottleneck transmission rate( bit/s) • R = total capacity of IP Path measure in packets of D bits • rtt = round trip time measured in seconds • p = probability of 3xx Response • ps = successful probability of packet transmission

31. Sample Computation • Call Dropping rate pcd

32. Enviroment condition for VoIP performance [4] , [5] • The aspects about VoIP Performance Analysis • Protocols • H.323 v.s. SIP • Network • Ethernet network v.s. wireless LAN (WLAN) network • Security for VoIP Communication • VPN protocols : PPTP v.s. IPSec

33. Delay in Ethernet Network • Both SIP and H.323 incurred higher delays in secure network-to-network environment. SIP H.323

34. Jitter in Ethernet Network • IPSec produced the highest jitter values for both H.323 and SIP communications.

35. Jitter in Wireless-LAN • IPSec-based VoIP communications generally incurred the highest jitter values.

36. Packet Loss Rates • IPSec and PPTP increased the packet loss rate in both Ethernet and WLAN. SIP H.323

37. Performance in Satellite Network [1] • Also provides IP-base data services • For remote region • As backup links

38. The purpose • The performance under • Delay • Random errors , burst errors • Link loading • Two codecs • 8 kb/s G.729 • 6.3/5.3 kb/s G.723.1

39. Test bed configuration

40. Baseline Tests • Bandwidth and bandwidth efficiency • Environment • No background traffic • No error • Link delay set 270ms • Run 15min with all 24 channel

41. Bandwidth Efficiency 5 5

42. Bandwidth • A single channel

43. Link Errors Tests • Random Error Tests and burst Error Tests • BERs (bit error rates) = BD/(B+GC) • Burst length (B) • Burst density (D) • Gap length (G) • Link capacity kb/s (C)

44. Random Error Tests

45. Burst Error Tests

46. Link Loading Tests • Environment • With different link loading levels • Link errors or not • Packet loss • Packet delay

47. Tests with an Error-Free Link

48. Tests with an Error-Free Link

49. Tests with Errors • Combine effect of both link loading and link errors. • Error ↑,background traffic↓ link loading level↓  link loading level can’t be pre- determined