1 / 12

World Telecommunications Congress 2006 30 April - 3 May 200 6 , Budapest, Hungary

Perceptual QoS Evaluation Model for Audiovisual Communication Services. T. Hayashi , K. Yamagishi, and H. Yoshino NTT , Japan. World Telecommunications Congress 2006 30 April - 3 May 200 6 , Budapest, Hungary. Outline. Goal of our study Concept of perceptual QoS evaluation model

Philip
Download Presentation

World Telecommunications Congress 2006 30 April - 3 May 200 6 , Budapest, Hungary

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. Perceptual QoS Evaluation Model for Audiovisual Communication Services T. Hayashi, K. Yamagishi, and H. Yoshino NTT, Japan World Telecommunications Congress 200630 April -3 May 2006, Budapest, Hungary

  2. Outline • Goal of our study • Concept of perceptual QoS evaluation model • Subjective quality assessment method • Proposed model • Effect of individual audiovisual quality • Effect of audiovisual delay • Conclusion World Telecommunications Congress 200630 April -3 May 2006, Budapest, Hungary

  3. Goal To find answers to the following questions: • How can we evaluate, design, and manage users’ perceptual QoS in order to provide comfortable audiovisual communication services? • How can users assess overall quality of audiovisual communication services, taking into account the balance among individual audiovisual quality, delay, and media synchronization? World Telecommunications Congress 200630 April -3 May 2006, Budapest, Hungary

  4. Perceptual QoS evaluation model Audio qualityestimation function Auditory source MOSA Multimedia quality integration function Multimedia quality (MOSMM) Audio delay (DA) Video delay (DV) DA DV Visual source Video quality estimation function MOSV Target of this study Task MOS: Mean opinion score World Telecommunications Congress 200630 April -3 May 2006, Budapest, Hungary

  5. Subjective quality assessment (1/3) • Experimental system 17-inch PC monitor 10-inch video window Camera Headphone Network emulator Microphone Desktop PC Audio/video IP packet loss rate and one-way audio/video IP packet transmission delay were controlled. Speech coding: G.722 (64 kbps) Video cording: MPEG4 part 2 (4 Mbps, VGA, 30 fps) World Telecommunications Congress 200630 April -3 May 2006, Budapest, Hungary

  6. Subjective quality assessment (2/3) • Experimental conditions 60 conditions: combinations of packet-loss rate and delay • Task • We used the task referred to as the Name-Guessing Task in ITU-T Rec. P.920. • The interactivity of conversation in this task was similar to a free conversation. World Telecommunications Congress 200630 April -3 May 2006, Budapest, Hungary

  7. Subjective quality assessment (3/3) • Assessment method • ACR (Absolute category rating) method • Subjects talked for one minute per condition and evaluated the quality using the five-grade quality rating scale. 5: Excellent, 4: Good, 3: Fair, 2: Poor, and 1: Bad • Each subjective quality was represented as a MOS calculated by averaging the scores of 32 subjects (non-experts). • The audio quality: MOSA, video quality: MOSV, and overall multimedia quality: MOSMM were individually evaluated in different rating sessions. World Telecommunications Congress 200630 April -3 May 2006, Budapest, Hungary

  8. Audiovisual quality estimation Audiovisual qualityMOSAVcan be evaluated from MOSA, MOSV, and a multiplicative interaction term. MOSAV: MOSMM for DA=DV=167 ms 4.0 CD: 0.96 RMSE: 0.14 MCI: 0.31 RMSE < MCI 3.0 Subjective MOSAV 2.0 CD: Coefficient of Determination RMSE: Root Mean Square Error MCI: Mean of 95% Confidence Interval for subjective MOS 1.0 1.0 2.0 3.0 4.0 Estimated MOSAV World Telecommunications Congress 200630 April -3 May 2006, Budapest, Hungary

  9. Effect of audiovisual delay • Influence of quality degradation for media synchronization was larger than that for delay. • Quality degradation at DA < DV was larger than that at DA > DV. 1200 MOSD: MOSMM without IP packet loss 1000 800 2.6 One-way video delay: DV [ms] 600 2.8 Experimental conditions 3.0 400 3.2 3.4 200 200 400 600 800 1000 1200 One-way audio delay: DA [ms] World Telecommunications Congress 200630 April -3 May 2006, Budapest, Hungary

  10. Estimation for quality of audiovisual delay Quality degradation due to audiovisual delay can be estimated by functions considering absolute audiovisual delay and differential delay. MOSD: MOSMM without IP packet loss 4 CD: 0.87, RMSE: 0.16, MCI: 0.32 3 Subjective MOSD DA = DV DA < DV DA > DV RMSE < MCI 2 1 1 2 3 4 Estimated MOSD World Telecommunications Congress 200630 April -3 May 2006, Budapest, Hungary

  11. Conversational quality estimation Multimedia quality MOSMM can be accurately evaluated from MOSAV, MOSD, and their interaction term. CD: 0.90 RMSE: 0.16 MCI: 0.31 4 3 Subjective MOSMM RMSE < MCI 2 1 1 2 3 4 Estimated MOSMM World Telecommunications Congress 200630 April -3 May 2006, Budapest, Hungary

  12. Conclusion • Perceptual QoS evaluation model for audiovisual communication services is proposed. • This model takes into account not only the individual audiovisual qualities but also the audiovisual delay and media synchronization. • The evaluation error of the proposed model was less than the statistical reliability of the subjective score, so the quality evaluation accuracy was sufficient for practical use. • Evaluating the effects of tasks and experimental parameters on the coefficients of the proposed model are for further study. World Telecommunications Congress 200630 April -3 May 2006, Budapest, Hungary

More Related