1 / 25

ETSI Testing Projects ICT-OSA/Parlay Workshop Brazil, March 2006

ETSI Testing Projects ICT-OSA/Parlay Workshop Brazil, March 2006. Anthony Wiles Manager ETSI Protocol and Testing Competence Centre. The ETSI PTCC. Protocol and Testing Competence Centre In-house team of experts providing direct support to ETSI Technical Bodies (for over 12 years)

dolf
Download Presentation

ETSI Testing Projects ICT-OSA/Parlay Workshop Brazil, March 2006

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. ETSI Testing Projects ICT-OSA/Parlay WorkshopBrazil, March 2006 Anthony Wiles Manager ETSI Protocol and Testing Competence Centre

  2. The ETSI PTCC • Protocol and Testing Competence Centre • In-house team of experts providing direct support to ETSI Technical Bodies (for over 12 years) • Development of protocols and profiles and the application of modern specification techniques (35%) • ‘Standards Engineering’ • Validation of standards (5%) • E.g., testbed activities (IPv6, SIP/H.323) • Development of test specifications (60%) • Conformance tests • Interoperability tests • Network Integration tests • Others (PHY, performance …) • http://www.etsi.org/ptcc ETSI Testing Projects - ICT Workshop, Brazil March 2006

  3. Typical ETSI Test Specifications • Cellular: GSM and 3G (UMTS) terminals • Now includes IMS • WiFi: HiperMAN, HiperACCESS, WiMax • VoIP: H.323, SIP, SIGTRAN • Service Creation: OSA/Parlay (API, IDL) • IPv6: Core, Security, Mobility, v4-v6 • Cordless phones: DECT • Radio communications: TETRA, DMR • Access terminals: FSK, SMS • Broadband: ISDN, DSL • Smartcards: Readers, cards, security modules • Intelligent Transport Systems (ITS): DSRC • Future: More Security, more ITS, GRID ... ETSI Testing Projects - ICT Workshop, Brazil March 2006

  4. IPv6 Test SpecificationsSTF 276 • Joint ETSI – eEurope project • Supported by the ETSI membership • Mainly vendors • Require IPv6 test specifications using ETSI methods and test languages • Designed to be part of 3GPP and TISPAN NGN IPv6 testing strategy (where applicable) • European Commission: • Mandate for IPv6 device interoperability • Amplify Europe’s voice in IPv6 testing • Liaising with the IPv6 Logo Program • For more info visit: www.ipt.etsi.org ETSI Testing Projects - ICT Workshop, Brazil March 2006

  5. Two Phases (2005 – 2007) • Phase 1: Core IPv6 • Total value approx. 0,5M€ • Jan – Dec 2005 • Phase2: IPSec, Mobile IPv6, IPv4-6 Transitioning • Total value approx. 1,0M€ • Jan 2006 – May 2007 • Provided as • Funding • Provision of experts • Hosting costs • Travel • All technical work done at ETSI by STF276 • Currently 11 participants in the STF ETSI Testing Projects - ICT Workshop, Brazil March 2006

  6. Outputs • IPv6 Testing Framework • Published (update due end 2006) • IPv6 Requirements Catalogue • On line as part of IPT Library • Test specifications • Interoperability test specifications • Conformance test specifications • Library of TTCN-3 test code freely available • On-line, extensible and reusable • All tests are validated (probably 30% of the effort) • On ETSI IPv6 Testbed • By member companies • For more info visit: www.ipt.etsi.org ETSI Testing Projects - ICT Workshop, Brazil March 2006

  7. Results Phase 1 • Over 1100 Core IPv6 Requirements identified • Including 3GPP • 300 Core IPv6 conformance tests • fully validated • 120 Core IPv6 Interoperability Tests • fully validated • All these test specifications including TTCN-3 code are free and open source ETSI Testing Projects - ICT Workshop, Brazil March 2006

  8. STF276@Work ETSI Testing Projects - ICT Workshop, Brazil March 2006

  9. 3GPP UMTS (terminal) Test Specifications (STF 160) • Nobody can make money without interoperable handsets (UE)! • UE Interoperability towards access technologies is the key issue for mobility • Conformant UE = Interoperability (of UE in 3G networks) • Good experience from GSM • Solution: conformance testing • Very detailed test specifications • Based on the 3GPP core specs • Before commercial products enter the market • Voluntary declaration, but certified • One of the most important UE quality benchmarks ETSI Testing Projects - ICT Workshop, Brazil March 2006

  10. logging and analysis Mobile Under Test SS testing Executable Test Suite (e.g., C++) Test Report implementation compilation Methodology for 3G Testing Core specs 25.331, 25.321, 2 24.008, 44.018, 44.060 PICS (capability applicability) 34.123-2, 51.010-2 Test Purposes,34.123-1Descript (prose), 34.108, 51.010-1 Test Cases (TTCN) 34.123-3, 51.010-5 ETSI Testing Projects - ICT Workshop, Brazil March 2006

  11. 3G Test Industry Commitment is Vital • The TTCN test cases are implemented in the System Simulators (SS) on different platforms • Must secure the commitment of the SS vendors • SS vendors organise verification and regression testing • Each and every test case is validated • Bug-free tests have a high real value • Important to ensure that a TC will execute correctly even after updates • Early identification of clients who deploy the tests • Prioritising TCs • Scheduling the deployment • Certifications ETSI Testing Projects - ICT Workshop, Brazil March 2006

  12. Funding • Test specs developed at ETSI by STF160 • Duration of several years • Shared costs to ensure the funding • 3GPP funding from Organisation Partners • Main source, approx 1M€ (61 mm) /year) • Voluntary contributions from the marketing partners • Range from 65k€ - 150k€ (5 – 12 mm) /year • Man-power (free-of-charge) • Range from 15 - 18 mm /year ETSI Testing Projects - ICT Workshop, Brazil March 2006

  13. Typical 3GPP UMTS UE Tests • Protocol and signalling tests via radio interface • Largest part of the testing process • Full conformance tests • USIM functional tests • Identification, PIN, account charging, network priority, phonebook • UICC tests (SIM electrical test) • power on/off, 1.8v/3v detection, presence detection • bit/character transmitting duration, command processing and error handling • Acoustic tests • Loudness rating, idle noise, echo … • Radio Frequency & Radio Resource Management tests • Characteristics and performance tests of transmitter & receiver ETSI Testing Projects - ICT Workshop, Brazil March 2006

  14. Users of 3GPP Tests • Global Certification Forum (GCF) • PCS Type Certification Review Board (PTCRB ) • GCF + PTCRB are the major clients for W-CDMA (FDD) & GSM/GPRS test specs • TD-SCDMA Industry Alliance (TDIA) • TDIA is the major clients for TD-SCDMA (LCR TDD) test specs • GCF WI-10 selected 400 TCs as high priority • Divided in 4 packages • Progressing sequentially one by one (milestone 80%) • UMTS mobile certification will be started if 80% of each package are validated ETSI Testing Projects - ICT Workshop, Brazil March 2006

  15. Approval Process • Test cases are approved by e-mail • No quality compromise • Two week consultation period • TCs are approvedone by one • Each test must run on at least one SS platform against two independent commercial mobiles • Log files and radio conditions as support information for approval • Reason for this • TTCN TC, SS and UE are developed at the same time • None of them is perfect ETSI Testing Projects - ICT Workshop, Brazil March 2006

  16. Results • 400 TCs in 8 ATSs, 25 MB code • Platform independent • Running on > 4 platforms • We have reached the GCF 80% validated milestone • 3G mobile certification starting at beginning of 2005 • The light is green for mass 3G mobiles entering the markets • 4 types of 3G mobiles are ready to go • Begun the real 3G test business • 4 validated 3G SS platforms available in the markets • 4 test houses in Europe opening 3G test services • 3G TTCN experts becoming hot in the labour market • We know the price of each 3G coded TC • 10 k€ /TC • 3G investment for 4 years: 4 mil.€ for 400 TCs • All TTCN code can be downloaded free of charge • Code is continuously maintained and delivered • Every three weeks for 3G members • Every three months for everybody ETSI Testing Projects - ICT Workshop, Brazil March 2006

  17. VoIP-related Test SpecificationsSTF 269, STF 270 and others • SIP (RFC 3261) • Over 400 conformance test cases • Fully validated (executables available on commercial tools) • STF 270 is doing the maintenance • H.225 • About 300 conformance test cases • Fully validated (executables available on commercial tools) • No ongoing maintenance • SIGTRAN • Test purposes for M3UA, M2UA, SCTP • TTCN-3 available commercially (not ETSI) • IMS SIP (together with SDP) • 3GPP RAN5 (STF 160) • TTCN-3 will be used • Work started March 2006 • Based on the generic SIP test suite (see above) augmented to concentrate on 3GPP SIP specific requirements ETSI Testing Projects - ICT Workshop, Brazil March 2006

  18. ETSI – WiMAX Test Specifications STF 252 • WiMAX Forum Board meeting in January 2005 • Agreed to harmonize the WiMAX/HiperMAN base specifications • Agreed to harmonize the WiMAX/HiperMAN test specifications • Agreed to exploit ETSI’s testing expertise • Agreed to accelerate the work using the ETSI STFs • Agreed that validation of the test specifications needed • complex interplay of all parties involved • WiMAX Forum, ETSI BRAN, equipment vendors, ETSI STF, test tool developers, test labs, etc. • ETSI WiMAX Co-operation agreement signed in 2005 ETSI Testing Projects - ICT Workshop, Brazil March 2006

  19. Results • Wave 1: Joint test specifications approved and published • Completed Dec. 2005 • PICS – Protocol Implementation Conformance Statement • TSS&TP – Test Suite Structure and Test Purposes • ATS – Approx 60 test cases implemented, validated and used in certification • First WiMAX products have been certified • Wave 2: New drafts reaching approval • PICS – new updated version with corrections • TSS&TP – new draft with close to 1400 Test Purposes • ATS – new draft with close to 300 additional test cases developed and entering validation ETSI Testing Projects - ICT Workshop, Brazil March 2006

  20. IPT LibrarySTF 295 • IP Testing Library • Developed by ETSI TC MTS • All MTS IP and IP-related conformance and IOP tests • PICS, TPs, Test Suites etc. • Due to be published April 2006 • SIP, IPv6, H.323, H.248 • Includes IPv6 Requirements Catalogue • Linked to TPs and TTCN-3 code • Powerful search and selection possibilities • E.g., allows test suite tailored to specific search criteria ETSI Testing Projects - ICT Workshop, Brazil March 2006

  21. Screen Shot of IPT Library ETSI Testing Projects - ICT Workshop, Brazil March 2006

  22. In 6 years, 66 Events 3500 engineers 1400 Companies Usefulness & Service Satisfactionranked at 4.35 / 5 PlugtestsTMETSI Interoperability Service • Created in 1999, specialized in the organization oftest events • Open to all organizations • Experts from ETSI and other technical communities • Skilled ETSI Staff managing test events ETSI Testing Projects - ICT Workshop, Brazil March 2006

  23. Why? • Provides useful feedback to the standardization process • Accelerates time to market via quick product debugging • Improves both quality and features of implementations • Helps to ensure end users’ satisfaction • Reduces deployment costs • Allows networking with the market key players COST & TIME SAVING! ETSI Testing Projects - ICT Workshop, Brazil March 2006

  24. Upcoming events in 2006 ETSI Testing Projects - ICT Workshop, Brazil March 2006

  25. Thank You! Phone +33 (0)4 92 94 43 26 e-mailptcchelp@etsi.org Websitewww.etsi.org/ptcc Anthony Wiles ETSI Protocol and Testing Competence Centre

More Related