1 / 8

OpenADE – ESPI Testing and Certification Considerations

OpenADE – ESPI Testing and Certification Considerations . Phil Beecher phil@beecher.co.uk. Testing Methods - Conformance. Conformance Testing Testing that a product is conformant to a specification

feoras
Download Presentation

OpenADE – ESPI Testing and Certification Considerations

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. OpenADE – ESPI Testing and Certification Considerations Phil Beecher phil@beecher.co.uk

  2. Testing Methods - Conformance • Conformance Testing • Testing that a product is conformant to a specification • Implementation Under Test (IUT) is exercised by a Test Harness to verify its behavior to the specification • The Test Harness can behave in a way conformant to the specification, • It may also behave in a non-conformant way, allowing negative test cases to be executed.

  3. Testing Methods - Interoperability • Interoperability Testing • Testing that a product will interoperate with one or more other implementations. • Implementation Under Test (IUT) is tested together with another (conformant) implementation to verify their combined behavior is interoperable. • Interoperability testing is ideally against a “Golden Unit”, i.e. an implementation which has been tested to be conformant to the specification. • The Golden Unit does not behave in a non-conformant way, as it is a conformant implementation, so negative testing is not possible.

  4. Test for Compliance • Compliance Testing • May comprise Conformance Testing, Interoperability Testing or both. • Phil’s preferred method is both • Test for conformance using a test harness which can provide good coverage of both positive and negative test cases. • Test for interoperability using a known mature, conformant implementation.

  5. Compliance Testing • Steps for Conformance Testing • Develop a pro-forma PICS • Develop a test plan • Develop test harness – who and how? • Verify test harness • Decide if test lab or labs are necessary • Assess lab capability and appoint lab(s).

  6. Options to Consider (1) • OpenADE develops Test Plan • Who develops (and maintains) Test Harness? • (ZigBee Alliance – each test lab develops its own test harness) • Wi-Fi Alliance – lead lab (Wi-Fi staff) develop test harness and supply to labs. • Do we use Conformance testing, Interop testing or both to assess product? • ZigBee uses a bit of a hybrid • Wi-Fi also uses hybrid • Bluetooth uses both: conformance followed by Interop with popular shipping devices

  7. Options to Consider (2) • How many test labs and do we restrict the number? • ZigBee has 3 in 2 countries • Wi-Fi has 15 in 8 countries • UCAIug 61850 appoints any who meet criteria • Is there sufficient business for more than 1? • Do we need any, or can we use an accredited test harness, with self-test? • Who is the certification body? • Does UCA do it? • Does ISO 65 accredited body (e.g. NEMA) do it?

  8. Things to decide • Number of labs? • Do we need a conformance test harness and interop test harnesses (Golden platforms)? • How do we decide who develops test harness? • How do we ensure availability of test tools / low bar for entry?

More Related