Issues of Data Standardization

1. Issues of Data Standardization A workshop on building interoperable spatial data for multiple uses Doug Nebert, GSDI Secretariat May 2003

2. Agenda • 12:00 – Overview, problem, issues to consider in 'standardizing data‘ • 13:00 – Lunch • 14:30 – Presentations on standardization issues • 15:30 – Implementing data standards for interoperability and SDI • Discussion

3. Introduction to Standardization Organizations • Many standardization activities exist with different roles and responsibilities that are relevant to implementing SDIs: • International Organisation of Standardization (ISO TC 211, TC 204, JTC-1) • World Wide Web Consortium (W3C) • OpenGIS Consortium (OGC) • National Standards Organizations

4. Roles of Consensus Organizations • ISO provides general purpose standards and specifications as guidance to implementation • Industry Consortia provide technical implementation specifications • National/Community groups define common practices, content, and interaction within and outside the group

5. ISO TC211 Background • 29 participating (voting) nations • 27 observing nations • 22 external liaison organizations • First Plenary Norway, 1994 • Next 16th will be in Switzerland next wek • 39+ work items initiated • Many at Draft International Standard

6. ISO TC211 Work Items

7. World Wide Web Consortium • Mission: “to lead the World Wide Web to its full potential by developing common protocols that promote its evolution and ensure its interoperability.” • A Recommendation is work that represents consensus within W3C and has the Director's stamp of approval. W3C considers that the ideas or technology specified by a Recommendation are appropriate for widespread deployment and promote W3C's mission. • Such specifications developed within W3C must be formally approved by the Membership. Consensus is reached after a specification has proceeded through the review stages of Working Draft, Proposed Recommendation, and Recommendation.

8. HTML HTTP PNG SOAP/XMLP SVG URI/URL XHTML XLink XML XML Query XML Schema XPath XPointer XSL and XSLT CSS DOM W3C Contributions

9. OpenGIS Consortium OGC Vision A world in which everyone benefits fromgeographic information and services made available across any network, application, or platform. OGC Mission Our core mission is to deliverspatial interface specificationsthat are openly available for global use. • Focus on interoperability of software at the interface level to promote plug-and-play components for geographic information interchange • Not-for-profit, international consortium • 220+ industry, government, NGO and university members

10. OGC Programs • Specification Development Program -similar to other Industry consortia (W3C, OMG, etc.). • Interoperability Program (IP) - a global, innovative, hands-on engineering and testing program designed to accelerate interface development and bring interoperability to the market. • Outreach and Community Adoption Program – education and training, encourage take up of OGC interfaces, business development,

11. OGC Specifications • Simple Features Access (SQL, CORBA, OLE) • Catalog Services • Grid Coverages • Coordinate Transformation Services • Web Map Server Interfaces • Geography Markup Language • Web Feature Service • Filter Encoding Specification • Styled Layer Descriptor

12. Software interfaces (Implementation Specifications) Regional SDI Coordination Endorsed practices and specifications OpenGIS Consortium, W3C Other NSDIs SDI ISO TC 211 National Standards Content standards, Authority for data Foundations for implementation. (Abstract standards) Geospatial Standardization GSDI

13. Class A liaison OGC: Web Map Server, Web Feature Server, GML, Web Coverage Server, Style Layer Descriptor, Catalog Service XML Protocol (XMLP), XML Signature, I18N Class C Liaison: XML, I18N W3C: HTTP, PNG, RDF, SOAP/XMLP (Web Services Activity), XML, Xlink, Xpath, Xpointer, XSL/XSLT, XML Schema ISO: Ref Model, Terminology, Conformance testing, Profiles, Spatial Schema, Temporal Schema, Feature Cataloguing Methodology, Spatial Ref by Coords and Ids, Quality, Metadata, WMS, GML, LBS, Registration of Geo-information Items Review & approve stds. Provide expertise & candidate stds. Metadata Profile, Data Content Standards, etc. Establish definition of and terms of engagement in *SDI Interactions OpenGIS Consortium (OGC) Internet Engineering Task Force (IETF) ISO TC 211 TC 204 World Wide Web Consortium (W3C) JTC-1 Facilitation Bodies • GSDI • ANZLIC • PCGIAP • FGDC • PAIGH • INSPIRE • GeoConnections • CODI - GEO • AGI • … National Standards Organizations Adopter/ Implementer Community

14. The players OpenGIS Consortium (OGC) World Wide Web Consortium (W3C) ISO Adopter/ Implementer Community National Standards National Committee NSDI blueprint Users Policies, Agreements, Technologies Coordination

15. Community Coordination • FGDC and GeoConnections provide national fora for agreement on the common adoption of a suite of standards and practices that as a whole will function as a Spatial Data Infrastructure • The Global Spatial Data Infrastructure (GSDI) initiative seeks to promote compatible SDIs worldwide

16. Scope of National Solutions • Data Content Standards • Geographic Names and Gazetteer • Geodetic Reference Systems, Projections • Feature Type Catalogs • National Information Profiles of International Standards • Data Policies and Laws

17. Data Content Standards • Specify semantics and preferred syntax for information exchange for a specific theme of data • May define core or comprehensive “packaging” of spatial data • May be expressed as an ISO Application Schema – a means to express structure • Supports usage in OGC and other Web Service interfaces

18. Interoperability based on a common data model for data exchange Common Content Model import export Mission System 1 API Mission System 2 API ‘Core’ Data Encoding Format export import API Mission System 3

19. Geographic Names and Gazetteer • Names of geographic places should be identifiable with unique IDs for easy reference and usage • First-order Subdivision (Province) • Second-order Subdivision (District) • Populated places • Named landscape features • Establishment of online gazetteer by each nation is a useful service

20. Names Resources • Establish a names arbitration group to define the official names in your country • Work with international organizations, such as UN to link names with “codes” • United Nations Group of Experts on Geographical Names UNGEGN • United Nations Toponymic Guidelines • http://unstats.un.org/unsd/geoinfo/UNGEGN-Topguidelines-WP6.pdf

21. UN Toponymic Guidelines Name of individual country 1 Languages 1.1 General remarks 1.2 National language(s) 1.2.1 Legal situation and practical application in administration and official cartography 1.2.2 Alphabet(s) and transcription(s) 1.2.3 General spelling rules for geographical names 1.2.4 Pronunciation 1.2.5 Grammatical pecularities which are essential for the treatment and understanding of geographical names 1.2.6 Distribution of main dialects and their characteristics; how far are dialectal name forms adapted to the standard form of the language(s)? 1.3 Minority languages 1.3.1 – 1.3.6, as under 1.2 2 Names authorities and names standardization 2.1 National names authority. Aims, functions, rules, address. 2.2 Provincial names authorities. Aims, functions, rules, addresses. 2.3 Names standardization. Legal aspects, procedures, progress. 3 Source Material 3.1 Maps: map series containing standardized names. Which sheets of a series contain already standardized names? 3.2 Gazetteers containing standardized names. 4 Glossary of appellatives, adjectives and other words necessary for the understanding of maps. 5 Abbreviations used on official maps; the decoding and meanings 6 Methods of differentiating toponyms from other text on national maps

22. UNGEGN Meeting, Berlin 2002 • Geographical names experts met in Berlin to discuss setting up official names registries • There were African attendees from: • Cameroon, Sao Tome and Principe • Botswana, Lesotho, Namibia, South Africa, • Madagascar, Mozambique, Uganda, • United Republic of Tanzania, Zambia, • Côte d'Ivoire, Gambia, Guinea, Liberia, Nigeria • Algeria, Djibouti, Libyan Arab Jamahiriya, Morocco, Tunisia, Sudan

23. Gazetteer Services(OGC Gazetteer Profile of WFS) • Online service protocol is being developed to query geographic name servers to assist in relevant place name and coordinate assignment • Metadata collection (cataloguing) can benefit from access to Web service to encode place as name and coordinates • Geospatial search will benefit from stronger match of place to coordinates

24. Place Name and Context Structure Request may include ambiguous place type and name or coordinate Response should include name, attributes, and spatial coordinates Gazetteer Interaction User Web-based application Gazetteer Service

25. Geodetic Reference Systems • Selecting and conforming to a national or trans-national horizontal and vertical datum • Many new systems consistent with geoid-based solutions (useful with GPS) • Ability to also report data in geographic (latitude, longitude, WGS84-geoid-based decimal degrees) helpful in regional-global analysis

26. Common Projections • To simplify the processing required in using map and image data from multiple organizations, a common set of projection parameters is often selected • Datum, projection, units, shifts all can define a national convention • Storage of data in geographic decimal degrees is good for transnational use

27. Feature Type Catalogs • Development of national online catalog for “well-known” classes of features and attributes: • Definition • Authority • Relationships • ISO has feature cataloguing methodology to assist in solution

28. National Information Profiles of International Standards • ISO standards are typically adopted by national members as National Profiles • Profiles permit clarification, localization, and extension of ISO content • Coordination or common adoption of multi-national profiles is desirable

29. Examples of Profiles • Metadata: extend and explain the international standard for local users • Feature Catalogue: develop a data dictionary of the published feature types, attributes, domains in a data set • Application Schemas: describe the precise structure of the data set being transferred

30. Data Access Policies and Laws • Review existing Information Technology legislation and policies • Establish data exchange policies • Address privacy, intellectual property, and security issues • Define terms of participation in national SDI environment

31. How do we really deploy geo-standards? Evaluate, Advocate, Implement

32. Making standards work for you • A “standard” is a published way of conducting business; an agreement by which multiple parties can better interact • There are literally many standards to chose from, and you don’t have to use them all • Just don’t invent a new methodology if a suitable, supported standard exists!

33. Build on what exists • There are many standards and specifications, enforced by implementation in software that should be adopted and endorsed by an NSDI • Construction of special standards, formats, should be minimized in order to capitalize on global investment in common standards

34. Flavors of Standards • Standards are both the category and solution • Specifications usually include a subset of instructions to construct and test a conformant solution • Profile is a constrained specification that is scoped for a specific purpose or environment • Guidelines usually dictate a general approach or methodology for consistency in a community • Recommendations denote approval but may behave like specifications • Technical Reports and IPRs document research into an area of discourse but usually not a solution • Discussion Papers only provide information for comment and consideration (not standards)

35. Where are standards made? • Formal accredited standards bodies • International Bodies (ISO, IEC, ITU) • National bodies (Standards Australia) • Accredited bodies (ASTM, NISO) • Industry/community consortia • OpenGIS Consortium • World Wide Web Consortium (W3C) • Internet Engineering Task Force (IETF) • De Facto Market Practices • Adobe PDF • Microsoft Word • ESRI shapefile

36. Formal Standards Bodies • International and National standards organizations rely on voluntary contributions of intellect and time to work out broad agreements • Approval process very formal • Each nation is a member of ISO, nations typically nationalize ISO standards • Each nation may accredit other organizations to build standards under their name

37. Industry/Community Consortia • Participation in broad technical organizations such as W3C and OGC should be considered by primary stakeholders • Two-way representation by community brokers (e.g. FGDC) on consortia improves buy-in, common interests, information dissemination • Results of consortia may have more industry impact than other standards organizations

38. Community Coordination • Because standards are written independently, the specific usage of certain standards or specifications in a community still needs to be defined • A Reference Model is suggested to elaborate on the expected dependencies and interrelations within your community • Several countries are building standards-based reference models that define the terms of interoperability

39. SDI Architecture • Individual standards, specifications, policies, and practices must function together in the context of a potentially distributed “system” of participants • Implementing an architecture will often require the definition of a reference model, not only for the parts but for the whole

40. GML 3.0 XML may return deployed as used for transport validates HTTP XML Schema validates against transformable to UML ISO Spatial Schema harmonized with expressed in Standards are monolithic yet interdependent WFS 1.0

41. stores service info UI UI Metadata UI UI managed through managed through are derived for each managed through Metadata DB/Index Data Catalog Service Service Registry/ Catalog Metadata enter/update is exposed to the Internet through a are loaded to or stored in synchronize API queries may reference instances performs lookup to grab operation signatures may be coupled to or integrated with are derived from API is exposed to the Internet through a Catalog Client may send data to GEOdata Access Service feeds server info to Spatial Data Data/File Management System makes maps from stored in distributes to and collates from multiple Web Mapping Service draws layers from API enhances query with API managed through Gazetteer API Gateway UI enhances query with Other Service API Thesaurus provides application access through 1 2 3 Symbols current planned Software/Service 1. builds query screens for 2. submits queries/requests to 3. returns search responses API UI Information SDI Interaction Diagram revised 7-May-2003 ddnebert@fgdc.gov interacts with interacts with API UI Interface Application Client Web Client now: planned.: Function

42. Reference Implementations • Within a community, standards-based software may be commissioned or identified to provide a trusted level of entry-level functionality • Software may be free, subsidized, or low-cost within the identified community • Should exhibit well-known behaviors for testing of other software • Provides a jump point for development of commercial and compatible solutions

43. Commercialization • To assure rapid uptake and integration of selected standards and specifications, there must be integrated vendor support of the standards • Works in the familiar desktop environment • Permits coupling of distributed systems • Allows access to remote information without need of a browser • Let your vendors know of your interest in standards through the Reference Model

44. Standardisation Presentations Modeling the Geodatabase, CJ Cote, ESRI Global Map Data Themes, Hiroshi Une, JICA Standard Data Themes in South Africa, Derek Clarke, Survey and Mapping

45. Developing Relevant Data Standards Doug Nebert, GSDI/FGDC Secretariat

46. XML, GML, and UML • XML – Extensible Markup Language, a way to structure information for exchange • GML – Geography Markup Language (vector data expressed in XML) • UML – Unified Modeling Language, a design and diagramming environment to model your information

47. XML 1.0 • eXtensible Markup Language is a partial subset of Standard Generalized Markup Language (SGML: ISO 8879:1988) • Intended to store the structure and relationship of information in a form that software an easily read and write it • Expressed as characters, though element contents can also be numbers or links

48. Things in angle brackets are called elements or tags </title> indicates closing of the element !! Elements can have modifiers called attributes. <title dated = “11-Jun-2005”>My new book</title> <author>Knut Hampson</author> How is XML structured? <title>My new book</title> <author>Knut Hampson</author> © Galdos Systems Inc. April 2002

49. <book> <title>Hunger</title> <author> <first>Knut</first> <last>Hamson</last> </author> <chapter> It was bitter cold this morning. Bitter cold. </chapter> </book> Validation of XML <!element book(title, author, chapter*)> <!element title #PCDATA> <!element author (first, last) > <!element last #PCDATA> <!element first #PCDATA> <!element chapter #PCDATA> Document Structure Description (or Document Type Declaration DTD) XML Source Document © Galdos Systems Inc. April 2002

50. Geography Markup Language (GML 2.1.2) • OGC “Adopted Specification” (GML 2.1.2 currrent) for encoding spatial information. • A set of XML technologies (schema fragments) for expressing spatial data on the Internet. • Emerging international standard for spatial data—endorsed by 200 + companies and agencies around the world. • GML Version 3.0 as Draft ISO Standard (19136) by late 2003 © Galdos Systems Inc. April 2002