1 / 23

Canada, the IPY, and Data Management

Canada, the IPY, and Data Management. World Data Center for Glaciology, Boulder. Facilitating the international exchange of snow and ice data. Mark A. Parsons IPY Data Policy and Management Sub-committee IPY Data and Information Service Electronic Geophysical Year.

orenda
Download Presentation

Canada, the IPY, and Data Management

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. Canada, the IPY, and Data Management World Data Center for Glaciology, Boulder Facilitating the international exchange of snow and ice data Mark A. Parsons IPY Data Policy and Management Sub-committeeIPY Data and Information ServiceElectronic Geophysical Year Cryospheric System Annual Science Meeting Toronto, Canada 24 February 2006

  2. IPY1 IPY2 ? IPY4 IGY (IPY3)

  3. ? IPY4 What will IPY4 bring? The Challenge! • Will you be able to find all the data relevant to your research and see relationships between data sets. Access • Will you be able to merge and integrate different data sets across experiments and disciplines? Interoperability • Will you be able to subset, visualize, and transform the data? Usability • Will you be able to retrieve and understand IPY4 data in 2050? Preservation Canada, IPY, and Data; Mark A. Parsons, CRYSYS, 24 February 2006

  4. Organization of IPY Data Management Data Policy & Management Subcommittee • scientists • data managers • funding agencies IPY Joint Committee eGY Programme Office Data & Information Service Users Projects Data Centers, Virtual Observatories, etc. Canada, IPY, and Data; Mark A. Parsons, CRYSYS, 24 February 2006

  5. Canada, IPY, and Data; Mark A. Parsons, CRYSYS, 24 February 2006

  6. DIS? Alternate Views of the DIS DIS? Canada, IPY, and Data; Mark A. Parsons, CRYSYS, 24 February 2006

  7. Alternate Views of the DIS Some Canadian pieces to the puzzle GSC ELOKA CNADC DIS CCIN Canada, IPY, and Data; Mark A. Parsons, CRYSYS, 24 February 2006

  8. A specific project—DADDI • Interoperable data from CRYSYS, SEARCH, and elsewhere to better understand Arctic coastal processes. • Involves CCIN, three DAACs, and possibly AWI. • A “Web Services” approach to data description, exchange, discovery, visualization, and access. • A possible prototype for IPY. • Stay tuned for a scoping workshop this summer in New York. Canada, IPY, and Data; Mark A. Parsons, CRYSYS, 24 February 2006

  9. But that’s not enough! You Must: • Require rigorous data management plans • Determine archive and identify data management point of contact within project • Document well and often • Negotiate roles, responsibilities, and milestones with archive and DIS • Make data freely available* • Ensure appropriate data attribution and ownership • Ensure long-term preservation and access including non-digital data. Canada, IPY, and Data; Mark A. Parsons, CRYSYS, 24 February 2006

  10. Better yet, you should: • Identify relevant historical data and data from other projects and make appropriate arrangements • Make data “interoperable” through standard formats, transfer mechanisms, descriptions—build coalitions • Facilitate model assimilation • Develop high-level outreach products Canada, IPY, and Data; Mark A. Parsons, CRYSYS, 24 February 2006

  11. Open Questions and Issues • How interoperable do you want to be? What does “portal” mean to you? • How does IPY data fit into current operational systems? • What about GEOSS—can IPY be a prototype? • Standards are essential, but which ones? (ISO19115, OAIS, OGC…) • Tech trends that can help us (XML (GML), ontologies, portals, etc.) • What do you think about the data policy? • Need a solid business model esp. for the long-term Data Committee and DIS partners meet next week. Canada, IPY, and Data; Mark A. Parsons, CRYSYS, 24 February 2006

  12. We welcome your feedback! Ellsworth LeDrewells@watleo.uwaterloo.ca Mark A. Parsonsparsonsm@nsidc.org

  13. Milestones Early 2006 • Identify data-management point of contact, preferably with some data management expertise • Identify and arrange funding for an archive Mid 2006 • Define specific roles and responsibilities and standards for data description, quality control, formats, etc • Identify other relevant data not directly part of the project and make similar arrangements Early 2007 • Submit catalogue metadata to the IPY Data and Information Service (DIS) and appropriate archive. During IPY as data are collected. • Submit full ISO19115 metadata to DIS and archive as data are collected. • Submit full, verified (QCed) data collection and comprehensive documentation to archive 2009 • Archives make data fully and publicly available through the DIS and other mechanism. Canada, IPY, and Data; Mark A. Parsons, CRYSYS, 24 February 2006

  14. Systems and Innovation Succeeded “Challenged” Failed The Standish Group’s “CHAOS report”. An assessment of 40,000 IT application projects Canada, IPY, and Data; Mark A. Parsons, CRYSYS, 24 February 2006

  15. Data Management Considerations or Themes • Manage technical innovation • Systems need people • Scientists and data managers working together • Preservation and Access—Two peas in a pod • The nature of the documentation • The nature of the data Canada, IPY, and Data; Mark A. Parsons, CRYSYS, 24 February 2006

  16. The People Part “A striking proportion of project difficulties stem from people in both customer and supplier organisations failing to implement known best practice.” Service counts. — Oxford University/Computer Weekly survey of public and private sector IT projects (emphasis added) However, people are much more able to adapt to change, uncertainty, and messy systems Canada, IPY, and Data; Mark A. Parsons, CRYSYS, 24 February 2006

  17. The People Part: Science and Data Management • Many have stated the need to involve scientists in data management, but… • It is also important to involve data managers in conducting science. • Field Experiments: • 20% increase in data quality (Parsons, et al. 2004) • 70% of experiment cost is data collection (Longley, et al. 2001) • Observing systems Canada, IPY, and Data; Mark A. Parsons, CRYSYS, 24 February 2006

  18. Preservation and Access—Two Peas in a Pod Scientific Data Stewardship: • “preservation and responsive supply of reliable and comprehensive data, products, and information for use in building new knowledge to…” —USGCRP, 1998 • “the long-term preservation of the scientific integrity, monitoring and improving the quality, and the extraction of further knowledge from the data” — H. Diamond et al., NOAA/NESDIS, 2003 Canada, IPY, and Data; Mark A. Parsons, CRYSYS, 24 February 2006

  19. Access. What is it? • Preservation requirements are well defined in the Open Archive Information System (OAIS) Reference Model, but • No similar model for access requirements — eGY could help • Not even a common definition of “access” and what restricts it • Unique access requirements for social science data and non-digital collections (physical samples, photographs, audio, etc.) Canada, IPY, and Data; Mark A. Parsons, CRYSYS, 24 February 2006

  20. Use existing standards, e.g. ISO19115 metadata standard OAIS Reference Model Describe uncertainty Challenge your assumptions Documentation “We must not … start from any and every accepted opinion, but only from those we have defined — those accepted by our judges or by those whose authority they recognize.” —Aristotle c. 350 BC Canada, IPY, and Data; Mark A. Parsons, CRYSYS, 24 February 2006

  21. 011000101001001111010111000111101100101010001110011100101010011101010100111000110101101000010000100101001001010110010010001010100100100101010101001010100101001010100000111110010110101010110100010111101011010110101010011000101001001111010111000111101100101010001110011100101010011101010100111000110101101000010000100101001001010110010010001010100100100101010101001010100101001010100000111110010110101010110100010111101011011000101001001111010111000111101100101010001110011100101010011101010100111000110101101000010000100101001001010110010010001010100100100101010101001010100101001010100000111110010110101010110100010111101011010110101010011000101001001111010111000111101100101010001110011100101010011101010100111000110101101000010000100101001001010110010010001010100100100101010101001010100101001010100000111110010110101010110100010111101011 The Data Itself Formats: • Archives and users may have different needs • Consider four themes (Raymond, 2004) • Transparency • Interoperability • Extensibility • Storage or transaction economy Canada, IPY, and Data; Mark A. Parsons, CRYSYS, 24 February 2006

  22. Data Management Principles (bumper stickers) Preservation without access is pointless; access without preservation is impossible. It’s about DATA not systems Involve scientists in data management & data managers in science Think about long-term archiving NOW! Document uncertainty! Keep things simple & flexible Consider the needs of current, future, and unknown users

  23. What’s Next? • The Data and Information Service should be created soon. • The Data Sub-Committee needs to consider these themes and principles when developing the IPY data policy. Canada, IPY, and Data; Mark A. Parsons, CRYSYS, 24 February 2006

More Related