1 / 8

Plans for an Enhanced NetCDF-4 Interface to HDF5 Data

Explore the features and goals of the enhanced NetCDF-4 interface leveraging HDF5, offering large file support, packed data, dynamic schema modifications, and more for advanced Earth science modeling.

jcurtis
Download Presentation

Plans for an Enhanced NetCDF-4 Interface to HDF5 Data

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. Plans for an Enhanced NetCDF-4 Interface to HDF5 Data Russ Rew, UCAR Unidata Program Mike Folk, NCSA/UIUC Ed Hartnett, UCAR Unidata Program Robert E. McGrath, NCSA/UIUC

  2. What is NetCDF? • Similarities with HDF5: • A scientific data model • A data access library and set of interfaces • A format for storing and accessing data • Differences from HDF5: • Simpler, more widely used (currently) • Less powerful, less performance-oriented • Current version: netCDF-3 is supported by UCAR Unidata Program

  3. NetCDF-4/HDF5 Goals • Combine desirable characteristics of netCDF and HDF5, while taking advantage of their separate strengths: • Widespread use and simplicity of netCDF • Generality and performance of HDF5 • Preserve format and API compatibility for netCDF users • Demonstrate benefits of combination in advanced Earth science modeling efforts

  4. NetCDF-4 Features Enabled by HDF5 • Large file support • Parallel I/O • Multiple dynamic dimensions • Packed data, compression • New data types • Dynamic schema modifications • Other possibilities: groups, Unicode, user-defined types, better coordinate support, …

  5. Backward Compatibility • Two kinds of compatibility: • API: • Extension of netCDF-3 API • Support existing netCDF-3 applications by recompiling/relinking • Format: • Access to existing netCDF-3 data • Access to HDF5 files written through netCDF-4 interface • Leverage C interface for other interfaces to Fortran90, C++, Perl, Ruby, … • Java interface must be developed separately

  6. netCDF-3 applications netCDF-4 applications HDF5 applications netCDF files netCDF-4 Library netCDF-4 HDF5 files HDF5 files HDF5 Library netCDF-3 Interface NetCDF-4 Architecture • Supports access to netCDF files and HDF5 files created through netCDF-4 interface

  7. NCSA Contributions • Overall design • Dimension scales • To support shared dimensions • Addition to HDF5 data model • Parallel I/O • Using Argonne/Northwestern work • Challenging performance goal

  8. Acknowledgments and URL • The netCDF-4/HDF5 project is funded by the NASA AIST program • Unidata is sponsored by the National Science Foundation • URL: www.unidata.ucar.edu/packages/netcdf-4

More Related