1 / 21

Easy access HDF files via Hyrax

Easy access HDF files via Hyrax. Kent Yang The HDF Group. Outline. Why OPeNDAP ? Hyrax features Hyrax modules Demos Suggestions. Why OPeNDAP?. Check metadata remotely (in various forms) Obtain the subset of data easily and efficiently Hide the original data sources

jaron
Download Presentation

Easy access HDF files via Hyrax

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. Easy access HDF files via Hyrax Kent Yang The HDF Group

  2. Outline • Why OPeNDAP? • Hyrax features • Hyrax modules • Demos • Suggestions

  3. Why OPeNDAP? • Check metadata remotely (in various forms) • Obtain the subset of data easily and efficiently • Hide the original data sources • netCDF, HDF4, HDF5 or GeoTiFF, GRIB • Many popular earth science tools can visualize and analyze the data via OPeNDAP • OPeNDAP output(including subsets) can be downloaded as other formats

  4. Doesn’t help Great!

  5. HDF(5) Hyrax modules • The visualization of HDF(5) data via Hyrax • What NASA users request the most

  6. Helpful Hyrax feature – BES keys • BES keys to dynamic change the program flow • Easy for users to choose what they want • Easy for developers to add/revise features requested by users

  7. Example on using BES keys Float CERES_Unfiltered_Radiances_all_toa_sw_rad[gmt_hour_index= 24][latitude =180][longitude= 360] Float /CERES_Unfiltered_Radiances/all_toa_sw_rad[24][180][360]

  8. Hyrax modulation • Core framework implemented in C++ • Just need to provide DDS, DAS and Data to Hyrax libdap and BES • A handler can be sophisticated • HDF handler examples • Retrieve latitude and longitude for HDF-EOS grid • Interpolate the latitude and longitude for HDF-EOS swath with dimension map • Make sure the dimensions, variable names and attributes follow CF conventions

  9. Non-CF CF Float /CERES_Unfiltered_Radiances/all_toa_sw_rad[24][180][360] Float /CERES_Unfiltered_Radiances/all_toa_sw_rad[24][180][360] Float CERES_Unfiltered_Radiances_all_toa_sw_rad[gmt_hour_index = 24][latitude =180][longitude= 360]

  10. Demos – HDF-EOS2 (AIRS grid)

  11. Demo 2 – HDF-EOS2 (MODIS swath) Swath with Dimension Map

  12. Demo 3 – HDF-EOS5 (OMI grid)

  13. Hyrax modules that help the easy access • Filenet CDF module • Can download DAP output as a netCDF-3 or netCDF-4 classic files • NcML module • Add/delete/revise metadata information of the data source • Aggregate granules

  14. Demo • GPM level 1B(HDF5)

  15. HDF(5) and file netCDF demo • FilenetCDF module can work with the HDF handlers to download the HDF files or subset of the files as netCDF-3 or netCDF-4 classic files that follow the CF conventions

  16. HDF(5) and NcML demo • NcML module can be used with HDF handlers to provide the missing CF information • AVHRR example

  17. HDF(5), file netCDF and NcML demo • HDF handlers, NcML module and file netCDF module can work together • AVHRR example

  18. Summary of modules to serve HDF files HDF(5) files DAP clients (IDV, Ferret, Panoply, IDL…...) Hyrax core Main engine HDF(5) handlers Alternative output File netCDF Supplement Info NcML netCDF files

  19. Demo - Beyond the web service • BES standalone program • Connect the HDF handlers, the file netCDF module and the NcML module • End users can build Hyrax and use this program as a command-line tool • Can it be a standalone tool? besstandalone -c /etc/bes/bes.conf -i in4-GPM-l1b > GPM-l1b.nc4

  20. Future - Performance • Improve the BES caching mechanism • Critical for large file access and aggregation service

  21. Earth Science Group Ted Habermann Aleksandar Jelenak H. Joe Lee Joel Plutchak John Readey Kent Yang

More Related