1 / 12

Data Composites: Enhancing Accessibility and Analysis

UCAR/JOSS develops composites to provide data in a uniform format with quality control, useful for model applications and problem detection. Learn about the general dataset processing life cycle for creating composite datasets.

mandym
Download Presentation

Data Composites: Enhancing Accessibility and Analysis

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. http://www.joss.ucar.edu/ihop/dm/

  2. Composites: What and Why A composite dataset is a collection (over some time period and region) of similar data (e.g. surface met) from a variety of sources, put into a common format, and passed through a uniform quality control. Why does UCAR/JOSS develop composites? - Provides data in a uniform format with QC. - Allows determination of network/site problems. - Useful for model applications. - Prevents duplication of effort.

  3. Incomplete? Incomplete? Incomplete? Time ACTION GENERAL DATASET PROCESSING LIFE CYCLE • Phone, email, paperwork, $, meetings, • procurement arrangements. • (Days to months) • (Minutes to months) • Auto retrieval, FTP, online requests, • etc. (minutes to days) • Verify AOI, TOI, Format; spot check • values, visual inspection or limited • processing (Minutes to days) • Format change, new parameters • manual “cleanup”, etc. • (Days to months) • Purchase hardware, configuration • management, etc. • (Minutes to days) • Put into common format, run data checkers • “check gross limits” (Days to months) • HQC for SFC, Gross limits for precip, • Composite level and value checkers, limited • statistics (% G, B, D by network, stn, • parameter for SFC). (days to months) 1 Contact SRC Agency to Acquire Data and Metadata 2 Wait for SRC Agency Data/Metadata Preparation days/months 3 Acquire Data and Metadata OR Create “Best” Metadata 4 Examine Data & Metadata for Consistency and Completeness 5 Develop New Software or Modify Existing Software 6 Acquire Computer System Space and Time weeks/months 7 Process Data and Metadata then Check 8 Form Composite with Like Datasets Add stn info to final list 9 Perform QC and Final Data Checks. Generate Statistics Processing Problems? months/year Prepare ASCii format data & docs for CD-Roms Create EBUFR format data & DOCS for CODIAC Prepare, manufacture, & distribute CD-Roms Load Data and Metadata onto CODIAC z:\cully\Esop96\esop3.ppt

  4. Hourly Surface Meteorological Data Composite (743 stations) CoAgMet (*) (23) AWOS (*) (110) RAWS (+) (10) SCAN (o) (6) GWMD (*) (18) Ameriflux (+) (3) HPCN (o) (72) Texas ET (o) (19) KVII-TV (+) (4) MADIS (o) (246) NMSU (*) (7) 5-min sites (o) (225)

  5. Composite Data Sets To Be Developed by UCAR/JOSS for IHOP-2002 • Surface Meteorological - 1 min, 5 min, and Hourly • Precipitation - 15 min, Hourly, and Daily • Soundings - High Resolution and 5 hPa

  6. 1-min Surface Meteorological Data Composite (65 stations) ASOS (41) ARM SMOS (14) ABLE AWS (5) NCAR Supp. (5)

  7. 5-min Surface Meteorological Data Composite (224 stations) Oklahoma Mesonet (115) NCAR ISS/ISFF (10) West Texas Mesonet (34) 1-min sites (65)

  8. 15-minute Precipitation Composite NCDC Coop (429) 5-min surface (224)

  9. Hourly Precipitation Composite NCDC Coop (509) NCEP hourly (1155) Hourly Met (743)

  10. Daily Precipitation Composite NCDC Coop (1581) Hourly Met (743) NCEP Daily (1611)

  11. IHOP 2002 DATA BASE STATUS(as of 30 September 2003) • 215 of 254 Data sets (85%) have been submitted or linked. 204 are available on-line (11 in progress) • Field Catalog (w/reports and preliminary products) is on-line • Data submission instructions and guidelines are available on-line • Upper Air Composites (hi-res/5 hpa) are available on-line • Surface Composites to be available Fall 2003 (TRICK or TREAT!) • For details see: http://www.joss.ucar.edu/ihop/dm/

More Related