1 / 41

Tutorial on the Manipulation of Netcdf Climate Data: Application to Statistically Downscaled Climate Projections

Tutorial on the Manipulation of Netcdf Climate Data: Application to Statistically Downscaled Climate Projections. Michael Notaro Nelson Institute Center for Climatic Research University of Wisconsin-Madison mnotaro@wisc.edu 608-261-1503 20th Annual Conference of The Wildlife Society.

decker
Download Presentation

Tutorial on the Manipulation of Netcdf Climate Data: Application to Statistically Downscaled Climate Projections

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. Tutorial on the Manipulation of Netcdf Climate Data: Application to Statistically Downscaled Climate Projections Michael Notaro Nelson Institute Center for Climatic Research University of Wisconsin-Madison mnotaro@wisc.edu 608-261-1503 20th Annual Conference of The Wildlife Society

  2. STATISTICAL DOWNSCALING Observational Data The source of observed data was roughly 4100 weather stations measuring daily maximum/minimum temperature and precipitation during 1950-2006 in ten of the central-eastern North American Landscape Conservation Cooperatives (LCCs), from the NWS Cooperative Observer Program. Model Projection Data The climate output that was analyzed was produced by 9 GCMs from the Coupled Model Intercomparison Project Phase 3, according to 3 emission scenarios (A2, A1B, B1). The coarse climate projections were downscaled to a 0.1° latx 0.1° longrid over 10 LCCs and debiased against observed temperature/precipitation from station observations within the NWS Cooperative Observer Program. By interpolating and debiasing probability distribution functions and their attributes, a realistic representation of the variance and extremes of temperature/precipitation was achieved, in addition to a realistic representation of the means of both variables. In computing the projected changes in a variable, the difference in the mean of that variable was computed from 1961-2000 to either 2046-2065 or 2081-2100. CMIP3 Models CCCMA_CGCM3_1 = Canadian Centre for Climate Modelling and Analysis CNRM_CM3 = Meteo-France / Centre National de RecherchesMeteorologiques CSIRO_MK3_0 = CSIRO Atmospheric Research, Australia CSIRO_MK3_5 = CSIRO Atmospheric Research, Australia GFDL_CM2_0 = US Department of Commerce / NOAA / Geophysical Fluid Dynamics Laboratory GISS_MODEL_E_R = NASA / Goddard Institute for Space Studies, US MIUB_ECHO_G = Meteorological Institute University of Bonn, Germany MPI ECHAM5 = Max Planck Institute for Meteorology, Germany MRI_CGCM2_3_2a = Meteorological Research Institute, Japan Project Funding The statistical downscaling project was funded by: (1) Michigan DNR through an EPA grant with the Great Lakes Restoration Initiative, (2) Upper Midwest and Great Lakes LCC, and (3) Wisconsin Focus on Energy. The downscaling was performed by Dr. David Lorenz. NCO

  3. Interactive mapping website for examining the downscaled data http://ccr.aos.wisc.edu/resources/data_scripts/LCC NCO

  4. NCDUMP NCVIEW NCO TOOLS NCL Additional tools: Text editor: Emacs or vi Ghostview

  5. BASIC UNIX COMMANDS cd = change directory Example: cd /data/Share/workshop/20c3m ls = list files in a directory pwd = list the name of directory you are in mv = move (or rename) file Example: mv oldfilenewfile Example: mv oldfile /data/Share/workshop/ cp = copy file Example: cpoldfilenewfile mkdir = create a directory Example: mkdirdirname rm = delete file Example: rm filename rmdir = delete directory Example: rmdirdirname emacs = text editor Example: emacs = opens emacs Example: emacs & = opens emacs in background vi = text editor Example:vi = opens vi Example: vi filename = opens old or new file NCO UNIX Tutorials: http://kb.iu.edu/data/afsk.html http://www.math.utah.edu/lab/unix/unix-commands.html http://people.ischool.berkeley.edu/~kevin/unix-tutorial/toc.html

  6. Change directory cd /data/Share/workshop/20c3m Open ncview and look at a temperature file for 1990. Ampersand puts ncview in background, so you can still type in terminal. ncview temp_01_1990.nc & Click on “tmax” button NCO

  7. NCO Variable: Maximum daily temperature 365 Days, beginning Jan 1, 1990 Range of values -18.06°C to 40.825°C Move your pointer across map & notice the x,y coordinates and associated temperature values change

  8. Click on Opts to set options Choose USA states Click OK State boundaries should appear NCO

  9. Quit = Exit ncview 3gauss = Toggle among color schemes Inv P = flip the map vertically Inv C = Invert color map Range = Modify the max/min displayed values Bi-Lin = Either show raster boxes or interpolated values Print = Save the plot as a postscript file or send to a printer NCO

  10. Click on ? The variable tmax is “maximum daily temperature” of type short with units of °C and a missing value of -32768 NCO

  11. Click on <<, <, ||, >, and >> to move backwards/forwards in time, just like on a DVD player. NCO

  12. Click on any grid cell on map. A time series will appear of daily maximum temperatures during 1990. Move your pointer over time series to see data values. Click on Print button to save the plot as a postscript file. Click on Dump button to write actual data values to a text file. NCO

  13. Click on 4 different grid cells on the map to compare their time series. NCO

  14. NCDUMP ncdump–h prcp_01_1982.nc NCO

  15. NCDUMP ncdump –c prcp_01_1982.nc List the actual latitudes, longitudes, times lon = -115.3, -115.2, -115.1, ……, -67 lat = 24.6, 24.7, 24.8, ……, 54.7 time = 0, 1, 2, ……, 364 The resolution is 0.1 degree. ncdump–k prcp_01_1982.nc Tells you “netCDF-4” Possible outcomes: Classic, 64-bit offset, netCDF-4, netCDF-4 classic model NCO

  16. NCO The temperature files, such as temp_01_1982.nc, contain multiple variables (tmax, tmin). Let’s say we want to create a new file, only containing tmin. ncks: “kitchen sink” ncks –v tmin temp_01_1982.nc newfile.nc (-v stands for variable) Now, to verify that it worked, use ncdump. ncdump –h newfile.nc The file only contains tmin, not tmax. So it worked. NCO

  17. NCO The downscaling data covers a large region, 24.6-54.7N, 115.3-67W. What if we wanted to extract precipitation data in Wisconsin only? ncks –d lat,42.,47.5 –d lon,-93.5,-86.5 prcp_01_1996.nc small.nc (remember to use decimal points) To check the data, ncviewsmall.nc Our dataset is now much smaller, only containing data around Wisconsin. NCO

  18. NCO How can we compute the mean precipitation for July 1985? On the Julian calendar, starting with 0, July runs from day 181 to 211. First, extract only the July days from the file. ncks –d time,181,211 prcp_01_1985.nc july.nc Now, use the record averager, ncra. ncrajuly.ncjulymean.nc We can view the results with ncview. ncviewjulymean.nc NCO

  19. NCO How can we compute the difference between the mean annual daily maximum temperature in 1990 and 1989? First, compute the annual mean for 1989. ncra temp_01_1989.nc 89.nc Second, compute the annual mean for 1990. ncra temp_01_1990.nc 90.nc Third, compute the difference between the two files using the operator, ncdiff. ncdiff 90.nc 89.nc diff.nc Finally, view the file with ncview. ncviewdiff.nc NCO

  20. NCO How can we extract daily precipitation (mm/d) from 1995 for Madison, Wisconsin and dump the values to a text file? Use ncdump to check the content of file prcp_01_1995.nc. ncdump–h prcp_01_1995.nc Notice that variable prcp is type short, with scale factor of 0.025 & offset of 819.175. If we print out values, they will not be easily used. They need to be converted to float. ncap2 –s ‘prcp=float(prcp)’ prcp_01_1995.nc float95.nc This created file float95.nc, containing precipitation in easily readable values. Now, let’s extract data values for Madison only. ncks –d lat,43.07 –d lon,-89.40 float95.nc madison.nc Finally, let’s pipe those values to a text file. ncdumpmadison.nc > madison.txt Open madison.txt with a text editor (emacs, vi) to see the result. NCO

  21. NCL: NCAR Command Language Every script should begin with load commands, to access key libraries. I’ll only show them once here, but always include them at the top. In a text editor, create a file called hello.ncl. Insert the following text and save. load "$NCARG_ROOT/lib/ncarg/nclscripts/csm/gsn_code.ncl" load "$NCARG_ROOT/lib/ncarg/nclscripts/csm/gsn_csm.ncl" load "$NCARG_ROOT/lib/ncarg/nclscripts/csm/contributed.ncl" load "$NCARG_ROOT/lib/ncarg/nclscripts/csm/shea_util.ncl” print(“hello”) * To save time, cut and paste these load commands out of file loadcommands. Now, run the script by: nclscript.ncl The word hello should be printed to the screen. You have run your first NCL script. NCO

  22. NCL: NCAR Command Language Given that the downscaled data is so large, it was written in a compact format, specifically netcdf4. We need to specify that we’re working with this unique type of data. setfileoption = specifies the type of data we’ll be working with setfileoption(“nc”,”Format”,”NetCDF4Classic”) Then, we can open the netcdf4 file. addfile = opens a data file “r” means readable, “w” is writable This command allows us to open these types of data files: netcdf, grib, hdf4, hdfeos, hdf5, hdfeoss, ccm, shapefile a=addfile(“temp_01_1991.nc”,”r”) NCO

  23. NCL: NCAR Command Language We can retrieve the latitude and longitude values from the file. lat=a->lat lon=a->lon We can print the latitude values. print(lat) NCO

  24. NCL: NCAR Command Language Our current script: script.ncl load "$NCARG_ROOT/lib/ncarg/nclscripts/csm/gsn_code.ncl" load "$NCARG_ROOT/lib/ncarg/nclscripts/csm/gsn_csm.ncl" load "$NCARG_ROOT/lib/ncarg/nclscripts/csm/contributed.ncl" load "$NCARG_ROOT/lib/ncarg/nclscripts/csm/shea_util.ncl” setfileoption(“nc”,”Format”,”NetCDF4Classic”) a=addfile(“temp_01_1991.nc”,”r”) lat=a->lat lon=a->lon print(lat) NCO

  25. NCL: NCAR Command Language Let’s also retrieve the variable tmax from the file. Remember that tmax was in a short format and so we want to convert it to float. short2flt = Converts short format into float, using the offset and scale factor We can check the attributes/dimensions of our new variable using the command printVarSummary. setfileoption(“nc”,”Format”,”NetCDF4Classic”) a=addfile(“temp_01_1991.nc”,”r”) lat=a->lat lon=a->lon tmax=short2flt(a->tmax) printVarSummary(tmax) Now run the script. Notice the dimensions (time,lat,lon) and type float. NCO

  26. NCL: NCAR Command Language Let’s try to print out the daily maximum temperature values for Madison from 1991. Recall that tmax had dimensions of time,lat,lon. When specifying the closest grid cell to Madison, provide the latitude and longitude inside { }. To acquire all time values, specify : for the time dimension. setfileoption(“nc”,”Format”,”NetCDF4Classic”) a=addfile(“temp_01_1991.nc”,”r”) tmax=short2flt(a->tmax) print(tmax(:,{43.07},{-89.40})) NCO

  27. NCL: NCAR Command Language Let’s collect some basic statistics on a data file, specifically 1985’s precipitation. We’ll use the functions: avg, min, and max. setfileoption("nc","Format","NetCDF4Classic") a=addfile("prcp_01_1985.nc","r") prcp=short2flt(a->prcp) print(avg(prcp)) print(min(prcp)) print(max(prcp)) NCO

  28. NCL: NCAR Command Language Let’s say we want to plot a time series of Madison’s precipitation during 1985. First, let’s retrieve Madison’s values and print them to the screen as a check. setfileoption("nc","Format","NetCDF4Classic") a=addfile("prcp_01_1985.nc","r") prcp=short2flt(a->prcp) madison=prcp(:,{43.07},{-89.40}) print(madison) NCO

  29. NCL: NCAR Command Language We want to plot time on x-axis and precipitation on y-axis. First, create a “workstation”, wks. We will be creating a postscript (ps) plot, named script.ps. setfileoption("nc","Format","NetCDF4Classic") a=addfile("prcp_01_1985.nc","r") prcp=short2flt(a->prcp) madison=prcp(:,{43.07},{-89.40}) wks=gsn_open_wks(“ps”,”script”) NCO Temperature Time

  30. NCL: NCAR Command Language We will use the function gsn_csm_xyto make an XY plot. gsn_csm_xy(workstation, x-axis variable, y-axis variable, resource) For the x-axis, we will use “ispan(1,365,1)” which means a range of integers spanning from days 1 to 365 at a 1-day interval. For simplicity, assign “False” to resource options in gsn_csm_xy. setfileoption("nc","Format","NetCDF4Classic") a=addfile("prcp_01_1985.nc","r") prcp=short2flt(a->prcp) madison=prcp(:,{43.07},{-89.40}) wks=gsn_open_wks(“ps”,”script”) plot=gsn_csm_xy(wks,ispan(1,365,1),madison,False) Run the script. To view the plot, use ghostview in command line: gvscript.ps NCO

  31. NCL: NCAR Command Language Let’s now try plotting a spatial map of the mean climatological maximum temperatures for March (averaged across 1961-2000). This information is contained in file temp_mean_1961_2000.nc. Let’s use ncdump to check the file’s content. ncdump –h temp_mean_1961_2000.nc Notice the dimensions of tmax: time 12 (one value per month) latitude 302 longitude 484 NCO

  32. NCL: NCAR Command Language Dimensions of tmax are time, lat, lon. To retrieve March only and all latitudes/longitudes, specify we want tmax(2,:,:). Remember the counting starts at zero, so March=2. setfileoption("nc","Format","NetCDF4Classic") a=addfile("temp_mean_1961_2000.nc","r") lat=a->lat lon=a->lon tmax=short2flt(a->tmax) march=tmax(2,:,:) NCO

  33. NCL: NCAR Command Language setfileoption("nc","Format","NetCDF4Classic") a=addfile("temp_mean_1961_2000.nc","r") lat=a->lat lon=a->lon tmax=short2flt(a->tmax) march=tmax(2,:,:) wks= gsn_open_wks ("ps", "script") res=True res@cnFillOn=True res@gsnAddCyclic=False res@mpMinLatF=min(lat) res@mpMaxLatF=max(lat) res@mpMinLonF=min(lon) res@mpMaxLonF=max(lon) res@mpOutlineBoundarySets = "AllBoundaries” plot = gsn_csm_contour_map_ce(wks,march, res) NCO Opens a new workstation Plot will be called script.ps

  34. NCL: NCAR Command Language setfileoption("nc","Format","NetCDF4Classic") a=addfile("temp_mean_1961_2000.nc","r") lat=a->lat lon=a->lon tmax=short2flt(a->tmax) march=tmax(2,:,:) wks= gsn_open_wks ("ps", "script") res=True res@cnFillOn=True res@gsnAddCyclic=False res@mpMinLatF=min(lat) res@mpMaxLatF=max(lat) res@mpMinLonF=min(lon) res@mpMaxLonF=max(lon) res@mpOutlineBoundarySets = "AllBoundaries” plot = gsn_csm_contour_map_ce(wks,march, res) NCO Creates a new set of resources named res

  35. NCL: NCAR Command Language setfileoption("nc","Format","NetCDF4Classic") a=addfile("temp_mean_1961_2000.nc","r") lat=a->lat lon=a->lon tmax=short2flt(a->tmax) march=tmax(2,:,:) wks= gsn_open_wks ("ps", "script") res=True res@cnFillOn=True res@gsnAddCyclic=False res@mpMinLatF=min(lat) res@mpMaxLatF=max(lat) res@mpMinLonF=min(lon) res@mpMaxLonF=max(lon) res@mpOutlineBoundarySets = "AllBoundaries” plot = gsn_csm_contour_map_ce(wks,march, res) NCO Specify to turn color fill on

  36. NCL: NCAR Command Language setfileoption("nc","Format","NetCDF4Classic") a=addfile("temp_mean_1961_2000.nc","r") lat=a->lat lon=a->lon tmax=short2flt(a->tmax) march=tmax(2,:,:) wks= gsn_open_wks ("ps", "script") res=True res@cnFillOn=True res@gsnAddCyclic=False res@mpMinLatF=min(lat) res@mpMaxLatF=max(lat) res@mpMinLonF=min(lon) res@mpMaxLonF=max(lon) res@mpOutlineBoundarySets = "AllBoundaries” plot = gsn_csm_contour_map_ce(wks,march, res) NCO Our data is not global, or “cyclic”

  37. NCL: NCAR Command Language setfileoption("nc","Format","NetCDF4Classic") a=addfile("temp_mean_1961_2000.nc","r") lat=a->lat lon=a->lon tmax=short2flt(a->tmax) march=tmax(2,:,:) wks= gsn_open_wks ("ps", "script") res=True res@cnFillOn=True res@gsnAddCyclic=False res@mpMinLatF=min(lat) res@mpMaxLatF=max(lat) res@mpMinLonF=min(lon) res@mpMaxLonF=max(lon) res@mpOutlineBoundarySets = "AllBoundaries” plot = gsn_csm_contour_map_ce(wks,march, res) NCO Specify the area to plot

  38. NCL: NCAR Command Language setfileoption("nc","Format","NetCDF4Classic") a=addfile("temp_mean_1961_2000.nc","r") lat=a->lat lon=a->lon tmax=short2flt(a->tmax) march=tmax(2,:,:) wks= gsn_open_wks ("ps", "script") res=True res@cnFillOn=True res@gsnAddCyclic=False res@mpMinLatF=min(lat) res@mpMaxLatF=max(lat) res@mpMinLonF=min(lon) res@mpMaxLonF=max(lon) res@mpOutlineBoundarySets = "AllBoundaries” plot = gsn_csm_contour_map_ce(wks,march, res) NCO Show state and country boundaries

  39. NCL: NCAR Command Language setfileoption("nc","Format","NetCDF4Classic") a=addfile("temp_mean_1961_2000.nc","r") lat=a->lat lon=a->lon tmax=short2flt(a->tmax) march=tmax(2,:,:) wks= gsn_open_wks ("ps", "script") res=True res@cnFillOn=True res@gsnAddCyclic=False res@mpMinLatF=min(lat) res@mpMaxLatF=max(lat) res@mpMinLonF=min(lon) res@mpMaxLonF=max(lon) res@mpOutlineBoundarySets = "AllBoundaries” plot = gsn_csm_contour_map_ce(wks,march, res) NCO Plot a contour map of march temperatures, using resources “res”

  40. NCL: NCAR Command Language setfileoption("nc","Format","NetCDF4Classic") a=addfile("temp_mean_1961_2000.nc","r") lat=a->lat lon=a->lon tmax=short2flt(a->tmax) march=tmax(2,:,:) wks= gsn_open_wks ("ps", "script") res=True res@cnFillOn=True res@gsnAddCyclic=False res@mpMinLatF=min(lat) res@mpMaxLatF=max(lat) res@mpMinLonF=min(lon) res@mpMaxLonF=max(lon) res@mpOutlineBoundarySets = "AllBoundaries” plot = gsn_csm_contour_map_ce(wks,march, res) Now, run the script and use ghostview to display the plot. nclscript.ncl gvscript.ps NCO

  41. NCL: NCAR Command Language NCO

More Related