Evaluation of AMPS Forecasts Using Self-Organizing Maps (SOMs)

1. Evaluation of AMPSForecasts UsingSelf-Organizing Maps (SOMs) John J. Cassano Cooperative Institute for Research in Environmental Science and Program in Atmospheric and Oceanic Sciences University of Colorado at Boulder Beardmore Glacier, Jan 2004

2. Outline • What are SOMs? • Application of SOMs for model evaluation studies • Application of SOM Analysis to AMPS data • Conclusions / Future Work

3. What are SOMs? • SOM - Self-Organizing Map • SOM technique uses an unsupervised learning algorithm (neural net) • Clusters data into a user selected number of nodes • SOM algorithm attempts to find nodes that are representative of the data in the training set • More nodes in areas of observation space with many data points • Fewer nodes in areas of observation space with few data points • SOMs are in use across a wide range of disciplines

4. Application of SOMsfor Model Evaluation Studies • Synoptic pattern classification • Frequency of occurrence of synoptic patterns • Determine model errors for different synoptic patterns

5. Application of SOM Analysis to AMPS Data • Train SOM with AMPS SLP data • Result is a synoptic pattern classification • Evaluate frequency of occurrence of synoptic patterns predicted by AMPS as a function of forecast duration • Map 0h, 24h, and 48h forecasts to SOM • Mis-mapping of AMPS forecasts • Model validation statistics for specific synoptic patterns (ongoing work) • Calculate model error statistics at points of interest (Willie Field) for different synoptic patterns • Are certain synoptic patterns prone to bias (e.g. error in predicted wind speed or direction)?

6. AMPS Data for SOM Analysis • SLP over Ross Sea sector of AMPS 30 km model domain • Summer only (NDJ) • 00Z AMPS simulations from Jan 2001 through Feb 2003 • 186 model simulations • Evaluate 0, 24, and 48 h AMPS forecasts

7. AMPS SOM Analysis Domain

8. Synoptic Pattern Classification

9. Frequency of Occurrence

10. Misprediction of Synoptic Patterns • Consider all of the time periods for which the model analyses map to a particular node • For these time periods determine which nodes the model predictions map to • From this analysis we can determine biases in the model predictions of specific synoptic patterns relative to the model analyses • Percent of cases that map to the correct node • Mis-mapping of model predictions between nodes

11. 86.7% 64.7% 73.2% 1 1 1 2 4 1 81.1% 71.1% 100% 2 1 AMPS 24h Forecasts

12. 86.7% 58.8% 63.4% 1 2 1 2 5 4 67.6% 65.8% 100% 1 2 AMPS 48h Forecasts

13. Model Errors for Synoptic Patterns • Compare model predictions to in-situ atmospheric measurements • Calculate model validation statistics for all time periods that map to each node • Look for model errors that vary from node to node • This is ongoing work using AMPS data • This technique has been applied to ARCMIP data

14. An ARCMIP exampleSHEBA Surface Pressure (DJF)

15. Conclusions / Future Work • The use of SOMs provides an alternate method of evaluating model performance • Identify synoptic patterns which are over or underpredicted • Determine model tendency for mis-prediction of certain synoptic types • Provide information on model errors related to specific synoptic patterns • Complete SOM analysis for entire AMPS archive (Jan 2001 - present) • Calculate model biases as a function of forecast time and synoptic patterns