Adaptive localization and adaptive inflation methods with the LETKF

1. 5/20/2010, 5th International EnKF Workshop, Bergen, Norway Adaptive localizationand adaptive inflationmethods with the LETKF Takemasa Miyoshi University of Maryland, College Park miyoshi@atmos.umd.edu With many thanks to E. Kalnay, B. Hunt, K. Ide, J.-S. Kang, H. Li, UMD Chaos-Weather group

2. Adaptive covariance localization

3. Covariance localization (e.g., Houtekamer and Mitchell 1998) • Empirical treatment for… • reducing sampling noise • increasing the rank Localized

4. Difficulties of localization • Difficulties include… • depending on (x, y, z, t) • reducing flow-dependence

5. Adaptive localization • Hierarchical filter by Anderson (2007) • Cross-validationby groups of ensembles • ECO-RAP by Bishop and Hodyss (2009) • Smooth the sample correlations raised to a power • High sample correlation = more reliable = more weight • Spatial smoothing to reduce noisiness of the sample correlation Cross- validation Localization weights

6. LETKF algorithm (Hunt et al. 2007) Local Ensemble Transform Kalman Filter Each grid point is treated independently. Multiple observations are treated simultaneously.

7. Localization in LETKF Analysis of the i-th variable: Two steps of localization: 1. Selecting a subset of global obs for the i-th variable are composed of only selected local obs. 2. Obs error std. is weighted by the localization factor so that far-away obs have large error. R-localization, Hunt et al. (2007)

8. Adaptive localization with LETKF The localization factor for R-localization is given by two adaptive components: A. Cross-validation (Anderson 2007) c1 m/2-member correl. c2 B. Use the sample covariance (Bishop and Hodyss 2009) High sample correlation = more reliable = more weight m-member correl.

9. Advantages and disadvantages • Advantages: • Minimal additional computations • Minimal changes to the existing LETKF code • Automatic inter-variable 4-D localization • Disadvantages • Sampling error issue remains.

10. Results with the Lorenz-96 model not-noisy noisy Not the best, but we can effectively avoid optimization.

11. Preliminary results with an AGCM

12. Adaptive covariance inflation

13. Covariance inflation (e.g., Houtekamer and Mitchell 1998) • Empirical treatment for… • covariance underestimation • Error covariance is underestimated due to various sources of imperfections: • limited ensemble size • nonlinearity • model errors No inflation 5% inflation inflation

14. Difficulties of inflation • Fixed covariance inflation has difficulties such as… • ensemble spread exaggerates observing density pattern • should depend on (x, y, z, t); tuning is very difficult Temperature spread of JMA’s global LETKF w/ fixed multiplicative inflation ~500 hPa ~50 hPa adapted from Miyoshi et al. (2010)

15. Additive inflation • introduces new directions to span the error space • solves the problem of the spread pattern • has difficulties in obtaining reasonable additive noise • random noise does not grow initially Temperature spread of JMA’s global LETKF w/ additive inflation ~500 hPa ~50 hPa adapted from Miyoshi et al. (2010)

16. Adaptive inflation Anderson (2007; 2009) developed an adaptive inflation algorithm using a hierarchical Bayesian approach. Li et al. (2009) used the statistical relationship derived by Desroziers et al. (2005): using In the EnKF, i.e., We estimate the inflation parameter:

17. Adaptive inflation in the LETKF Analysis of the i-th variable: R-localization, Hunt et al. (2007) When computing Ti, we compute the following statistics simultaneously: Normalization factor 3-dimensional field of inflation

18. Time smoothing Due to the limited sample size in the local region at a single time step, it is essential to apply time smoothing to include more samples in time. We use the Kalman filter approach. a tuning parameter that determines the strength of time smoothing For example, i.e., more samples, more reliable. p denotes the number of observations (i.e., sample size) For example, when p = 100,

19. More considerations Localization function: is a measure of reliability Considering the sample size weighted by the reliability:

20. Results with the Lorenz 96 model 5% inflation adaptive inflation 3.07 2.71 0.31 0.31 Observed No obs Observed No obs 6-month average after 6-month spin-up

21. Results with the SPEEDY model Estimated adaptive inflation

22. Time series of adaptive inflation Overshooting gives stable performance still spinning up 4-mo. exp.

23. Averages of the final 1 mo. of 4-mo. experiments ~20% reduction of RMSE 4% fixed inflation adaptive inflation Too large spread Improved spread m/s m/s

24. Regular obs network

25. With regular obs network ~20% reduction of RMSE 4% fixed inflation adaptive inflation Improved spread m/s m/s

26. Expt. with model errors (Ji-Sun Kang, 2010) CO2 data assimilation with the SPEEDY-C model by Ji-Sun Kang All variables are prognostic using the SPEEDY-VEGAS model with dynamical vegetation Nature run: (u, v, T, q, ps, C, CF) Analyzed variables: prognostic like a parameter with fixed vegetation

27. Results with model errors m/s m/s K 0.1g/kg hPa ppmv 10-8kg/m2/s by Ji-Sun Kang (2010)

28. RMSE and Spread by Ji-Sun Kang (2010) Fixed inflation Globally constant adaptive inflation This adaptive inflation This adaptive inflation with model bias correction

29. CF estimates by Ji-Sun Kang (2010)

30. RMSE and Spread of CF estimates Fixed inflation Globally constant adaptive inflation This adaptive inflation by Ji-Sun Kang (2010)

31. Using both adaptive localization and adaptive inflationsimultaneously Preliminary results with Lorenz-96

32. Adaptive localization and inflation Adaptive inflation significantly stabilizes the filter, giving nearly optimal performance.

33. Future challenges • Application to other systems • Realistic NWP models • Ocean models • Martian atmosphere models • Assimilation of real observations • Model errors • Temporally varying observing network • (e.g., aircraft, satellites)

34. The LETKF code is available at: http://code.google.com/p/miyoshi/