INTRODUCTION TO DTC ENSEMBLE TESTBED (DET)

1. Zoltan Toth1 1 GSD/ESRL/OAR/NOAA 2 NCAR/RAL Team Members: Barbara Brown2, Tara Jensen2, Huiling Yuan1, Isidora Jankov1, Ed Tollerud1, Linda Wharton1, Paula McCaslin1 Acknowledgements: Steve Koch, Jun Du, Yuejian Zhu, Geoff Dimego, Bill Lapenta DET website: http://www.dtcenter.org/det INTRODUCTION TODTC ENSEMBLE TESTBED (DET)

3. About 3,220,000 results (0.28 seconds) Search Results Amazon.com: Gypsy Music From Hungary And Romania: Zoltan And His ...Gypsy Music From Hungary And Romania by Zoltan And His Gypsy Ensemble. This product is manufactured on demand using CD-R recordable media. ...www.amazon.com › ... › Europe › Eastern Europe › Romania - Cached - SimilarAmazon.com: Zoltan & His Gypsy Ensemble: Albums, Songs, Bios, PhotosVisit Amazon.com's Zoltan & His Gypsy Ensemble Store to shop for Zoltan & His Gypsy Ensemble albums (CD, MP3, Vinyl) and other Zoltan & His Gypsy ...www.amazon.com/Zoltan-His...Ensemble/e/B000APVOT6 - Cached - SimilarShow more results from www.amazon.com

Zoltan & His Gypsy Ensemble - Rhapsody MusicListen to Zoltan & His Gypsy Ensemble FREE on Rhapsody.com. Rhapsody lets you explore every style of music without paying per song.www.rhapsody.com/zoltan-his-gypsy-ensemble - Cached – Similar Zoltan and His Gypsy Ensemble – Free listening, concerts, stats ...Listen free to Zoltan and His Gypsy Ensemble: Gypsy Tango, Hungarian Song And Dance & more, plus 1 picture. Gypsy, Hungarian and Balkan.www.last.fm/music/Zoltan+and+His+Gypsy+Ensemble – Cached Zoltan & His Gypsy Ensemble MP3 Downloads - Zoltan & His Gypsy ...Zoltan & His Gypsy Ensemble on MP3.com - listen to audio streams, check out photos, watch videos, write a review, discover similar artists, find news and ...www.mp3.com/artist/zoltan-and-his...ensemble/summary/ - Cached – Similar Zoltán Kodály – Háry János-Suite | for large orchestra - Universal ...Zoltán Kodály 16.12.1882—06.03.1967. Picture – Zoltán Kodály .... Student ensemble Student ensemble also played the following 5 works: ...www.universaledition.com/Zoltan-Kodaly/composers-and.../2721 - Cached[PDF] ON THE ECONOMIC VALUE OF ENSEMBLE BASED WEATHER FORECASTS Yuejian ...File Format: PDF/Adobe Acrobat - Quick Viewby Y Zhu - 2001 - Cited by 110 - Related articlesON THE ECONOMIC VALUE OF ENSEMBLE BASED WEATHER FORECASTS. Yuejian Zhu1, Zoltan Toth1, Richard Wobus1, David Richardson2, and Kenneth Mylne3 ...www.emc.ncep.noaa.gov/gmb/ens/papers/econval.pdf

4. OUTLINE / SUMMARY • Why and how DET was formed? • Community and NOAA ready • Sept 2009 Workshop • Major tasks • Infrastructure, test & evaluation, transition • Timeline • Planning, basic capabilities, benchmark, testing • Questions • For WG discussion • Workshop expectations • Recommendations for DET infrastructure & candidates for testing

5. Formation of DET in FY2010 Confluence of necessary ingredients • Need • Strong interest from both community & agencies • Knowledge base • Roadmap from Sept 2009 Workshop • National Workshop on Mesoscale Probabilistic Prediction at NCAR • Opportunity • Increase in funding for DTC from NOAA • Initiative • DTC Director’s Office sets planning process in motion (Oct 09)

6. OBJECTIVES & LIMITATIONS • “Provide an environment in which extensive testing and evaluation of ensemble-related techniques developed by the NWP community can be conducted such that the results are immediately relevant to the operational centers (e.g., NCEP/EMC and AFWA)” • Link research & operational communities • R2O – Testable techniques from research to operational community • O2R – Operational codes and requirements from operational to research community • Constraints • Limited funding ~ 3 FTEs – New task lead to be hired soon • GSD & NCAR collaborative effort • Excited about working with community on new project • No funding for research • Except via very limited visitor program • Initial focus on mesoscales

7. DET MAJOR TASKS • Develop and maintain DET infrastructure • Establish NCEP operational system as benchmark • Test and evaluate new community methods • Transition successful methods to NCEPand other agencies • Link up with ensemble work in other test-beds Input from revived WRF Ensemble WG Close collaboration with NCEP

8. INFRASTRUCTURE • Objective • Assemble software and procedures to carry out main function of testing and evaluation of community methods • Structure • Six modules identified • Ensemble configuration • Initial perturbations Ensemble generation – meso-specific • Model-related uncertainty • Statistical post-processing • Products / services Use of info – general, applies to all ens • Verification • Requirements • Modularity to facilitate transition to operations • Portability for ease of execution where computational resources are available • Flexibility to allow testing of new ideas

9. DET MODULES Module 1: Configuration Module 2: Initial Perturbations Module 3: Module Perturbations Module 4: Statistical Post-Processing External Input(HMT, HWT, HFIP, etc) Module 5: Product Generation Module 6: Verification

10. BENCHMARK • Purpose • Ensemble system/module used as reference when testing and evaluating new community methods • Definition • NCEP (or other center’s) operational ensemble system/module at any time • Facilitate community’s easy access to and tools to work with benchmark • Choice of first DET benchmark • Adopt current NCEP operational system/module • By time task completed benchmark may be obsolete • Contribute to building system/module for next NCEP implementation • In collaboration with NCEP • Accelerates NCEP implementation schedule • Avoids falling behind at start • Related tasks • Set it up, update, make it available to community

11. TEST AND EVALUATION • Core DET task • Carried out according to detailed protocol • Independent from developers • Use operational and other metrics • First application • Test & evaluate benchmark system / module against current operational system as reference

12. TRANSITION TO OPERATIONS • Objective • Offer NCEP & other agencies carefully tested and well performing modules • IT solution • Use subversion control • Potential NCEP actions • Possible modification • Streamline, combine with other modifications, etc • Further test & evaluation • Final selection • Implementation into operations

13. LINK WITH OTHER TESTBEDS • Objective • Leverage and coordinate DET work with other DTC testbeds and agency programs/projects for mutual benefits • Examples • HMT, HWT, HFIP - Ensemble development & evaluation work • DET to support ensemble work in various applications • Share infrastructure, best methods & practices, etc • GSI – Hybrid Variational / Ensemble-based data assimilation • DET to focus on ensemble system (perturbations, etc) • GSI task to focus on data assimilation aspect?

14. DET TIMELINE – STARTING MARCH 2010 • First year deliverables – Planning, basic capabilities • Detailed plan for infrastructure (6 modules) • Basic capability for first two modules • Protocol for T&E • Plan for & commence collaboration with HMT • Second year deliverables – Basic capabilities, benchmarks • Basic capability for all 6 modules • Benchmark for modules 2, 3, & 6 • T&E for Benchmarks 2 & 3 • Contribute to NCEP implementation for modules 2&3 (FY11-12) • Contribute to design and evaluation of HMT, HWT, & HFIP ensembles • Third year deliverables – Benchmarks, T&E • Benchmark for modules 4-6 • T&E for Benchmark 4 • Contribute to NCEP implementation for module 4 (FY12-13) • TET of community methods for modules 2&3 • Fourth year deliverables • TET of community methods for all modules • Update benchmarks

15. DET MODULES Module 1: Configuration Module 2: Initial Perturbations Module 3: Module Perturbations Module 4: Statistical Post-Processing External Input(HMT, HWT, HFIP, etc) Module 5: Product Generation Module 6: Verification

16. QUESTIONS • Interest from agencies beyond NOAA? • AFWA, ONR? • Expansion to global ensemble? • Needed? Expected benefits? • Resource implications? • NUOPC interagency interest? • EnKF and similar DA applications tackled jointly with DTC GSI task? • DET – Creation of ensemble • GSI – Data assimilation step

17. QUESTIONS – CONT. • Candidate methods for testing? • How to prioritize? NCEP involvement at all steps • How to measure success for DET? • Define metrics for quality & performance • Improvement over benchmark of tested methods? • Where to perform DET T&E? • Only limited resources on host (GSD, NCAR) computing facilities • In search of other opportunities… • Community outreach • Alternate biannual workshops • DET Ensemble Research / Development Workshop • NCEP Ensemble User Workshop • User support via tutorials, what else?

18. WORKSHOP EXPECTATIONS • Review ongoing research for potential testing by DET • Identify candidates and possible collaborative efforts • Review DET activities / plans • Are community and NCEP needs met? • DET’s role in national mesoscale ensemble effort • Breakout Group discussions on above • WRF Ensemble WG meeting Friday am • Further discussion and recommendations

19. OUTLINE / SUMMARY • Why and how DET was formed? • Community and NOAA ready • Sept 2009 Workshop • Major tasks • Infrastructure, test & evaluation, transition • Timeline • Planning, basic capabilities, benchmark, testing • Questions • For WG discussion • Workshop expectations • Recommendations for DET infrastructure & candidates for testing

20. BACKGROUND

21. GSI-EnKF Hybrid Approach EnKF initial perturbations and forecasts Analysis error estimate Ensemble-based forecast error covariance estimate EnKF sequential DA GSI var. DA • Data assimilation step done twice • Initial perturbations reflect uncertainty in ENKF, not GSI

22. Alternate Hybrid Approach ET (or other) initial perturbations and forecasts Ensemble-based forecast error covariance estimate Analysis error estimate GSI var. DA • Only one (variational) Data assimilation • Initial perturbations reflect uncertainty in GSI • Challenge to create good estimate

23. HOW TO REPRESENT MODEL RELATED UNCERTAINTY? • Multiple models or model versions • Pragmatic approach • Different models / model versions being developed by community • Instead of competition, use them all in ensemble mode • Diversity of opportunity - Not by “design” • Ad hoc approach, trial and error – try in ensemble, see if it works or not • Scientifically not well founded? • If we wanted to intentionally create different models / model version • What design directives could we use? • Stochastic modeling • Build a single model that can represent all model related uncertainty by design • Consider stochastic effect of subgrid-scale and other unresolved processes • Scientifically more appealing • Requires community to embrace new paradigm • Focus on model development for ensemble use • Different groups work on various aspects of same model?