PARAMESH: A PARALLEL, ADAPTIVE GRID TOOL FOR THE SPACE SCIENCES Kevin Olson Drexel University Philadelphia, PA Presen

1. PARAMESH: A PARALLEL, ADAPTIVE GRID TOOL FOR THE SPACE SCIENCESKevin OlsonDrexel University Philadelphia, PAPresented, AISR MeetingMay 2008University of Maryland

2. COLLABORATORS Peter MacNeice (NASA/GSFC) Joan Centrella (NASA/GSFC) Don Lamb (U. of Chicago) other collaborators include: C. Mobarry, R. DeFainchtein, M. Gehmeyer, M. Bhat, C. Packer, M. Rilee, J. VanMetre, D. Choi (NASA/GSFC) R. Devore (NRL), D. Swesty, A. Calder, M. Zingale (SUNY/SB), J. Dursi, K. Riley, A. Siegel, T. Linde, D. Sheeler, A. Dubey, K. Weide (U. Chicago) Initial funding provided by NASA/ESTO-CT

3. TALK OUTLINE AN OVERVIEW OF PARAMESH AISRP GOALS and ACCOMPLISHMENTS SOME APPLICATIONS and SCIENCE RESULTS USING PARAMESH

4. An Quick Overview of PARAMESH

5. PARAMESH: what is it ? A package designed to ease the task of adding parallelization and dynamic, adaptive mesh refinement (AMR) to an already existing uniform mesh, serial code A library of subroutines and accessible data structures Written in Fortran90 and C (NAG, Lahey, Intel, Portland Group, HP-Compaq, IBM, SGI, g95) Interprocessor communication using MPI Version 4.1 released March 2008. PARAMESH WEB site: http://www.physics.drexel.edu/~olson/paramesh-doc/Users_manual/amr.html

6. A subset of Berger-Oliger, block-adaptive scheme Computational Volume is recursively bisected into �blocks�, forming a tree data structure.

7. Blocks are ordered and distributed to processors using a space filling curve.

8. Each Block is a logically cartesian, uniform mesh of cells. Each cell in a block can store user specified data at cell centers, corners, edges or faces

9. Support for consistent fluxes and ensuring conservation for finite volume schemes Support for averaging data at cell edges to ensure consistent circulation integrals around cell faces

10. ACCOMPLISHMENTS UNDER AISR

11. AISR ACCOMPLISHMENTS Extend and improve PARAMESH Parallel I/O (HDF5, MPIIO), I/O formats for the graphics packages ChomboVis and Visit. C Interface Improved support for multigrid solvers Improve divergence of B control (Balsara algorithm supported, used in FLASH 3.0) Improved support for non-cartesian coordinate systems (cylindrical and spherical coordinates supported) Many performance enhancements.

12. AISR ACCOMPLISHMENTS Effective Open Source Development Code managed using Sourceforge Developed coding standards and a developers� guide was posted to the WEB site. Improved automatic testing procedure. Self-documenting comments using �Robodoc�. Hard to find other developers. Users make feature requests, but are not interested in developing code (exception was FLASH code team). Integrated new versions of PARAMESH into actual, working, space science applications (ARMS, HAHNDOL, FLASH)

13. SOME SPACE SCIENCE APPLICATIONS and RESULTS USING PARAMESH(ARMS, HAHNDOL, AND FLASH)

14. Other Space Science Applications CASIM (M. Benna and P. Mahaffy at GSFC) MHD application for modeling comet-solar wind interaction YDFCT (D. Odstrcil at NOAA) MHD application for modeling multiple interacting CME�s, integrated into CCMC ZeusAMR (W. Abbett et al. at U.C. Berkeley) Combination of Zeus MHD code and PARAMESH for modeling magnetic flux emergence from the sun IBEAM (D. Swesty et al. SUNY-SB) Modern Astrophysics framework, radiation hydrodynamics for modeling gamma ray burst fireballs Plus others, the list continues to grow, hundreds of users worldwide.

15. ARMSR. DeVore (NRL)P. MacNeice, K. Olson (NASA/GSFC) Solves the equations of MHD (DeVore, 1991) Code for which PARAMESH was originally developed Used for solar physics applications Numerical schemes: FCT with constrained transport for MHD and multigrid for implicit formulation of non-linear thermal conductivity

16. Coronal Mass Ejection S. Antiochos (NASA/GSFC) et al.

17. Solar Polar Jet FormationS. Antiochos (NASA/GSFC) et al.

18. HAHNDOLJ. Centrella, D. Choi, B. Imbiriba, J. Baker, D. Fiske, & J. Van Meter (NASA/GSFC), D. Brown, L. Lowe (N.C. State) General Relativity code, Solves Einstein Equations Goal: To simulate gravitational waves resulting from the collision of super-massive black holes in order to help interpret data from LISA mission (to be launched someday). Numerical Schemes: Multigrid, Finite Difference Incorporates latest version of PARAMESH. Used to perform one of the largest simulations on Columbia System at NASA/AMES

19. Gravitational Wave Propagation(J. Centrella et al., NASA/GSFC)

20. FLASH ASTROPHYSICS CODEFLASH code team, University of Chicago Fryxell et al., 2000, ApJS, 131, 273. WEB site: www.flash.uchicago.edu Implements various CFD Schemes, MHD, Nuclear Reactions, Stellar Equations of State, and self-gravity using multigrid. Designed to model Astrophysical thermonuclear �flashes� (X-ray bursts, Novae, and Type 1a Supernovae). FLASH 3.0 recently released. Incorporates version 4.0 of PARAMESH.

21. X-ray Burst (M. Zingale, SUNY/SB)

22. FLASH Validation Experiments

23. FLASH Validation Experiments

24. Type Ia SupernovaD. Lamb, et al., U. of Chicago

25. CONCLUSIONS Parallel, Adaptive Mesh Refinement has wide applicability in the space sciences PARAMESH provides a useful and flexible tool for adding parallel AMR to a wide variety of applications, allowing the efficient solution of problems in the Space Sciences. The major goals that were originally proposed to AISR were accomplished. Thank you AISR !