Assessment of Predictive Capability

1. Assessment of Predictive Capability James Paul Holloway CRASH Review Meeting October 19 2009

2. Several initial UQ projects are complete • Investigated (with CRASH 1D) the importance of: • Initial radiation field, boundary conditions, EOS, initial field fit • Explored (with Hyades 1D) sensitivity of shock location at 13 ns over 15-D input space • Included experimental, physical and numerical parameters • Constructed a physics informed emulator (PIE) for CRASH 1D initial fields at 1.3 ns • Identified 40 scalar parameters to describe Hyades output fields • Constructed an emulator for these over the input variables • Investigated of sensitivity of PIE data to 15 Hyades inputs • Constructed a predictive model over 8-D input space CRASH 1D • Performed sensitivity study of 3D CRASH over a 4-D input space See later talks by Fryxell & Bingham and various posters

3. Predictive Roadmap • Emulator construction • Calibration of physics parameters • Estimating predictive distributions • Analysis elements: • Experimental design • Sensitivity studies • Year 1 & 2 • Integration of team members • Exercise of analysis tools using Hyades, CRASH 1D and CRASH multi-D • Year 3 • Use analysis & experiments to identify and improve modeling elements – quantify sources of error • Numerical error estimation • Uncertainty due to experimental and physics parameters • Suggest experiments based on UQ analysis • Improve preprocessor (PIE) and extend to 2D • Demonstrate improved predictions • Year 4 • Analysis of 5th year experimental regime to estimate sensitivities and predictive distribution • Suggest experiments based on UQ analysis • Year 5 • Provide improved predictions of 5th year experiment and prediction pdf

4. We are heading towards the year 5 elliptical tube • Do computations and experiments with circular tubes • Can do computations with elliptical tubes • Must predict output y and its pdf for the elliptical tube Input space x Year 5 experiment Simulations Year 1-4 experiments

5. We will use statistical modeling to guide improvement of the CRASH code • Use assessment of predictive capability to guide • Improvement in CRASH code physics and numerics • Identification of needed field experiments or diagnostics to produce predictions for an extrapolated field experiment • Aim towards predicting results in an unexplored region of the field experimental input space (the oval tube) • Carry this out with a first principles physics code, avoiding the use of tuning parameters or unphysical choices of physics parameters Experiments Assessment Code

6. Predictive science is more than prediction • Knowledge of the sensitivity of the output to variations in the input • Which input parameters, when varied over their probable ranges, generate the significant variations in output • An understanding of the significant sources of uncertainty in output • An estimate of the predictive distribution of outputs over • Uncertainty in physics parameters • Uncertainty in experimental parameters • Uncertainty in statistical fitting parameters • Successful prediction means that the experimental result is, with reasonable probability, within the range predicted by the code

7. We understand CRASH as a map from input to output • CRASH maps inputs to output • We want to characterize the variations in around interesting ranges of inputs • Sensitivities • Predictive distributions • Numerics converged sufficiently to be small contributor to overall uncertainty Focus of current UQ efforts

8. Some of our key inputs include: • Laser energy and pulse shape • Initial Xe pressure • Be drive disk thickness • Time of radiograph observation • Tube geometry • EOS data • Opacities • Mesh parameters • Limiter choice • Time step controller These are uncertain These are uncertain These introduce error

9. Key outputs include: • Shock position at observation time • Distance of edge of shock from tube wall • Angle of xenon edge just downstream of shock • Average thickness of Xe layer Determine sensitivities of andpredict pdf of outputfor a new experiment , accounting for uncertain inputs X-ray radiograph from experiment with average velocity 140 km/sec at 14 ns

10. We have explored the influence of problem setup • Initial radiation field • Left (irradiated) boundary location • EOS treatment • Initial field fit Varied mesh, initial field, boundary location

11. We have build a physics informed emulator: PIE • Use a set of 10 key spatial locations and interpolate hydrodynamic fields at 1.3 ns • Create mapping of Hyades inputs to these 40 parameters • Uncertainty in fit will contributeuncertainty to CRASH results

12. Sensitivity analysis using statistical fitting • Sample input space (Latin hypercube sampling) • Compute outputs • Fit a model • Gaussian Process model • Flexible regression models • Estimate global sensitivity due to an input by: • perturbing input variable randomly over samples • refitting • computing RMS difference • Estimate local sensitivities using length parameters from GP model • Marginalize over other variables to explore variation due to one or two variables The LHD used for CRASH 3D sensitivity study

13. We can identify important influences on shock location Based on 512 Hyades runs and MART fit Informs selection of parameters for 3D study • Be Thickness • Laser Energy • Xe Density • Be and Xe gammas • Be and Xe opacity scale factors See related study on CRASH 3D in Fryxell’s talk

14. A predictive model provides a pdf of outputs • Construct predictive model of outputalong with posterior calibration pdf • Combine with estimated or input ranges to account for input uncertainty • Such a model provides: • Mean and variance of output – predictive distribution • Tool to explore error in the model relative toexperiment • Approach to determine a next experiment or simulationto reduce uncertainty

15. CRASH 1D Predictive Model Construction

16. Measurements • Select 8 as training data, predict a 9th.

17. The predictive model gives us a way to explore physics • Data for model are field experiments and set of code runs • Model data as sum of 3 terms: • Emulator of the code (Gaussian process model) • Model discrepancy (Gaussian process model) • Measurement error • Large discrepancy compared to measurement uncertainty implies prospects for prediction are poor • physics or numerics of code is wrong • or measurements are misleading • Code improvements can be judged against reduction in discrepancy in key regions of input space • Must avoid conflating discrepancy with calibration Results using CRASH 1D in Bingham’s talk

18. Our immediate next steps build on this work • Explore the CRASH 1D Predictive Model • Explore discrepancy over inputs and reflect on the physics • Compare joint computation of discrepancy and calibration with sequential calibration and computation of discrepancy • Repeat with multi-group diffusion and explore discrepancy • Include uncertainty in initialization • Quantify sources of uncertainty • Use predictive model to suggest new code runs and experiments • Minimize uncertainty • Minimize discrepancy • Move towards a predictive model using CRASH multi-D • Propagate uncertainty information through EOS/Opacity model

19. We are heading towards the year 5 elliptical tube • We can perform • Experimental design • Sensitivity studies • Emulator construction • Calibration of physics parameters • Estimating predictive distributions • Next steps • Use analysis & key experiments to identify and improve key modeling elements – quantify sources of error • Numerical error estimation • Uncertainty due to experimental and physics parameters • Suggest experiments based on UQ analysis • Improve preprocessor (PIE) and extend to 2D • Demonstrate improved predictions • Analysis of 5th year experimental regime to estimate sensitivities and predictive distribution • Suggest experiments based on UQ analysis • Provide improved predictions of 5th year experiment and prediction pdf Year 5 experiment Year 1-4 experiments

20. Related Posters • Automatic Detection of Shock Wave Features N Patterson, R. P. Drake, K Thornton • Building a Dataset of Results Modeling Radiative Shocks in 1D HYADES for Uncertainty Quantification M. J. Grosskopf, R. P. Drake, B. Fryxell, F. W. Doss, C. C. Chou, D Bingham • Calibrating HYADES as initial conditions for CRASH MJ Grosskopf, R. P. Drake, B Torralva, F. W. Doss • Gaussian Process Regression and Bayesian MARS for CRASH Initialization with HAYDES Duchwan Ryu, Ryan McClarren, Anirban Mondal, Zach Zhang, Ashin Mukherjee, Jason Chou, Bani Mallick, Derek Bingham, Bruce Fryxell, Vijay Nair, Ji Zhu • Gradient-Enhanced Stochastic Expansion: A Survey of Response Surface Construction Methods Colin Miranda, Krzysztof Fidkowski, Kenneth Powell • Preliminary 1D CRASH Resolution Study / Variable Screening Chuan-Chih (Jason) Chou, Paul Drake • Verification of Uncertainty Quantification Software Bruce Fryxell, Jason Chou, Ashin Mukherjee, Zach Zhang, Vijay Nair, Mike Grosskopf, Derek Bingham, Bani Mallick, Duchwan Ryu

21. Questions?

22. Additional Slides Follow

23. CRASH Personnel directly engaged in UQ • Bruce Fryxell, James Holloway, Paul Drake • Derek Bingham, Vijay Nair, Bani Mallick • Marvin Adams, Ryan McClarren • Jason Chou • Chris Fidkowski • Mike Grosskopf • Anirban Mondal • Ashin Mukherjee • Avinash Prabhakar • Duchwan Ryu • Zach Zhang • Ji Zhu

24. Sensitivity study shows mesh convergence Average shock location at 13 ns converged Average shock location at 13 ns converged

25. Hyades emulator construction and sensitivity runs • 15 dimensional input space sampled • 512 point space filling LHD constructed • TAMU constructed PIE emulator for 1.3 ns • UM carried out sensitivity analysis for shock location at 13 ns

26. Experimental Design for CRASH 1D analysis • 256 Points over 8 dimensions (4 thetas, 4 x’s) orthogonal array based space-filling LHD8 purely for computer run inputs • 64 Points over the product space