Stochastic optimization of energy systems

1. Stochastic optimization of energy systems Cosmin Petra LANS@MCS Argonne National Laboratory

2. A) Project Overview • Real-time optimization (power dispatch and unit commitment) of power grid in the presence of uncertainty (renewable energy, smart grid, weather) • Stochastic formulations reduce both short-term (production) and long-term (reserve) costs, stabilize prices, and increase the reliability. • LANS@ANL team: MihaiAnitescu, Cosmin Petra, Miles Lubin (algorithms and implementation), Victor Zavala and Emil Constantinescu (modeling and data) • Funding: DOE Applied Math (2009-2012), DOE ASCR MMICC center (2012-2017) • DOE INCITE Award (2012-2013) - 10 mil core hours for 2012.

3. B) Science Lesson • What does the application do, and how? • Stochastic optimization = decisions taken now are influenced by future random conditions (multiple scenarios) • Unit Commitment: Determine optimal on/off schedule of thermal (coal, natural gas, nuclear) generators. Day-ahead market prices. (solved hourly) • Economic Dispatch:Set real-time market prices. (solved every 5-10 min.) • Scenario-based parallelization • The “now” decisions cause coupling • PIPS suite (PIPS-IPM, PIPS-S) - parallel implementations that exploits the stochastic structure at the linear algebra level.

4. C) Parallel Programming Model • MPI + OpenMP • Scenario computations accelerated with OpenMP (sparse linear algebra) • Inter-scenarios communication with MPI • Distributed dense linear algebra for the coupling (done with Elemental) • C++ • Cmake build system • Runs on “Fusion” cluster, “Intrepid” BG/P • Asynchronous implementation may require new programming model (X+SMP). • Yeah, I know … 99.99% X will be MPI

5. D) Computational Methods • Standard interior-point method (PIPS-IPM) and dual simplex (PIPS-S) • In-house parallel linear algebra • Linear algebra kernels • Sparse: MA57, WSMP, PARDISO. • Dense: LAPACK, Elemental • Next: PIPS-L – Lagrangian decomposition for integer problems • “Dual decomposition” method • Based on multi-threaded integer programming kernels (CBC,SCIP) and PIPS-IPM • Asynchronous – master-worker framework to deal with load imbalance in scenarios

6. E) I/O Patterns and Strategy • I/O requirements minimal, one file per MPI process at starting. • We end up with the optimal cost (a double) and decision variables (vectors of relatively small size) • Restarting done by saving the intermediate iterates (vectors) • Future plans: Parallel algebraic specification of the problem • Generating the input data IN PARALLEL given an algebraic/mathematical description of the problem (AMPL-like script) • Currently done in serial

7. F) Visualization and Analysis • Output is small, no special analysis required • less

8. G) Performance • Bottlenecks to better performance? • SMP sparse kernels (PIPS-IPM) • memory bandwidth (PIPS-S) • Bottlenecks to better scaling? • Dense kernels (PIPS-IPM) • load imbalance(PIPS-S, PIPS-L) • Collaboration with Olaf Schenk - PARDISO – SMP sparse rhs • PIPS-L – asynchronous optimization algorithms

9. H) Tools • How do you debug your code? • cerr, cout

10. I) Status and Scalability • PIPS-IPM scaling • Efficiency likely to decrease with faster SMP scenario computations • Factors that adversely affect scalability • Serial bottlenecks: dense linear algebra for the “now” decisions • Using Elemental improves scaling for some problems

11. I) Status and Scalability • PIPS-S scaling efficiency is • 31% on Fusion from 1 to 256 cores • 35% on Intrepid from 2048 to 8192 cores • Factors that adversely affect scalability • Serial bottleneck (“now” decisions) • Communication ( 10 collectives per iteration, cost of 1 iteration=O(ms) ) • Load imbalance • Intended to be used on up to few hundred of cores • PIPS-S is the first HPC implementation of simplex

12. J) Roadmap • 2 years from now? • Solve grid optimization models with • Better resolution and larger time horizon • Larger network: continental US grid • More uncertainty • Integer variables