 |
|
 |
||||||
|
||||||
 |
Home
| Mission
|
about SciDAC
|
Contact Us |
|||||
 |
 |
 |
||||
|
Alumni ProjectOverview of TOPS – Terascale Optimal PDE SimulationsPIs: S. Benson1, G. Biros5, X. Cai8, E. Chow3, J. Demmel7, F. Dobrian6, J. Dongarra9, M. Dumett3, V. Eijkhout9, R. Falgout3, O. Ghattas4, B. Hientzsch5, P. Husbands2, D. Keyes6, M. Knepley1, S. Li2,7, T. Manteuffel8, O. Marques2, S. McCormick8, M. Minkoff1, J. Moré1, T. Munson1, E. Ng2, A. Pothen6, R. Serban3, B. Smith1, P. Vassilevski3, O. Widlund5, C. Woodward3, H. Zhang1, ISIC affiliates: R. Armstrong (CCTTSS), D. Bailey (PERC), D. Brown (TSTT), P. Colella (APDEC), L. Freitag (TSTT), J. Glimm (TSTT), Applications affiliates: A. Bhattacharjee (CMRS), S. Jardin (CEMM), K. Ko (AST), A. Mezzacappa (TSI), H. Najm (CFRFS) 1Argonne National Lab, 2Lawrence Berkeley National Lab, 3Lawrence Livermore National Lab, 4Carnegie Mellon U., 5New York U., 6Old Dominion U., 7U. California-Berkeley, 8U. Colorado-Boulder, 9U. Tennessee The Terascale Optimal PDE Simulations (TOPS) project is researching, developing, and deploying a toolkit of open source solvers for the partial differential equations (PDEs) that arise in many application areas, including fusion, accelerator design, and the collapse of supernovae. These algorithms - primarily multilevel methods - aim to reduce computational bottlenecks by one or more orders of magnitude on terascale computers, enabling scientific simulation on a scale heretofore impossible. Along with usability, robustness, and algorithmic efficiency, an important goal is to attain the highest possible computational performance by accommodating to distributed hierarchical memory architectures. Nonlinear PDEs give mathematical expression to many core DOE mission applications. PDE simulation codes require implicit solvers for the multirate, multiscale, multicomponent, multiphysics phenomena of hydrodynamics, electromagnetism, chemical reaction, and radiation transport. Problem sizes are typically now in the millions of unknowns; with emerging multi-terascale systems, this size is expected to increase a thousand-fold over the span of the project. Moreover, such simulations are increasingly used in parameter identification, process control, and optimization contexts, which require repeated, related PDE analyses. The convergence rates of conventional solvers degrade as the size of the system increases. This creates a double jeopardy for applications – as the cost per iteration grows, so does the number of iterations. Fortunately, the physical origin of these problems provides a natural way to generate a hierarchy of approximate models, through which the required solution may be obtained efficiently, by bootstrapping. The most famous examples are multigrid methods, but other types of hierarchical representations are also exploitable, making use of lower fidelity models, lower order discretizations, and even lower precisions. The philosophy that underlies optimality is to make the majority of progress towards a high quality result through inexpensive intermediates. The efforts defined for TOPS, the co-PIs undertaking them, and the collaborations with other projects have been chosen to revolutionize large-scale simulation through incorporation of existing and new optimal algorithms and code interoperability. TOPS provides support for the software packages Hypre, KINSOL, PETSc, ScaLAPACK, Sundials, SuperLU, TAO, and Veltisto, some of which are in the hands of thousands of users, who have created a valuable experience base on thousands of different computer systems. Most PDE simulation is ultimately a part of some larger scientific process that can be hosted by the same data structures and carried out with many of the same optimized kernels as the simulation itself. By adding a convenient software path from PDE analysis to eigenanalysis, for instance, PDE analysis can be pipelined into an analysis of the stability of the PDE solution in the presence of small perturbations. With singular value analysis, a PDE analysis can be pipelined into the construction of reduced-order models - all with reuse of distributed data structures and optimized solver kernels. Similarly, TOPS will increase the value of simulations executed in a fully nonlinearly implicit solver framework by providing sensitivity analysis of the solution with respect to parameters and optimization (parameter identification, control, design, and data assimilation). TOPS linear and nonlinear solvers and time-integrators constitute core technologies for these higher-level tasks (see Figure 1).
Figure 1. The five classes of solver software to be developed as part of TOPS. All five, in a PDE context, share grid-based data structures and considerable parallel software infrastructure. In its first year, TOPS has provided users of its Bell-prize winning PETSc toolkit (mainly in fluid flow and fusion, but also externally in medical imaging, reservoir modeling, etc.) with all of the solvers in Hypre and SuperLU, without change of the user interface. TOPS is working with users to tune the new solvers, which offer scalability to finer resolutions on more processors. TOPS also powers highly resolved modal analysis of fields in accelerators in the AST project. Internally, PETSc and Hypre have been rewritten using SIDL, created by the CCTTSS ISIC. Model applications in the PETSc distribution are used by the PERC ISIC to demonstrate their performance analysis and optimization tools. Within TOPS, sparse matrices from SciDAC applications are being used in memory hierarchy-based performance improvements. Two important TOPS collaborations that will open the path to many applications beyond those that employ its solvers directly are: on-going work to support APDEC’s adaptively refined meshes and planned work to support TSTT’s overlaid meshes and high-order discretizations. Adaptive and/or high-order discretizations are ultimately required by AST, TSI, CEMM, and CMRS, with which TOPS collaborates directly at present using simpler meshes and discretizations. TOPS co-PIs also perform fundamental algorithmic research in response to known application requirements. For instance, a new form of multigrid relaxation targets Maxwell problems in electromagnetics. A new nonlinear form of preconditioning targets coupled nonlinear problems. Physics-based preconditioning suggested by users is driving development of the application-solver interface.
|
||||
Home | ASCR | Contact Us | DOE disclaimer |
|
|
