Large-Scale Simulations of Turbulent Stellar Convection Flows
and
the Outlook for Petascale Computation

Paul Woodward
University of Minnesota

We wish to simulate the helium flash in an asymptotic giant branch (AGB) star of a few solar masses that is a member of an early generation of stars and hence is metal poor. As the convection zone above the helium burning shell grows and approaches the region containing unburned hydrogen, the turbulent convection can entrain some of the stably stratified, hydrogen-rich fluid above it due to shear instability and subsequent material transport. This entrained fluid, after being transported downward by the turbulent convection flow, will eventually burn when it reaches a region of sufficiently high temperature. The resulting flame front will propagate through this turbulent flow at a speed dependent upon the fraction of fuel that has mixed into the gas and the turbulent velocity field. The nuclear burning will determine the abundances of s-process elements that are produced as well as the amount of energy released. The energy release reacts back on the flow by generating buoyancy, but the entropy difference that creates this buoyancy will be strongly modified by turbulent material mixing. Turbulence plays an essential role in this entire process, and without resolving, or at least accurately modeling this turbulence we cannot expect to accurately predict the outcome, either in terms of the rate at which energy is released or in terms of the s-process element abundances produced.

With present computational capabilities, we have the power to perform first-principles simulations of any of these turbulent processes, such as the entrainment of the hydrogen-rich gas or the flame propagation, separately in a simplified and isolated context. Using the results of these simulations, we can construct models of the phenomena which will allow computations of more of the full system to proceed, but which will necessarily reduce our confidence in the result. This process of simulation and modeling will be illustrated by studies of homogeneous, compressible turbulence and of turbulent fluid mixing in shear layers carried out by my team at the University of Minnesota during the last few years. Results of simulations performed on grids of up to 8 billion cells will be presented, and an analysis of these computational experiments in terms of statistical models will be discussed.

It is also interesting to look forward to petascale computational capabilities and to see what they will enable in the context of this specific scientific problem. First, a simplified model of a generic petascale computing system will be set out and 3 coding approaches briefly contrasted that could each take advantage of up to a million CPU cores for simulating the helium flash in an AGB star. A major benefit of petascale computation for this problem would be to allow simulation for a substantial period of time of the entire spherical shell including the convection zone above the helium burning shell and its environs. If we simply scale up the simple approach of our sPPM benchmark code to petascale computation, we can afford to simulate this entire region in complete detail for simulated times of the order of hours, allowing for many turnover times of the largest convective eddies. However, carrying out such a first principles simulation for the month-long duration of the helium flash would be impractical. To simulate the entire helium flash, we need to introduce models for the small-scale turbulent behavior and to reduce the grid size substantially. To run this much smaller problem on the same petascale equipment requires that we program teams of processors, say 64 CPU cores to a team, to work cooperatively in a very tightly coupled mode to update small portions of the problem domain. We are already developing such programs for the Cray XT3. Present estimates suggest that this approach would, on petascale hardware, allow the entire, month-long helium flash to be simulated in a couple of weeks. In a third approach, we would allow the computation to become highly irregular by using AMR techniques to place grid cells only where they are most needed and by handling multifluid advection and nuclear reactions only where they are needed. This introduces a large benefit by reducing the necessary amount of computation, but it also introduces major challenges for efficient parallel computation. These benefits and challenges will be briefly discussed.

After considering how long even a petascale computing system would require to simulate the helium flash, one might well ask why we don’t consider some simpler scientific problem while we wait for this machine to be built. With present computing capabilities we can, nevertheless, make very significant progress. And this progress, namely the creation and validation of the statistical models of the turbulent phenomena and the proper incorporation of them into our codes, is absolutely necessary if we are to solve this problem in a practical simulation once the petascale hardware arrives. Using these models on today’s hardware, we can simulate the helium flash process in detail for short periods set at intervals over the time in which the helium burning luminosity shoots up, and we can simulate the longer periods between these bursts of detailed computation with 1-D models that are enlightened by these bursts of detailed information. Also, our need for models of turbulent mixing and turbulent flame propagation is shared by a broad community of scientists and engineers, so that our work on this problem should have benefits beyond the understanding it brings us of the helium flash phenomenon and the s-process elements it produces.