Alumni Project

Electromagnetic Systems Simulation – Prototyping through Computation

N. Folwell1 , L. Ge1 , A. Guetz1 , V. Ivanov1 , K. Ko1 ( PI ), M. Kowalski1 , R. Lee1 , Z. Li1 , C. Ng1 , Y. Sun1 , R. Uplenchwar1 , M. Wolf1

Affliliated Researchers:
E. Ng3 , W. Gao3 , P. Husbands3 , X. Li3, A. Pinar3 , C. Yang3 (SAPP/TOPS), G. Golub2 , Y. Liu2 , K. Ma4 , G. Schussman4 (SAPP), D. Brown5, K. Chand5 , K. Devine6 , L. Diachin5 , W. Henshaw5 , P. Knupp6 , Y. Luo7 , M. Shephard7 , T. Tautges6 , D. White5 (TSTT) 1 Stanford Linear Accelerator Center, 2 Stanford University, 3 Lawrence Berkeley National Lab., 4 UC Davis, 5Lawrence Livermore National Lab., 6 Sandia National Lab., 7 Rensselear Polytechnic Institute

Summary

The Office of Science's accelerator facilities have started to reap benefits from DOE's investment in Terascale Computing as an increasing number of challenging design and analysis problems are being solved through large-scale simulations. Central to this major advancement is the SciDAC Accelerator Simulation Project which supports the development of a suite of parallel electromagnetic codes significantly strengthened through computer science and applied mathematics (CS/AM) collaborations that improve their accuracy and maximize their performance on Terascale Computers.

1. Introduction

The DOE's SciDAC project, “Advanced Computing for 21st Century Accelerator Science and Technology”, has supported a well-integrated, multi-institutional, multi-disciplinary team to focus on the large-scale simulations necessary for the design and optimization of electromagnetic systems essential to accelerator facilities throughout the Office of Science complex. Significant progress has been made in the development of a set of parallel electromagnetic codes that are further improved through SAPP and ISICs collaborations, and in the application of this comprehensive terascale simulation capability to challenging problems facing major DOE accelerator projects such as the Positron-Electron Project (PEP)-II, Next Linear Collider (NLC), and the Rare Isotope Accelerator (RIA).

2. Parallel Code Development and CS/AM Collaborations

Two new field solvers, S3P that calculates the scattering matrix of open structures and T3P , a finite element time-domain solver, have been added in the past year to the suite of 3D parallel electromagnetic codes that already consists of Omega3P (eigensolver), Tau3P (transient solver), and Track3P (particle-tracker). These codes comprise a comprehensive toolset for modeling near- and long-term projects. SAPP and ISIC collaborations in CS/AM to further advance this new software capability include:

a). New algorithms such as AV formulation and a multilevel hierarchical preconditioner to accelerate convergence in Omega3P ,

b). A linear solver framework that enables the use of either direct solvers (e.g., WSMP, SuperLU) or iterative methods (Iterative Template Library) in large sparse systems,

c). Adaptive mesh refinement in Omega3P and S3P to improve solution accuracy at much reduced computational cost,

d). Better partitioning schemes for modeling with Tau3P to obtain better scalability (e.g., RCB1D in place of ParMETIS),

e). Advanced visualization techniques to extract useful features efficiently from large, multi-stream, unstructured data sets.

These efforts have contributed to the success of many simulations with 10X improvement in speed, accuracy, and convergence rates in some cases. Further gains can be expected as these technologies continue to mature.

3. Electromagnetic Systems Simulations

a). PEP-II , a High Energy Physics (HEP) facility, currently operates at 2X design luminosity and is aiming for another twofold increase. Beam heating in the Interaction Region (IR) could limit the accelerator from meeting that goal. Tau3P has been used to calculate the heat load in the present design (Fig. 1). Higher currents and shorter bunches will require modifications to the IR to reduce the increased heating. T3P will be able to model the entire IR geometry (from crotch to crotch) and to simulate the actual curved beam paths in any new IR design.

Figure. 1 Tau3P simulation of beam heating in PEP-II IR.
Figure. 1 Tau3P simulation of beam heating in PEP-II IR.

b). RIA , a proposed Nuclear Physics project ranking 3 rd in SC's 20-year Science Facility Plan (SFP), is designing a variety of RFQ cavities (Fig. 2) for its low-frequency linacs. Due to lack of accurate predictions, tuners are designed to cover frequency deviations of about 1%. With Omega3P, frequency accuracy of 0.1% can be reached which will significantly reduce the number of tuners and their tuning range. Parallel computing will play an important role in prototyping RIA's linacs and help reduce their costs.


Figure. 2 (Left) A model of RIA's hybrid RFQ and (Right) an enlarged portion of the corresponding Omega3P mesh.

c). NLC , a TeV scale accelerator for HEP research and of high priority on the 20-year SFP, adopts the Damped Detuned Structure (DDS) as its main linac design. The DDS uses damping through external manifolds and detuning by cell-to-cell variation for suppressing long-range dipole wakefields to control the emittance growth of long bunch trains. Because of the complex geometry, wakefield analysis of the DDS has been limited to equivalent circuit models. Using Tau3P , the DDS has been simulated with a transit beam so that the complete wakefield was found directly for the first time (Fig. 3). Omega3P and S3P analyses in the frequency domain are under way to verify the result. Ongoing DDS simulations also include the study of surface field increase due to the finite RF drive pulse ( Tau3P) , and dark current generation resulting from surface emissions under high fields ( Track3P) .

55-cell DDS model;
Transient electric field in the structure after beam transit;
Dipole wakefields in DDS (blue), detuning only
Figure 3 (Top) 55-cell DDS model; (Middle) Transient electric field in the structure after beam transit; (Bottom) Dipole wakefields in DDS (blue), detuning only (Pink).

4. Concluding Remarks

High-resolution, system-scale simulation utilizing terascale computers such as the IBM/SP at NERSC, has been made possible by SciDAC-supported code development efforts and the CS/AM collaborations with SAPP and the ISICs. This team-oriented approach to computing is beginning to make a qualitative difference in the R&D of major DOE accelerators, existing or planned.

For further information on this subject contact:
Dr. Kwok Ko
Advanced Computations Department
Stanford Linear Accelerator Center
2575 Sand Hill Road, MS27
Menlo Park, Calif. 94025
Phone: 650-926-2349
E-mail: Kwok@slac.stanford.edu

back to project page

 


Home  |  ASCR  |  Contact Us  |  DOE disclaimer