Particle Physics Data Grid (PPDG): An Interdisciplinary Collaboration Contributing to and Benefiting from Shared Grid Technology

PIs: Richard Mount, SLAC, Miron Livny, Wisconsin, Harvey Newman, Caltech
Steering Committee: John Huth, Harvard (ATLAS), Tim Adye, RAL (BaBar), Lothar Bauerdick, FNAL (CMS),
Lee Lueking, FNAL (D0), Chip Watson, TJNAF, Jerome Lauret, BNL (STAR),
Miron Livny, Wisconsin (Condor), Jennifer Schopf, ANL (Globus), Ian Foster, ANL (Globus),
Reagan Moore, SDSC (SRB), Arie Shoshani, LBNL (SRM)
Coordinators: Ruth Pordes, FNAL, Doug Olson, LBNL
Liaisons: Paul Avery (iVDGL), Larry Price (HICB), Mike Wilde(GriPhyN), Torre Wenaus, Ian Bird (LCG)
(www.ppdg.net)

Summary

The close collaboration of computer science teams with the physicists and software engineers in the PPDG collaboration is contributing extensions to, and deployment of, Grid software technologies of benefit to the larger scientific community. The improved technology is then shared with other SciDAC projects and collaborators. Some samples of these benefits are given below.

The PPDG collaboration, consisting of four leading grid middleware groups and six high-energy and nuclear physics experiments, has proven to be very effective at identifying, developing, and adopting improvements to grid middleware beneficial not only across different physics experiments but for the broader science community as well. The interaction with other SciDAC projects provides an important mechanism for mutual sharing of technology improvements across the science community.

PPDG has contributed many improvements to the Globus Toolkit¹ in the form of requirements specifications, testing and hardening of many components including Grid Security Infrastructure, GridFTP, Replica Location Service, Monitoring and Discovery Service, GRAM job submission, and the Community Authorization Service.

PPDG had a similar impact on technologies and middleware developed by the Condor project² – most noticeably the grid enabled job manager (Condor-G) and workflow management. Development of new features in Globus and Condor are supported by the SciDAC projects "A High-Performance Data Grid Toolkit" and "Security and Policy for Group Collaboration." Improvements to the Globus Toolkit and Condor are shared across the whole science community as they have been adopted by most grid applications as key infrastructure.

Collaboration with PPDG has led to the addition of key features to the Storage Resource Manager³ (SRM), including space reservation and management needs of disk and tape storage resources. Based on the extensive PPDG experience, the SRM technology has been used by other SciDAC projects, the Earth Science Grid (ESG), and Lattice QCD (LQCD). ESG benefits from an interface with High-Performance Storage System (HPSS) as well as a version that works with a large shared disk. In LQCD, SRM-compliant storage interfaces are being deployed at JLab and the international LQCD grid. SRM was also easily adapted to the National Center for Atmospheric Research (NCAR) mass storage system. Development of SRM is supported by the SciDAC middleware "Storage Resource Management for Data Grid Applications" project and includes technology and optimizations from the SciDAC-supported "Scientific Data Management Integrated Software Infrastructure Center" (SDM ISIC).

Improvements to the Storage Resource Broker⁴ (SRB) from work in PPDG includes extensions to the metadata catalog based on requirements of PPDG experiments as well as abstraction of the data transport interfaces to support the diverse data access technologies used across different applica-tion communities. Ongoing work includes developing the capability to federate SRB metadata catalogs useful to the globally distributed nature of large physics experiment data handling needs.

PPDG collaborated closely with the DOE Science Grid to establish a Registration and Certificate Authority⁵ (CA) for the science research community, called the DOEGrids CA. PPDG and its colleagues have been early users of the facility, and helped extend adoption of these certificates by the wider particle physics community in the US, and developed mutual trust with the European Data Grid⁶ project in Europe. The DOEGrids CA now supports numerous communities including ESG and the FusionGrid.

A specific example of benefits being shared with other science domains is the work on job scheduling for the D0 experiment in PPDG has led to extensions to Condor-G that facilitate late binding between a job and a grid resource. This new feature was used to make the task of analyzing very large collections of proteins by the Basic Local Alignment Search Tool (BLAST) tool more effective.

In addition to the shared technology benefits accomplished to date in PPDG, a number of additional opportunities have been identified. Development of a web services interface to Condor-G from the SciDAC-supported "Pervasive Collaborative Computing Environment" will likely be of benefit to PPDG applications in their future transition to Open Grid Services Architecture compliant grids. The high-dimensional indexing techniques from the SDM ISIC are of potential benefit to PPDG experiments as a flexible, more efficient way of getting the desired subsets of events from the Grid in the data analysis phase. Discussions with the SciDAC project for "Distributed Security Architectures" are helpful as the resource authorization needs of the physics experiments’ grids become better defined. Collaboration with the "Bandwidth Estimation, Measurement Methodologies, and Applications" project contributes to the PPDG monitoring group and development of future plans for performance monitoring and troubleshooting of end-to-end applications.

¹ http://www.globus.org/toolkit
² http://www.cs.wisc.edu/condor
³ http://sdm.lbl.gov/srm-wg
⁴ http://www.npaci.edu/DICE/SRB
⁵ http://www.doegrids.org/
⁶ http://www.eu-datagrid.org/

back to project page