The Oak Ridge Leadership Computing Facility and Fermilab are executing the integration of the Adaptive Input/Output System and the BigData Express high-speed data transfer service.
Editor’s note: This is a version of a feature article that was first released by Oak Ridge National Laboratory.
Projects large enough to run on high-performance computing (HPC) resources pack data—and a lot of it. Transferring this data between computational and experimental facilities is a challenging but necessary part of projects that rely on experiments to validate computational models.
Staff at two U.S. Department of Energy (DOE) Office of Science User Facilities — the Oak Ridge Leadership Computing Facility (OLCF) and Fermi National Accelerator Laboratory — facilitated this process by executing the integration of the Adaptive Input/Output System (ADIOS) and the BigData Express (BDE) high-speed data transfer service.
Now ADIOS and BDE developers are changing the way researchers can transport and analyze data by incorporating a new methodology into the tool that allows for compressing and streaming of data coming out of simulations in real time. The methodology is being tested by OLCF user C. S. Chang, a plasma physics researcher at Princeton Plasma Physics Laboratory (PPPL) who studies the properties of the plasmas that exist in giant fusion devices called tokamaks. Chang seeks an understanding of the power needed to run ITER and the heat load to the material wall that will surround its plasma, both of which are key to fusion’s viability. ITER is an international collaboration working to design, construct, and assemble a burning plasma experiment that can demonstrate the scientific and technological feasibility of fusion power for the commercial power grid. ITER, which counts DOE’s Oak Ridge National Laboratory (ORNL) among its partners, is currently under construction in southern France.
“If users can separate out the most important pieces of data and move those to another processor that can recognize the intended prioritization and reduce the data, it can provide them with feedback that they may need to stop a simulation if necessary,” said Scott Klasky, leader of the ADIOS framework and group leader for ORNL’s Scientific Data Group.
Wenji Wu, principal investigator of the BDE project and principal network research investigator of Fermilab’s Core Computing Division, added, “The new approach leverages the software-defining network [SDN] capabilities for resource scheduling and the high-performance data streaming capabilities of BDE.”
SDN allows users to dynamically control network resources rather than manually request to connect.
“This combination enables real-time data streaming with guaranteed quality of service, whether it be over short or long distances,” Wu said. “In addition, this approach yields small memory footprints.”
Although the project is still in the development phase, preliminary tests allowed Chang and his team to successfully transfer fusion data between the OLCF — located at ORNL — and PPPL.
“With this new methodology, users can stream data on the network without ever touching the file system and request network resources on the fly,” said ADIOS and BDE researcher Qing Liu, who has a joint appointment with the New Jersey Institute of Technology and ORNL.
Without streaming capabilities, scientists can perform only after-the-fact analyses for many experiments, such as KSTAR, the Korean Superconducting Tokamak Advanced Research. But with simulations and experiments increasing in size, near–real-time monitoring and control are becoming necessary. The new ADIOS–BDE integration could also play a major role in large experimental projects, such as the fusion project Chang is leading and the Square Kilometer Array, an effort involving dozens of institutions to build the world’s largest radio telescope. The new streaming capabilities could more easily enable the capture of short-lived events such as pulsars — neutron stars that emit electromagnetic radiation — that the telescope aims to record.
“KSTAR wants to transfer their data as the experiment is happening, to process their data during the experiment,” Klasky said. “These additions to ADIOS will enable both sides to quickly perform data analysis and visualization in real time.”
Seo-Young Noh, director of the Global Science Experimental Data Hub Center at the Korea Institute of Science and Technology Information, leads a group that has contributed significantly to the BDE project.
“Our work has made cross-Pacific, real-time data streaming possible,” Noh said.
Klasky, Liu, and their collaborators will give a best paper plenary talk related to these new capabilities—titled “Understanding and Modeling Lossy Compression Schemes on HPC Scientific Data” — at the 32nd IEEE International Parallel and Distributed Processing Symposium. The team noted that the new ADIOS methodology will allow scientists to efficiently select the type of compression that will best fit their scientific and research needs, affording them the ability to analyze their data faster than ever before.
Liang Zhang, the developer of BDE data streaming capabilities, is working with Liu to enhance and test the tool. They expect the tool’s new capabilities to be fully tested and deployed by late 2019. This work also involves ADIOS researcher Jason Wang and BDE researchers Nageswara Rao, Phil DeMar, Qiming Lu, Sajith Sasidharan, S. A. R. Shah, Jin Kim, and Huizhang Luo.
Fermilab is America’s premier national laboratory for particle physics and accelerator research. A U.S. Department of Energy Office of Science laboratory, Fermilab is located near Chicago, Illinois, and operated under contract by the Fermi Research Alliance LLC, a joint partnership between the University of Chicago and the Universities Research Association Inc. Visit Fermilab’s website at www.fnal.gov and follow us on Twitter at @Fermilab.
ORNL is managed by UT-Battelle for the Department of Energy’s Office of Science, the single largest supporter of basic research in the physical sciences in the United States. DOE’s Office of Science is working to address some of the most pressing challenges of our time. For more information, please visit science.energy.gov.
Fermilab’s Long-Baseline Neutrino Facility and Deep Underground Neutrino Experiment break new ground in many ways. Together, they make up the first international megascience experiment hosted by a U.S. Department of Energy laboratory. Using cutting-edge neutrino detection technology at a scale never before attempted, scientists will send a beam of neutrinos to a detector farther away than any previous experiment. And it represents the largest construction project to date that Fermilab will manage across multiple U.S. states, with infrastructure at Fermilab’s Illinois site and in space leased by Fermilab at the Sanford Underground Research Facility in Lead, South Dakota.
To support Fermilab services provided for LBNF, DUNE and other DOE-funded experiments that will operate at Sanford Lab, Fermilab has established a new organizational unit there and hired South Dakota native Patrick Weber to lead it.
Weber, who will take on the role of head of Fermilab’s South Dakota Services Division on June 11, currently serves as policy advisor to South Dakota Governor Dennis Daugaard. As part of his broad portfolio within the governor’s office, Weber has worked with the South Dakota Science and Technology Authority that operates Sanford Lab.
“Two of the things I’ve loved best about being in the governor’s office are getting to confront a wide range of issues and being involved with SDSTA and DUNE,” Weber said. “Working for Fermilab, I look forward to addressing a different, but still wide, range of issues and focusing on a project that I’m genuinely passionate about.”
Weber has visited Sanford Lab, Fermilab and the CERN laboratory in Geneva, Switzerland, another key partner in LBNF/DUNE.
“The challenges of an international project of this scope are unprecedented in the United States, and I look forward to working with our global partners to support the DUNE experiment that will help us understand neutrinos and the nature of matter in the universe,” Weber said.
As head of Fermilab’s South Dakota Services Division, Weber will work to maintain a strong partnership with Sanford Lab while providing the scientists and engineers working in South Dakota on LBNF/DUNE with the same operational and support services they would enjoy at Fermilab’s main site in Illinois.
“Patrick’s experience in South Dakota government and education and his expertise in navigating challenging issues across a wide range of stakeholders will be a real asset to Fermilab and to LBNF/DUNE in the years ahead,” said Fermilab Deputy Director for LBNF Chris Mossey. “We are excited to have him joining the team, supporting Fermilab activities and the construction project in South Dakota, and strengthening our partnerships with SDSTA, SURF and the state of South Dakota.”
In addition to his work with SDSTA, Weber served as Governor Daugaard’s liaison to the South Dakota Department of Education, Corrections, Housing, and Labor and Regulation. Prior to his time in the governor’s office, Weber served for three years as a member of the South Dakota Board of Regents, which oversees all six of the state’s public universities, serving more than 36,000 students. Weber holds a law degree from the University of South Dakota School of Law and a bachelor’s degree in mathematics from South Dakota State University.