In March, Fermilab Chief Information Officer Liz Sexton-Kennedy got in touch with Frank Würthwein, executive director of the Open Science Grid, to find out whether Fermilab could work with the Open Science Grid to contribute computing power to COVID-19 research.
It turned out that Würthwein was thinking along similar lines.
“Then he really ran with it,” Sexton-Kennedy said.
Shortly thereafter, members of the Open Science Grid — a network of organizations that provides computing services for science research of all stripes — were contributing tens of thousands of core-hours to COVID-19 projects. That number quickly grew to more than a million core-hours.
As part of the unified response, scientists and engineers at the Department of Energy’s Fermilab spurred themselves into high gear, preparing computing clusters — sets of connected computers — as COVID-19-research machines.
As of April 27, Fermilab has contributed a total of 1.8 million core hours to the pandemic-fighting effort carried out by projects such as Folding@home, which is simulating how viral proteins fold to help scientists design better therapeutics. Brookhaven National Laboratory, another DOE national lab and OSG member, has contributed 2.9 million core hours.
As a member of the Open Science Grid, Fermilab is dedicating a number of computing clusters for COVID-19 research. Photo: Reidar Hahn, Fermilab
“Understanding how the COVID-19 virus proteins fold — it’s all electrochemistry. Basically, it’s physics in the end that they’re simulating,” Sexton-Kennedy said. Like particle physics, “it’s computationally complicated science that takes lots of computing resources to figure out.”
Both Fermilab and Brookhaven’s high-throughput computing capabilities have for over a decade met the needs of experiments at the Large Hadron Collider, an enormous particle physics research facility in Europe, each analyzing data from billions of particle collisions.
During the times that select Fermilab and Brookhaven computers get a break from particle collision analysis, they’re free to crunch data outside particle physics. That’s where the Open Science Grid comes in. Among other tasks, the OSG evaluates research proposals to determine which are a good fit for its networks. In offering its resources to COVID-19 proposals, it provided the kind of vetting that the labs wouldn’t have been able to assume on their own. All the computational work is handled remotely.
Providing the world’s scientists with powerful computing capacity fuels research that would otherwise not be possible. Recognizing this, the U.S. Department of Energy supports Fermilab and Brookhaven science programs for the use and development of the OSG, and the National Science Foundation funds universities that partner with the OSG.
Brookhaven National Laboratory is contributing significant computing capacity to COVID-19 research through the Open Science Grid. Photo: Brookhaven National Laboratory
This effort is part of a larger COVID-19 High Performance Computing Consortium designed to provide access to the world’s most powerful high-performance computing resources in support of COVID-19 research. Consortium members include industrial partners, academic leaders, and agencies within the federal government, including the Department of Energy and many of its national laboratories. OSG prioritizes jobs to meet the needs of the consortium first.
“To maximize the chances that important COVID-19 research ideas reach us, we joined multiple national and international calls for computing requests: the COVID-19 High Performance Computing Consortium, a call with EGI for Advanced Computing Research in Europe, and the Worldwide LHC Computing Grid COVID-19 task force. We are also coordinating with all of the National Science Foundation software institutes,” said Würthwein, who is also a physics professor at the University of California, San Diego, and the lead for high-throughput computing at the university’s San Diego Supercomputer Center.
The computing power at Fermilab, Brookhaven and other OSG institutions can be used to model how virus proteins interact with receptors on cells in the human respiratory system. Simulations run on these systems can also sort through billions of potential drug molecules to narrow the search for drugs that might interfere with those interactions or disrupt the function of a virus protein in other ways.
Recent developments in machine learning accelerate this discovery process by improving the selection of leads. Modeling drug-protein interactions can be broken down into a larger number of independent calculations that can be farmed out to many computer cores. This characteristic makes the simulations well-suited to high-throughput capabilities at Fermilab, Brookhaven and the OSG infrastructure.
“This is a core competency that we can bring to the fight against this pandemic,” said Eric Lançon, director of Brookhaven’s Scientific Data and Computing Center and a member of the Open Science Grid executive team.
“I’m proud of our staff for being able to respond and think about what we can do usefully,” Sexton-Kennedy said. “We have had plans to create an institutional cluster for over a year, and the people involved got an extra bit of enthusiasm thinking that, ‘Hey, one of the first uses for this facility could be COVID research.'”
Learn more about how Open Science Grid is helping fight the COVID-19 pandemic.
This work is supported by the Department of Energy Office of Science and the National Science Foundation.
Fermi National Accelerator Laboratory and Brookhaven National Laboratory are supported by the Office of Science of the U.S. Department of Energy. The Office of Science is the single largest supporter of basic research in the physical sciences in the United States and is working to address some of the most pressing challenges of our time. For more information, visit science.energy.gov.