Fermilab-led computing project receives large 2025 DOE INCITE award

When tackling a complex project, the tools and equipment researchers use can greatly impact the results. That is the basis of the Department of Energy’s Office of Science Innovative and Novel Computational Impact on Theory and Experiment program, known as INCITE. This year the program awarded 81 computational science projects access to leadership-class supercomputers at DOE’s Argonne and Oak Ridge National Laboratories to accelerate discovery and innovation.

One of the largest recipients of an INCITE award this year is a project led by Andreas Kronfeld at Fermi National Accelerator Laboratory. The project, called Advances in Quark and Lepton Flavor Physics with Lattice QCD, was the largest particle physics project among the recipients this year. The award will allow the project’s researchers, who are working at several national labs and U.S. universities, to use the Aurora and Frontier supercomputers to continue the study of the behavior and interactions of subatomic particles against the Standard Model of Particle Physics.

“We are honored to have received the renewal of INCITE and the largest award in particle physics,” said Andreas Kronfeld, Fermilab researcher and the project’s principal investigator. “This will allow us to continue conducting large-scale numerical calculations on world-class supercomputers that will impact experiments at Fermilab, CERN, KEK and other accelerator laboratories.”

The Advances in Quark and Lepton Flavor Physics with Lattice QCD project uses numerical simulations based on the lattice gauge theory formulation of quantum to address fundamental questions in elementary particle physics: the hadronic vacuum polarization contribution to the anomalous magnetic moment of the muon, also known as muon g-2, semileptonic decays of the heavy bottom and charm quarks, and CP violation. Precise calculations from supercomputers are needed to compare theoretical calculations with experimental measurements.

“It is especially exciting that this year we will be able to finish a calculation of the muon g-2 after several years of key support from INCITE,” said Ruth Van de Water, project member and Fermilab scientist.  “Recent developments, including results from the Fermilab Muon g-2 experiment make this topic incredibly timely!”

Being awarded time to use supercomputers is valuable to complex projects because their enormous capability enables researchers to tackle complex systems with more precision and efficiency than with general-purpose computers.

The INCITE program is the primary means by which the facilities fulfill their mission to advance open science by providing the scientific community with access to their powerful supercomputing resources. It is jointly managed by the Argonne Leadership Computing Facility and the Oak Ridge Leadership Computing Facility.

At the Sanford Underground Research Facility in Lead, South Dakota, site of the former Homestake gold mine, technicians and engineers are employing a modern approach to moving massive steel beams for the Deep Underground Neutrino Experiment, hosted by the U.S. Department of Energy’s Fermi National Accelerator Laboratory.

The unique system they have constructed is an air cart — a hovercraft using air casters that each generate little more force than a push from a person’s hand to create a foundation of compressed air for floating heavy items above the ground.

“It seems like it shouldn’t work, but it does,” said Jeff Barthel, a rigging specialist at SURF. “I think in our final setting we ended up with a beam that weighs about 12,000 pounds. It’s amazing, that it only takes about 10 to 20 pounds per square inch of pressure on each caster, just a little bit of concentrated air pressure, to float this heavy object on air.”

With a nod to the history of the former Homestake Mine, the team named the air cart “Maggie II” in memory of a beloved workhorse that 120 years ago was renowned for her prowess in helping miners move large amounts of material.

“Maggie carried more gold ore during the days of Homestake than any other animal,” Barthel said. “This air cart is going to transport more than one thousand pieces of steel, so we thought we should call it Maggie II. It’s a fitting tribute to the past.” 

From dreams to design 

The air casters that support the hovercraft at SURF were engineered for the specific purpose of moving 1,124 steel beams that are needed to construct cryostats for the largest international physics experiment ever undertaken on American soil. The Long-Baseline Neutrino Facility/Deep Underground Neutrino Experiment (LBNF/DUNE) is currently being built in Illinois and South Dakota. As in many great undertakings of science and engineering, logistical and technical problems yield novel solutions. Air casters have been around since the 1960’s and engineers found they were the right fit for this job.  

“The design started as pretty basic, with just wheels and a cart,” said Sanmitra Pingulkar, a Fermilab mechanical engineer. “This didn’t work because the load was too heavy, and the ground is uneven. One person on our team had used air casters before, so he floated that idea as a solution.” 

The air casters proved effective not only for managing the heavy weight, but also for the awkward shape of the steel beams — many of which are shaped like a gigantic letter L.   

“The L-beam has three characteristics that make it tough to work with: it’s heavy, it’s very narrow, and it has a very high center of gravity,” said Barthel. “The air caster addresses all of those things.”

For instance, if crews were to place the L-beam directly in the center of the cart, only two of the air casters would sustain the majority of the load. This resulted in an unstable method of transporting the beam to the cage — an elevator for delivering material more than a mile underground.  

“The L-beam is difficult,” said Pingulkar. “Depending on where you place the beam, you get different weight distribution on the cart. Eventually, we found a way to distribute the weight that was a 30/70 split, which provided greater stability for the cage while transporting it underground.”  

A small role in a huge undertaking 

Maggie II is just one part of LBNF/DUNE. The entire project is a colossal scientific undertaking that includes a chain of particle accelerators and detectors at Fermilab that will project a beam of neutrinos 800 miles though the curvature of the Earth to a set of five-story-high particle detectors located a mile underground inside massive caverns at SURF.   

A team from Fermilab and SURF recently completed a successful test lift, which involved moving one of the six-ton, L-shaped steel beams needed for the underground construction of the DUNE detectors.  

Once the beams are lowered 4,850 feet underground, Maggie II will again be used to move the beams out of the elevator cage and into the caverns, where they will be assembled into a structure to support the massive cryogenic vessels for the Long Baseline Neutrino Facility. 

International partnership

A team of engineers and technicians from around the world who are involved in the construction of LBNF/DUNE worked together to build the hovercraft.  

“This didn’t come from one person,” said Pingulkar. “We had people from CERN, from SURF, from Fermilab — everyone contributed in some way to the design. It was very collaborative.”  

Similar air pressure technology is employed at CERN to move the massive CMS particle detector. During a recent planned shutdown of CMS, each of its 15 circular slices, nearly 50 feet tall, was moved back and forth using pressurized discs to allow technicians to reach inner components which are normally hidden.   

The DUNE collaboration extends far beyond the development of Maggie II. In total, DUNE comprises more than 1,400 scientists and engineers from over 200 institutions in 36 countries, plus CERN. The steel is an in-kind contribution from CERN and is the first time the European science organization has provided infrastructure for a project on American soil.   

In 2024, crews completed excavation for LBNF/DUNE’s massive caverns. Work is currently underway to outfit the caverns with conventional infrastructure, such as electricity, water, HVAC and fire suppression equipment. Beginning in late 2025 and early 2026, Maggie II will begin moving the massive steel beams underground for construction of the detector support structures.

Fermi National Accelerator Laboratory is America’s premier national laboratory for particle physics and accelerator research. Fermi Forward Discovery Group manages Fermilab for the U.S. Department of Energy Office of Science. Visit Fermilab’s website at www.fnal.gov and follow us on social media.

Sanford Underground Research Facility is operated by the South Dakota Science and Technology Authority (SDSTA) with funding from the Department of Energy’s Office of Science. We are America’s Underground Lab. Our mission is to advance world-class science and inspire learning across generations. Visit SURF at www.sanfordlab.org.

As the 2025 Fermi Forward Discovery Group Guest Composer, Amy Nam will use her skills in music to transform concepts behind the physics research conducted at Fermi National Accelerator Laboratory into auditory experiences the public may enjoy.

The Guest Composer program at Fermilab fosters collaboration between scientists and artists and is aimed at strengthening the laboratory’s ties to the community by creating new pathways to spark interest in science. The program is funded by Fermi Forward Discovery Group, which manages and operates Fermilab on behalf of the U.S. Department of Energy.

“Connecting music to science can expand the reach of Fermilab’s research to a broader audience,” said Natalie Johnson, head of the Fermilab Office of Education and Public Engagement. “Approaching scientific research through the lens of music offers fresh perspectives and conveys the ideas in a new and meaningful way. Our goal in connecting Fermilab science to a wider audience is to inspire and encourage more young people to pursue careers in STEM fields.”

With Nam’s proposed project, “The Harp Inside Schrödinger’s Box,” she plans to create an equal parts musical concert and educational experience by juxtaposing original harp music with spoken word passages inspired by quantum physics research. With the help of scientists and the Office of Education and Public Engagement, Nam will create an auditory and educational experience made available to the public at Fermilab’s Ramsey Auditorium.

“Translating science through art makes the science tangible to more people, that’s the hope,” said Nam. “Reaching new audiences, who may not otherwise seek out information about particles physics or quantum computing, that’s the goal.”

Nam wishes to compose music informed by Fermilab’s QUIET and LOUDresearch laboratories. In these twin labs, one located 100-meters underground and one at the Earth’s surface, researchers gather data to understand how cosmic radiation affects the performance of superconducting qubits. This research could contribute to optimizing future quantum computers.

“Amy’s clear vision of weaving scientific ideas into music and poetry is compelling,” said Georgia Schwender, the Visual Arts Coordinator at Fermilab who also manages the FermiForward Guest Composer program. “As America’s premier particle accelerator laboratory, Fermilab’s goal is to understand the fundamental particles that make up our universe and the forces that govern their behavior. To help people who may not have science backgrounds reach a new understanding of what we investigate and find, it’s critical to create ways to present this research through new perspectives.”

Fermi National Accelerator Laboratory is America’s premier national laboratory for particle physics and accelerator research. Fermi Forward Discovery Group manages Fermilab for the U.S. Department of Energy Office of Science. Visit Fermilab’s website at www.fnal.gov and follow us on social media.

Jelena Berenc is an artist known for using bold pen strokes and detailed pencil shading to explore the idea of quantum fluctuations in spacetime. This year, she will bring her vision to Fermi National Accelerator Laboratory as the 2025 Fermi Forward Discovery Group Guest Artist.

The Guest Artist program fosters collaboration between scientists and artists to help the public gain more understanding about cutting-edge research at Fermilab. The program is funded by Fermi Forward Discovery Group, which manages and operates Fermilab on behalf of the U.S. Department of Energy.

During her time at the laboratory, Berenc will team up with scientists to create art based on their research. The pieces she creates will be shown in art displays for public viewing at the laboratory.

“Connecting science to art helps spread awareness of Fermilab’s research to a wider community,” said Natalie Johnson, head of Fermilab’s Office of Education and Public Engagement. “Looking at physics research through different perspectives can make the science more accessible, with the ultimate goal of inspiring young people to pursue STEM careers.”

Berenc wants to learn more about virtual particles, theorized to be tiny fluctuations in the fields that make up all physical matter. She will collaborate with researchers at Fermilab who have studied these particles through the Muon g-2 experiment.

 “Scientific concepts expressed through a visual language help audiences understand the fundamental rules of the universe,” said Berenc.

Berenc brings with her years of experience collaborating with STEM researchers and institutions. Berenc recently curated an art show in Chicago that featured installations from both artists and experts in fields that included mathematics, physics, astrophysics and biology.  

Berenc’s art curation expertise makes her uniquely equipped to organize an art exposition that showcases the art created in collaboration with Fermilab researchers.

“I was captivated by the duality in Berenc’s work,” said Georgia Schwender, the visual arts coordinator at Fermilab and manager of the guest artist program. “She takes concepts and explores their complexity with precision. Her thought process, visual language and creative journey are deeply compelling.”  

Berenc’s self-described art method, information realism, guides her creative process.

“Using this process, I don’t allow my feelings or beliefs to inform the art,” said Berenc. “Instead, I use the information in front of me.”  

An example of her method is her piece “Book of Knowledge,” a 1,000-page book filled with 500,000 tiny marks, she calls bits. The bits make up only 4% of the pages, while the other 96% is blank, symbolizing how much scientists may still not understand about the fundamental structure of the universe. 

“As America’s premier particle accelerator laboratory, our goal is to understand the fundamental particles that make up our universe and the forces that govern their behavior,” said Schwender. “To make our discoveries more accessible, it’s important to present the material in ways that allow everyone to appreciate the significance of the work.”

Fermi National Accelerator Laboratory is America’s premier national laboratory for particle physics and accelerator research. Fermi Forward Discovery Group manages Fermilab for the U.S. Department of Energy Office of Science. Visit Fermilab’s website at www.fnal.gov and follow us on social media. 

More than 100 participants gathered at Fermi National Accelerator Laboratory for the 2025 CMS Data Analysis School — an immersive, hands-on program to train the next generation of physicists for the CMS experiment. The event was hosted earlier this year by the LHC Physics Center, or LPC, at Fermilab, and brought new members of the CMS collaboration together for an intense week of lectures, hands-on exercises and a group competition.

The CMS experiment is one of the major particle detectors at the Large Hadron Collider at CERN, and the school featured a rigorous curriculum aimed at familiarizing participants with the physics, detector technology, software and data analysis of the experiment. Sixty-four students, primarily new graduate students and postdoctoral researchers, along with 46 facilitators and nine lecturers participated in this year’s data analysis school. The facilitators included volunteers from the CMS experiment and this year’s LPC Distinguished Researchers, who provided invaluable mentorship throughout the week.

The program combined lectures and practical exercises to offer a deep dive into particle physics analysis. Twelve short exercises focused on specific aspects of the CMS experiment, such as particle identification and reconstruction. In addition, participants engaged in seven longer exercises, each tackling complex analyses like top quark measurements and searches for new long-lived particles. The exercises were designed to equip attendees with the essential skills needed to work with large data sets.

As part of the program’s concluding activities, seven student groups presented their analyses to a panel of senior CMS collaborators. These presentations were judged based on the quality of the analysis, teamwork and presentation skills. Each group’s performance was assessed by a distinguished panel of senior CMS researchers.

Guest speakers from across the physics community offered lectures and insights throughout the week. Among the presenters were interim Fermilab Director Young-Kee Kim, Fermilab Deputy Director for Science and Technology Bonnie Fleming, CMS collaboration Spokesperson Gautier Hamel de Monchenault and additional experts in particle physics, including Tulika Bose, Eliana Gianfelice-Wendt, Lindsey Gray, Frank Hartmann, Gordan Krnjaic, Corrinne Mills and Isobel Ojalvo.

Corrinne Mills, a long-time collaborator with CMS, shared her thoughts on the significance of the event. “The data analysis school is an excellent opportunity not only to get up to speed quickly on CMS analysis, but also to meet other students and collaborators on the experiment,” Mills said. “I actually went through the data analysis school as a student when I first came to CMS as a new assistant professor at the University of Illinois at Chicago, and it was a great experience.”

The school also covered key topics such as the CMS publication process, how to present results to a scientifically literate audience, and how to handle tough questions about the details. During one of the school’s lunches, members of the Fermilab Accelerator Division joined students and facilitators to answer questions about the laboratory and its research.

The 2025 CMS Data Analysis School was organized by outgoing LPC co-coordinators Kevin Black and Bo Jayatilaka, along with current LPC co-coordinator Isobel Ojalvo and incoming LPC coordinator Jim Hirschauer. The school’s success was made possible by the dedicated efforts of CMS LPC support staff: Gabriele Benelli of Brown University, Marguerite Tonjes of the University of Illinois at Chicago, and David Yu of the University of Nebraska.

Fermi National Accelerator Laboratory is America’s premier national laboratory for particle physics and accelerator research. Fermi Forward Discovery Group manages Fermilab for the U.S. Department of Energy Office of Science. Visit Fermilab’s website at www.fnal.gov and follow us on social media.

Senior scientist Sowjanya Gollapinni has been elected as the new co-spokesperson for the international neutrino project, the Deep Underground Neutrino Experiment, hosted by the U.S. Department of Energy’s Fermi National Accelerator Laboratory. Gollapinni, currently a scientist at the Department of Energy’s Los Alamos National Laboratory, will succeed Mary Bishai, an experimental particle physicist at Brookhaven National Laboratory, who co-led the collaboration since 2023. Gollapinni’s two-year term began April 1.  

“Mary has done a wonderful job leading the collaboration over the last two years and she will continue to contribute her uncommon skills and drive to the success of DUNE Phase-I and to the realization of DUNE Phase II,” said Sergio Bertolucci, co-spokesperson of DUNE and professor of physics at the University of Bologna in Italy. “I look forward to working with Sowjanya to establish DUNE as the best-in-class next generation neutrino experiment.”

DUNE is an international flagship experiment to unlock the mysteries of neutrinos. Fermilab is the host laboratory for DUNE, in partnership with funding agencies and more than 1,400 scientists and engineers from all over the globe. 

With a focus on neutrino research, Gollapinni has been involved with the DUNE collaboration since 2015, in several scientific and technical leadership roles. Before moving to Los Alamos, Gollapinni was a faculty at the University of Tennessee, Knoxville’s physics and astronomy department. 

“It’s a great privilege to have been elected as co-spokesperson, and I’m honored at the trust the collaboration has placed in me,” Gollapinni said. “The next two years will be a pivotal time for DUNE, and I look forward to taking DUNE to the next stages of development as the collaboration gets closer to making this project a reality.” 

She added “It will be a pleasure to work with Sergio. He has been critically involved in the experiment for a long time and has made foundational contributions to what we are now calling DUNE.”

Referencing the 2023 Particle Physics Project Prioritization Panel report, Gollapinni added, “The P5 report could not have supported Phase-I and Phase-II of DUNE any stronger. My focus over the next two years will be successful execution of Phase-I and building a roadmap for Phase-II to realize the full scope of DUNE.” 

The infrastructure that will house DUNE is currently being built at two sites: Fermilab’s campus in Batavia, Illinois and a mile below the surface at the Sanford Underground Research Facility in Lead, South Dakota. 

Last year, LBNF/DUNE achieved a major milestone with the completion of excavation for the colossal caverns that will house the experiment’s far detectors. Since that time, workers have begun installing conventional infrastructure, such as lighting, electricity and fire suppression equipment in the space one mile underground in Lead, South Dakota. 

DUNE is a science experiment, but it is also a collaboration that is made up of people. We need to ensure the collaborators are successful in their science and research, but also in their careers especially early career members.

On the collaborative nature of the experiment, Gollapinni said, “DUNE is very international. I value international partnerships and the commitments they bring to the table greatly. DUNE will not be possible without them.To me, DUNE would not have come this far if it were not for the incredible hard work and intellectual contributions of the people involved. So, the welfare of the collaboration is of paramount importance to me.”