From left: Donatella Torretta, William Badgett and Angela Fava fine tune the White Rabbit synchronization system for the Fermilab Short-Baseline Neutrino Program. Photo: Reidar Hahn
Being on time is important – just ask Lewis Carroll’s leporine friend – and one group who knows this more than most are particle physicists, whose work revolves around keeping track of near-light speed blips of matter.
As particle accelerators and experiments have become increasingly complex and choreographed over the decades, technology behind the scenes has had to innovate to keep up. One such example is White Rabbit, a clever timing and data transfer system that is playing a key role in modern particle physics.
“We are always pushing our experiments to higher and higher precisions,” said Angela Fava, scientist on Fermilab’s ICARUS neutrino detector and part of the team exploring White Rabbit at Fermilab. “White Rabbit is really useful because it can reach time precisions down to less than a billionth of a second.”
Keeping time
In modern particle accelerators, many separate components have to be activated in sequence in a timely manner to identify and track particles passing by at the speed of light. This requires very precise synchronization and timing systems to determine when these events should occur – an egg timer won’t cut it here.
Until recently, this timing has usually been achieved with devices that are hard-wired into experimental equipment, such as the General Machine Timing (GMT) system at CERN. But GMT has limitations, including a low data bandwidth, the capacity to only send signals one way through the network, and an inability to self-calibrate — to internally calculate how long a signal has taken to travel — which results in timing errors.
As experiments grow in complexity and require nanosecond coordination, physicists have been left with a need for a one-size-fits-all system that can provide the required time synchronization and still be compatible with systems from multiple sources and vendors that are already in place.
The solution is White Rabbit, an open-source system that builds on common and accessible Ethernet technology – the same technology behind wired internet access. The system works kind of like an everyday computer network, too, with circuit boards called “nodes,” controlled by a specially written program.
Up to around 1,000 nodes can be linked in one White Rabbit network, all connected together with a web of optical fibers – up to 10 kilometers long – to exchange information. As the technology develops, the system will likely be able to support even more nodes separated by greater distances.
Since precise timing is so important in modern experiments, White Rabbit’s power comes in its ability to keep itself synchronized, no matter the cable length between nodes or other external factors. Even relatively small changes in cable temperature can affect travel time on the scale of nanoseconds, for example.
A White Rabbit system works kind of like a hierarchy, where one of the nodes in a network is designated a “master” and is responsible for keeping all the other nodes in check. The external time is fed into the master from high-precision atomic oscillators via orbiting GPS satellites, the same technology on which Google Maps navigation is based.
This exact time is digitally attached to blips of data – which, for example, include control instructions for accelerators – that constantly fly around the network. By sending the time tags back and forth between nodes, which GMT isn’t able to do, the system can calculate the time delays it takes for data to travel through cables and correct for them, keeping all the nodes in synchronization with the correct time and ensuring experimental events are kept coordinated.
Fava and scientist Donatella Torretta, together with William Badgett at Fermilab, are currently working on installing White Rabbit into some of Fermilab’s experiments, including the Short-Baseline Neutrino (SBN) Program, which will study neutrinos – tiny, elusive particles. The first use of White Rabbit in North America, the system can be used to time-tag the neutrinos from their production at the beam source through to the detector at the end of the experiment.
On the SBN ICARUS detector, White Rabbit can also be used to get an extremely accurate tagging of unwanted cosmic particles that come from space and get in the way of the experiment, potentially hiding the neutrino signatures.
“It would be possible to run ICARUS without White Rabbit, but it’s lot easier if we use it,” said Fava. “And it’s all in real-time too, so it saves on our computing power and storage.”
Pulses on the oscilloscope indicate when the particles arrive in the detector. Photo: Reidar Hahn
Open science
White Rabbit was first conceived in around 2008 as an international collaboration between CERN, the GSI Helmholtz Centre for Heavy Ion Research in Germany, and other partners, and was introduced to boost the abilities of the Large Hadron Collider.
From the beginning, the collaboration has made both the hardware and software for the timing system openly available to anyone around the world. The physical equipment can be purchased from commercial vendors, while the software is completely free and easily accessible online.
“Everybody benefits when science is open,” said Torretta, who learned about White Rabbit at a demonstration workshop at CERN. “As the technology develops, it’s becoming more and more popular.”
Torretta has since attended further workshops to learn more, including one recently in Barcelona, which was organized by White Rabbit experts from CERN.
The CERN development team also took care to ensure the design was as general as possible, so as to allow a large range of practical applications for the technology, including outside of science. A group in the Netherlands has even used White Rabbit to transmit official time between Dutch cities with nanosecond accuracies.