Caught on camera: Dark Energy Survey’s independent discovery from ROC West

These two photos show two moments in time surrounding the merging of two neutron stars. In the left image, taken about one day after the merger, the optical afterglow of the resulting explosion is visible as a small star at roughly the 11 o’clock position on the outskirts of the galaxy NGC 4993. In the right image, taken about two weeks later, the optical afterglow has completed faded away. Images: Dark Energy Survey

At this moment, it’s hard to imagine being one of the first people to see and photograph anything in our universe, but that’s what many members of the Dark Energy Survey (DES) strive to do. The recent observation of the neutron star collision and merger on Aug. 17 was one such rare, momentous event, and one of the places it was first observed was right here in Fermilab’s Remote Operations Center-West (ROC West) by Fermilab scientists Douglas Tucker and Sahar Allam.

The DES gravitational-wave follow-up team, led by Brandeis University scientist Marcelle Soares-Santos, formerly at Fermilab, had only a few hours to prepare for the event, which was only visible for approximately an hour-and-a-half the night of the collision. Researchers at the Laser Interferometer Gravitational-Wave Observatory (LIGO) had detected the gravitational waves signaling the event the morning of Aug. 17 and notified other astronomy groups, including Fermilab’s DES team. It was essential that the Fermilab team had everything in place for that critical 90 minutes. Each of the astronomy groups analyzed their photos, independently discovered the neutron star merger and confirmed the discovery within minutes of one another. Using photos from the Dark Energy Camera (DECam), DES was the second to independently discover the optical afterglow of the merger.

The distance from Fermilab to Chile, where the DECam is located, along with the unscheduled nature of the gravitational-wave follow-ups, made it essential to develop ROC West as a remote operations location for DES. Computing added the necessary tools to remotely access and control the DECam from Fermilab.

“Having ROC West as a remote DES station is a great accomplishment,” Allam said. “It has all the facilities and resources you need to connect to the work without struggling with laptops. Many smaller projects find it much more efficient to observe remotely.”

Setting up the DES resources in ROC West required computing experts with myriad specialties. The Core Computing Division’s audio/video teleconferencing team installed a Polycom videoconferencing system; the Scientific Linux and Architecture Management Group set up Linux workstations; network architect Gregory Stonehocker added the necessary networking; scientist Liz Buckley-Geer was instrumental in setting up the consoles; and many others within the Core Computing, Neutrino, Particle Physics and Scientific Computing divisions contributed as well. Without these remote capabilities, Fermilab would not have been able to reach DECam in Chile fast enough to view such unscheduled transient events like this neutron star merger. Instead, the DECam would have to be staffed continuously by the DES gravitational-wave follow-up team — an expensive proposition for only a few hours of observation.

Rather than worrying about logistics and staffing, the DES team used the time between the LIGO notification and the observation window to convert the broad sky area LIGO/Virgo reported into coordinates on the sky for the DECam to image in its search for the explosion. Capturing photos required more than a simple click of camera button. It was a feat of foresight, teamwork and experience. Preparation started months prior, in the spring of 2017. Fermilab scientist Jim Annis prepared algorithms well in advance. Without a good set of coordinates covering the full target area, DECam would be off, and, despite the camera’s large field of view, DES would miss the entire event. Annis also worked on the timing of the DECam observation to ensure the merger was observed at the ideal time based on the sun and weather conditions.

Once the sun set in Chile that fateful night, Tucker and Allam logged in to the remote console that allowed them to control DECam and start the observation software. The images were processed in parallel on FermiGrid and the Open Science Grid. The high-throughput processing engineered by scientific computing specialist Ken Herner ensured the large, high-resolution photos were quickly processed and ready for analysis so the DES team could quickly discover the neutron star merger.

“It was very exciting,” Tucker said. “We were honored to be among the first to see something like this happen. We are looking forward to analyzing the data and learning more.”

To learn more, attend the Computing Techniques Seminar on Jan. 16 at 1 p.m. in Wilson Hall One West.