High-Performance Computing and Communications Organizations Pool Capabilities to Support Vast Bandwidth Needs for Particle Physics and Other Applications
RENO, Nev. – A group of research and industry technology leaders today announced two demonstrations at the SC07 Conference in Reno, Nev. to show leading-edge capabilities designed for the high-bandwidth needs of the research community worldwide. The demonstrations involve the transport of large volumes of data at rates significantly in excess of 10 gigabits-per-second (Gbps) by infrastructure built to support the Large Hadron Collider (LHC), over a 40 Gbps network and the use of Generalized Multi Protocol Label Switching (GMPLS) User to Network Interface (UNI) signaling between routers and optical systems to provision bandwidth on demand.
“The cooperation between these leading technology organizations is a model of collaboration that propels innovation. Our members from industry and the research community are working together to push the technology boundaries so scientific researchers can do things that would have been unimaginable ten years ago,” said Rick Summerhill, chief technology officer, Internet2. “These demonstrations are exemplary of the benefits resulting from sharing knowledge and resources to usher in the next wave of technological advancements.”
The first demonstration involves high-volume data transmission between Fermi National Accelerator Laboratory’s (Fermilab) LHC Tier 1 mass storage system and the SC07 show floor over a wide-area 40 Gbps network infrastructure, the highest-speed networking service available today. The LHC Tier 1 will transfer data at speeds significantly in excess of 10 Gbps to the LHC Tier 2 computational infrastructure in the Caltech booth on the SC07 show floor.
The second demonstration involves the use of Generalized Multi Protocol Label Switching (GMPLS) User-to-Network Interface (UNI) signaling between routers and optical systems to enable the optical and IP layers in the network to work together for maximum intelligence, providing flexibility to network users.
Over the course of the two-day SC07 exhibition session, each participating organization – Caltech, ESnet, Fermilab, Infinera, Internet2, Juniper Networks, and Level 3 Communications – is leveraging its unique networking, computation, and storage capabilities to support the demonstration.
Demonstration participants include:
- California Institute of Technology, a leading institution of higher education and home to the first LHC Tier-2 center at Caltech’s Center for Advanced Computing Research, is using its latest computing cluster to receive and store multi-Terabyte volumes of data. The Caltech high energy physics and network engineering teams, funded by the U.S. Department of Energy and the National Science Foundation in the U.S. LHCNet and UltraLight projects, are leading developers of high speed data transfers over long distances and managing transatlantic networking in support of the LHC physics program in conjunction with CERN and ESnet. The Caltech team will deploy an open source Java application to allow stable reading and writing of data with Transmission Control Protocol (TCP) at disk read/write speeds over long distance networks.
- ESnet, the Department of Energy’s high-speed network providing interoperable communications infrastructure to the U.S. national laboratories and enabling the work thousands of scientists and collaborators worldwide, is providing domestic connectivity for the demonstration through its Chicago Area MAN, built in partnership with IWIRE (Illinois Wired Infrastructure for Research and Education). Its new network, ESnet4, is built specifically to support the requirements of large-scale scientific research while improving the reliability of IP connectivity.
- Fermilab, a national laboratory funded by the Office of Science of the U.S. Department of Energy and operated by the Fermi Research Alliance, LLC, is the prime host site in the U.S. for data gathered by the CMS experiment at the LHC in Geneva. The lab is simulating LHC CMS Tier-1 to Tier-2 data movement by sending multiple 10 Gbps streams of data to the Chicago POP in the Internet2 network to demonstrate the high bandwidth requirements and capabilities of the facility.
- Infinera, a leading provider of digital optical communications systems, is deploying its optical systems to carry the data across the nationwide Internet2 network. Infinera’s Digital Optical Networking systems use bandwidth virtualization to enable 40 Gbps services to travel over existing optical infrastructures, including those designed for 10 Gbps services. With integrated bandwidth management and GMPLS-powered service intelligence, Infinera systems are able to provide a wide range of GMPLS features and interoperability with Juniper routers.
- Internet2, focused largely on the investigation and implementation of leading edge network technology including 40 Gbps to support its members’ future bandwidth needs, is providing the flexible, scalable network to enable the large scale data transfer. By implementing the 40 Gbps solution on its backbone for the demonstration, Internet2’s network will act as a test bed to drive the development of new bandwidth services and improve general knowledge surrounding emerging technologies.
- Juniper Networks, the leader in high performance networking, is providing the Juniper Networks T640 and new T1600 multi-terabit routers to handle the Fermi Lab LHC traffic on either side of the optical transport network. The traffic will initially pass to the Infinera optical transport gear in Chicago before arriving at SC07. The data transfer will use 40 Gbps-capable OC-768 interfaces designed to address the increased bandwidth and collaboration demands of the research and multimedia communities.
- Level 3 Communications, an international communications company operating one of the largest Internet backbones in the world, is offering supporting services for the high bandwidth demonstrations. As an active member of the research and education community, Level 3 provides connectivity solutions that are highly focused on enabling innovation and collaboration across the community.
For additional information about participating demonstration organizations, please visit:
http://www.caltech.edu/
http://www.es.net/
http://www.fnal.gov/
http://www.infinera.com
http://www.internet2.edu/
http://www.juniper.net/
http://www.level3.com/
Recommended reading
A minute with Sajid Ali Syed, computational science and AI research associate
Always looking for a new trail to ride his bike on or an opportunity to catch a live show in town, Sajid Ali Syed is a research associate in the Computational Science and AI Directorate for high-energy physics applications at Fermilab.
DOE invests $13.7 million for research in data reduction for science
The U.S. Department of Energy recently announced $13.7 million in funding for nine research projects that will advance the state of the art in computer science and applied mathematics. One of the recipients of this funding, Fermilab scientist Nhan Tran will lead a project to explore methods for programming custom hardware accelerators for streaming compression.
Coffea speeds up particle physics data analysis
The prodigious amount of data produced at the Large Hadron Collider presents a major challenge for data analysis. Coffea, a Python package developed by Fermilab researchers, speeds up computation and helps scientists work more efficiently. Around a dozen international LHC research groups now use Coffea, which draws on big data techniques used outside physics.