Computing Techniques Seminar Sept. 15: dCache, approaching the Exascale Cloud era

Friday, Sept. 15, 1 p.m.
Wilson Hall One West
Speaker: Dr. Patrick Fuhrmann, DESY

For over a decade, dCache has been Fermilab’s workhorse for storing, accessing and managing massive amounts of data. It serves a variety of scientific communities by providing the necessary standard or domain-specific data access and authentication mechanisms. Due to its ability to understand the capabilities of different media types, like SSD, regular spinning disks, shingled disks (SMR) or custodial systems, it offers an optimal storage cost for a wide range of use cases from scratch storage to long-term archiving. Moreover, policy-based internal media transfers, including the creation of multiple replicas of data files within or outside of dCache, protects the system from data losses, reducing the cost of operations. The seamless integration between dCache and the Fermilab custodial storage product, Enstore, offers significant advantages over other data management products. Recently, high availability has been added to dCache, allowing the system to run 24/7, surviving single component failure with little to no degradation of overall system performance.

As dCache has become popular, not only for mass storage applications, but also for everyday cloud operations, it will allow modifying existing files in particular areas of the dCache namespace while keeping the immutable option for mass data in the same dCache instance.

Fermilab is benefiting greatly from being a founding partner in the international dCache collaboration by having full control over its technological direction and the ability to adjust the dCache software according to its particular needs within short development cycles. Moreover, since dCache is an international open source project, Fermilab takes advantage of code contributions from DESY, the Nordic Data Grid Facility (NDGF), the University of Applied Science in Berlin and other highly respected members of the demanding scientific Big Data community. In this context, dCache became part of development stack of initiatives within the European Open Science Cloud.

This presentation will report on the status of the dCache technology as well as the dCache.org collaboration, its funding and its short and midterm plans.