ESnet and NERSC Take Scientific Data Collaboration to the Next Level

April 12, 2021 By Elizabeth Ball


When the Dark Energy Science Collaboration (DESC), one of the projects associated with the upcoming Legacy Survey of Space and Time (LSST), needed a fast, secure, and convenient way to share large data sets with scientists around the world, they turned to the National Energy Research Scientific Computing Center (NERSC) for help. Having used NERSC resources to generate simulations and store data, it was time for DESC to enable sharing that data outside the collaboration. Working with NERSC staff they found a solution — and a leap forward for anyone sharing scientific data: the Modern Research Data Portal (MRDP), a high-performance networking resource developed by engineers at the U.S. Department of Energy’s Energy Sciences Network (ESnet) in collaboration with computer scientists at Argonne National Laboratory and the University of Chicago.

The ten-year LSST, conducted by the Vera C. Rubin Observatory under construction in northern Chile, will soon be producing vast amounts of imaging data. By conducting the largest-ever astronomical survey using the world’s largest digital camera, the science collaborations working with Rubin Observatory hope to answer foundational questions about the universe.

Observatory systems will need to be ready to receive and analyze massive amounts of data when the facility is complete; in preparation, DESC is currently simulating portions of those data sets, producing realistic faux data to mimic the workings of the finished observatory.

“LSST will observe about half the sky for ten years. We picked a 300-square-degree patch within that survey area and simulated a five-year survey,” said DESC Computing Coordinator Katrin Heitmann. “After we simulated the data, we used the Rubin Science Pipelines to process it. It was quite an elaborate process, but we wanted to test the processing pipelines in detail and generate an LSST-like data set. Now the science collaboration has started running their analysis pipelines on the simulated data and trying to get cosmology out.”

With the release of these data sets, scientists around the world, both inside and outside LSST DESC can view and download them. “Since this is a rather rich data set,” said Heitmann, “we thought it would be nice to share it [with scientists] beyond the collaboration.” That’s where the MRDP comes in.

Read More:

Leave a Reply


* indicates required