Impact - Data Science

French supercomputing lab enables public-private partnerships in exascale data processing

BY DR CHIARA FERRARI (SKA FRANCE, OCA, CNRS), DR DAMIEN GRATADOUR (SKA FRANCE, OP-PSL, CNRS) AND JACQUES TISSOT (NEOVIA INNOVATION)

In France, the coordination of SKAO-driven supercomputing activities is stimulating the development of new partnerships between research institutions and private enterprise. These collaborations are helping to address challenges around big data processing which can find applications far beyond radio astronomy.

The ECLAT laboratory (Extreme Computing Lab for Astronomical Telescopes), coordinated by the French national centre for scientific research CNRS, brings together the experts from Inria, Paris Observatory and Côte d’Azur Observatory, as well as the manufacturer Bull SAS, operating under the Eviden brand and part of the Atos group, to work on research and development activities in preparation for the data processing challenge posed by the SKA telescopes.

The challenge is immense. The SKA telescopes will operate continuously, producing so much data that it is impossible to store all of it. It will therefore have to be processed largely in real time to transform celestial radio signals into science-ready data products that can be analysed by astronomers. The data captured by the antennas will far exceed Netflix-scale bandwidth, with rates comparable to 100 million 4K streams at this stage of the data journey. This will be reduced “on the fly” down to only a few thousand streams’ worth, ultimately archiving ~700 PB a year.

To meet these unprecedented needs, ECLAT is working with national and international collaborators to develop computing solutions that are both ultra-highperformance and energy efficient, a major asset for these isolated installations and for the decarbonisation of digital technology.

To design solutions tailored to the specific needs of astronomers in the age of data deluge embodied by the SKA project, SKA-France – which coordinates French contributions to the SKA project – established the ECLAT joint laboratory to provide an innovative platform where industry can collaborate with scientists over the long term.

Credit: EVIDEN