3 minute read
Contact - issue 18
French supercomputing lab enables public-private partnerships in exascale data processing
BY DR CHIARA FERRARI (SKA FRANCE, OCA, CNRS), DR DAMIEN GRATADOUR (SKA FRANCE, OP-PSL, CNRS) AND JACQUES TISSOT (NEOVIA INNOVATION)
In France, the coordination of SKAO-driven supercomputing activities is stimulating the development of new partnerships between research institutions and private enterprise. These collaborations are helping to address challenges around big data processing which can find applications far beyond radio astronomy.
The ECLAT laboratory (Extreme Computing Lab for Astronomical Telescopes), coordinated by the French national centre for scientific research CNRS, brings together the experts from Inria, Paris Observatory and Côte d’Azur Observatory, as well as the manufacturer Bull SAS, operating under the Eviden brand and part of the Atos group, to work on research and development activities in preparation for the data processing challenge posed by the SKA telescopes.
The challenge is immense. The SKA telescopes will operate continuously, producing so much data that it is impossible to store all of it. It will therefore have to be processed largely in real time to transform celestial radio signals into science-ready data products that can be analysed by astronomers. The data captured by the antennas will far exceed Netflix-scale bandwidth, with rates comparable to 100 million 4K streams at this stage of the data journey. This will be reduced “on the fly” down to only a few thousand streams’ worth, ultimately archiving ~700 PB a year.
To meet these unprecedented needs, ECLAT is working with national and international collaborators to develop computing solutions that are both ultra-highperformance and energy efficient, a major asset for these isolated installations and for the decarbonisation of digital technology.
To design solutions tailored to the specific needs of astronomers in the age of data deluge embodied by the SKA project, SKA-France – which coordinates French contributions to the SKA project – established the ECLAT joint laboratory to provide an innovative platform where industry can collaborate with scientists over the long term.
ECLAT researchers have worked closely with a state-ofthe-art data processing tool for radio astronomy, the DDFacet algorithm, already used successfully on LOFAR, one of the SKA pathfinders. DDFacet has enabled the publication of more than 250 scientific articles and the completion of the largest radio survey of the sky, LoTSS. As part of ECLAT, Eviden teams contributed to the parallelisation and implementation of a distributed version of DDFacet, meaning they adapted the algorithm so that it can simultaneously exploit tens of thousands or more computing cores distributed across a supercomputer. This adaptation paves the way for even faster and more efficient processing on a larger scale for radio astronomy imaging algorithms.
The impact goes far beyond fundamental research. This close collaboration between scientists and industry enables France to strengthen its expertise in exascale computing architectures and develop unique energy optimisation skills. Innovations resulting from ECLAT can be reused in strategic fields such as seismology, volcanology, climate modelling and designing low-carbon data centres for industrial applications.
By investing in the SKAO via ECLAT, France is not only facilitating major astronomical discoveries; it is also fostering a sustainable technological and economic advantage, positioning its researchers and industrialists at the forefront of global efforts towards carbon-free highperformance computing and AI for scientific discoveries.
