Scientists and astronomers from Switzerland and South Africa recently discussed Astronomy Big Data were at a bilateral workshop which unpacked how challenging astronomy problems involving Big Data could be solved using Machine Learning (ML) and Artificial Intelligence (AI).

Big Bata refers to the study and applications of data-sets that are so big and complex that traditional data-processing software are rendered ineffective. The key challenge surrounding Big Data is to turn it into information and knowledge that has commercial value, brings scientific and technological advances or has a broader impact on society.

The workshop was a part of the Swiss/South African Joint Research Programme (SSAJRP), a collaborative platform in astronomy to address the scientific Big Data challenge faced by pathfinder telescopes to the Square Kilometre Array (SKA) telescope and the Large Synoptic Survey Telescope (LSST) such as UKZN’s HIRAX (Hydrogen Intensity Mapping and Real-time Analysis Experiment), MeerKAT (Meer Karoo Array Telescope), the BINGO (BAO from Integrated Neutral Gas Observations) telescope, and the DES (Dark Energy Survey) telescope.

The principal investigators for the projects in the SSAJRP- Professor Kavilan Moodley of UKZN’s Astrophysics and Cosmology Research and Dr Martin Kunz of the University of Geneva (UNIGE)- have extensive expertise in the required methods and techniques and are well placed to make significant contributions to the project. The Swiss-South African collaboration entails scientists developing methodologies and tools to build a framework to solve the challenges they face. Developing this framework and associated methodologies will result in cutting-edge discoveries in fundamental science that will benefit other fields of science using Big Data such as geospatial sciences and climate science as well as a variety of industries.

The workshop comprised two days of information exchange and discussions about the plan for the projects and how they fit into the overall collaborative project. Some of the topics covered by the speakers featured 21cm cosmology, next-generation modelling for connecting HI (atomic hydrogen) to dark matter, data challenges relating to the HIRAX telescope, quantum machine learning and data processing in cosmology using the examples of Planck and Euclid.

The proposed workflow that was discussed incorporates a number of different aspects of the Big Data challenge, including a data acquisition layer which is proposed to validate the measured data through calibration and high-end instrument model calibration. The data measurement process also involves pre-processing and sampling of the large pathfinder datasets. These techniques will ensure a high fidelity of data and a model to test for systematic effects in the data measurement process. The data analysis layer involves theoretical modelling that involves mathematical and computational techniques. Machine learning techniques will extract significant value from the large pathfinder datasets and offer the promise of new scientific discoveries.

‘This bilateral project between South Africa and Switzerland represents a great opportunity for both countries to profit from each other in terms of experience, knowledge and access to astronomical data,’ said Kunz. ‘I should add that every visit to South Africa, like on the occasion of this workshop, is a pleasure thanks to the warm and kind hospitality that is unfailingly extended to us.’

“The workshop was extremely productive and resulted in knowledge sharing and collaborative plans to guide various projects. The Swiss-South African collaboration will create high-impact outcomes that will address astronomy big data challenges” added Moodley.