Overview
The European Incoherent Scatter Scientific Association (EISCAT) operates three incoherent scatter radars, instruments probing the Earth’s ionosphere. Two are located in northern Fennoscandia and one on Svalbard. EISCAT are now developing the next generation radar, EISCAT_3D, consisting of antenna arrays in northern Norway, Sweden and Finland. The purpose of this data pilot is to use EUDAT services to establish a unified archival and data search system for the existing EISCAT incoherent scatter radars. The outcome will be used to explore whether and how EUDAT services can be customised for data archival and discovery for the future EISCAT_3D radar system.
The Scientific Challenge
Accessible EISCAT data are divided into levels. EISCAT_3D data will be similar but data volumes will be considerably larger due to the volumetric data at high bandwidths. The data rates and volumes are expected to be in the order of magnitude as other large scientific experiments such as the LHC. The present archives of EISCAT data at level 2 and 3 are completely separate and use different systems for access. The proposed pilot is intended to unify the access to data at these two levels. The project can be divided into the following tasks: (1) Archive, index and stage data. (2) Data discovery and search. Data at levels 2 and 3 will have to be connected to each other. Several versions of level 3 data corresponding to the same level 2 data may exist, depending on analysis methods and parameters of the analysis algorithms. (3) Data visualization. E.g., browse level 3 data visually for occurrence of aurora, and download data from these events. EISCAT_3D will also require volume rendering of level 3 data in four dimensions (time development of parameters in a volume) or seven dimensions (time development of a velocity vector field). (4) Access control and user authentication. Different access rules apply to EISCAT data at different levels.
There is currently no fine-grained access control in the EISCAT data access systems. An authentication system must be implemented in order to grant access to EISCAT users following the data policy, regardless of their geographical location at the moment of data download.
Who benefits and how?
With the development of a functional archive for the EISCAT data, the EUDAT pilot will make a foundation for new discoveries and significant scientific breakthroughs. The system will be robust and allow refinements and further developments of the access of data. Important is also the training of the users, with valuable feedback, making the updated system ready for wider use. The system is also expected to lay a foundation for the development of a data archive for EISCAT_3D.
The design of the next generation incoherent scatter radar system, EISCAT_3D, opens up opportunities for physicists to explore many new research fields. On the other hand, it also introduces significant challenges in handling large-scale experimental data which will be massively generated at great speeds and volumes. This challenge is typically referred to as a big data problem and requires solutions from beyond the capabilities of conventional database technologies. The overall ambition is to provide the users of incoherent scatter radar with tools that improves opportunities for scientific discovery. This competence centre is also important for the build-up towards EISCAT_3D and the tools developed will form a base for further development.
Technical Implementation
EISCAT operates three incoherent scatter radars, instruments probing the Earth's ionosphere, at high latitudes in northern Europe. Two of the systems are located in northern Fennoscandia and one on Svalbard. Currently EISCAT are developing the next generation radar, EISCAT_3D, which will consist of antenna arrays at several sites in northern Norway, Sweden and Finland. EISCAT_3D will provide volumetric measurements of several physical parameters, improving our understanding of how the upper atmosphere connects to space and the Sun.
A system controlled by the EISCAT community sends a level 2 dataset to B2SAFE. The dataset is registered and replicated according to the data management policy of the EISCAT community. B2SAFE extracts and associates technical metadata from the level 2 dataset with the dataset making it harvestable by B2FIND. In B2FIND, a researcher queries datasets by giving a set of conditions. B2FIND returns a list of datasets fulfilling the search conditions. The researcher assesses the list and selects a number of datasets for further analysis. The researcher assesses the datasets by reviewing the dataset description. The entitled researcher can reuse the dataset. The researcher stages data from B2SAFE to a computing cluster through the use of PID shown by B2FIND. After the datasets has been processed and analysed, the researcher sends a resulting level 3 dataset to B2SHARE. Alternatively, the resulting level 3 dataset can be sent to B2SHARE by an automatic process through the B2SHARE API.