Competence centre EISCAT 3D

EGI-Engage Competence centres:

Main page

ELIXIR

BBMRI

MoBrain

DARIAH

LifeWatch

EISCAT_3D

EPOS

Disaster Mitigation |

EGI-Engage Knowledge Commons

CC Coordinator: Ingemar Häggström

CC members' list: cc-eiscat3d AT mailman.egi.eu

Type of Competence Centre: Science-oriented

Target user communities: EISCAT_3D users

List of organizations representing the user communities: EISCAT Scientific Association

Duration of the CC: 30 PMs

Starting at Project Month: 1 PM

Ending at Project Month: 30 PM

The design of the next generation incoherent scatter radar system, EISCAT_3D introduces significant challenges in handling large-scale experimental data which will be massively generated at great speeds and volumes. The CC will build an e-Infrastructure to meet the requirements of the EISCAT_3D data system, will support the EISCAT science community in their acquisition, curation, access to and processing of the data, and will train data scientists who can explore new approaches to solve problems via new data-centric way of conceptualising, organising and carrying out research activities.

Objectives

1. To build common e-Infrastructure to meet the requirements of a big scientific data system such as EISCAT_3D data system
2. To demonstrate the developed e-Infrastructure can support the EISCAT science community in their acquisition, curation, access to and processing of the data
3. To train data scientists who can explore new approaches to solve problems via new data-centric way of conceptualising, organising and carrying out research activities, which will lead to new discoveries and significant scientific breakthroughs.

Tasks

Task 1: User Support and Training

(M15-M30) Effort: 3 PM

Objectives: Disseminate the possibilities for users with the portal.
Arrange training events co-located with other activities for EISCAT users. The yearly radar-school and the yearly EISCAT_3D user meeting are suitable for co-location of training events. Two training events are planned. The first focuses on the basic functionality of the portal and is held after the portal is available on a production platform. The second training event focuses on the new features implemented in the mini-projects (Task 3 and 4). Sigma is the lead partner, will provide the training and the training material. EISCAT will contribute with relevant use-cases for the training.

Task 2: Deploy the portal as a production system

(M1-M12) Effort: 6 PM

Objectives: Deploy the pilot as a production system.

Develop the pilot into a portal that is of production quality. Include all of the existing EISCAT data in the archive. Sigma leads this project, and provides the development effort. EISCAT provides the data, and testing of the portal.

Task 3: Basic reanalysis within the portal

(M13-M24) Effort: 6 PM

Objectives: Enabling basic reanalysis within the portal.

Basic reanalysis within the portal with user-specified constraints. Instead of downloading large sets of data, it is desirable to be able to do some basic reanalysis of the low levels of data with other constraints than were used in the standard analysis. Basic constraints are defining the volumes of space and the time intervals to integrate. In total there are about 100 analysis parameters to set, and the portal should identify them and provide tools to change them.

EISCAT leads this task providing specifications for the new capabilities. CSC provides the development effort to integrate these into the portal.

Task 4: Use level 3 data as metadata

(M8-M30) Effort: 12PM

Objectives: Enable the use of level 3 data of EISCAT as metadata.

Setup the level 3 data of EISCAT to complement the radar metadata. The level 3 data of EISCAT are the derived ionospheric physical parameters like densities, temperatures and drifts at selected volumes of space. The data is currently stored in a separate database, Madrigal, with no connection to the lower levels of the data. This task should complement the radar metadata, parameters of the radar hardware, with the physical parameters. Currently the only link between the data sets are the time stamps, but it's desirable to use a set of identifiers to more clearly follow how the different levels of data have been formed, exactly which set of level 1 data were used to derive the level 2 and finally level 3 data. Expand the portal of the pilot to be able to search the data based on the expanded metadata set, and allow download of selected levels of data. Investigate further how the access rights of the data should be followed.

EISCAT leads this complex task, specifying how to implement these advanced features. SNIC and CSC provides the development and testing resources.

Task 5: Exploitation

(M13-M30) Effort: 2PM

Objectives: Maintain the portal on a production platform

Deploy, maintain and update the production version of the portal. SNIC leads this task providing suitable resources for operating the portal. EISCAT provides new data to be added to archive.

Deliverables and milestones

The following gives an overview of deliverables/Milestones scheduled

Code	Title	Delivery PM	Delivery CM	Delivered date	Status
M1.1	One half-day training event co-located with the 2016 radar school. An event to train the users in the basic use of the portal.	M19
M1.2	One half-day training event co-located with the 2017 EISCAT_3D user meeting. An event to train users in the more advanced, newly developed features of the portal.	M29
D1.1	Report on user satisfaction survey.	M30
M2.1	Production version of the portal.	M11
D2.1	Documentation of the portal.	M12
M3.1	Release of portal with basic reanalysis implemented.	M23
D3.1	Documentation for the upgraded portal.	M24
D4.1	Define identifiers for the different levels of data and the connection between them.	M13
M4.1	First version of portal ready for tests by selected users.	M20
M4.2	Second version ready after user feedbacks and ready for production.	M28
D4.2	Documentation of the new enhanced portal.	M30
M5.1	Production version of portal deployed on a production platform.	M13