LoFAR

Community Information

Community Name

The Low Frequency Array

Community Short Name

LOFAR

Community Website

http://www.lofar.org/

Community Description

LoFAR will be the first large radio telescope system wherein a huge amount of small sensors are used to achieve its sensitivity instead of a small number of big dishes. For the astronomy application, LOFAR is an aperture synthesis array composed of phased array stations. The antennas in each station form a phased array, producing one or many station beams on the sky. Multi-beaming is a major advantage of the phased array concept. It is not only used to increase observational efficiency, but may be vital for calibration purposes. The phased array stations are combined into an aperture synthesis array. The Remote Stations are distributed over a large area with a maximum baseline of 100 km within the Netherlands and 1500 km within Europe.

Community Objectives

LOFAR started as a new and innovative effort to force a breakthrough in sensitivity for astronomical observations at radio-frequencies below 250 MHz

Main Contact Institutions

ASTRON, CSIC, BSC, IAA

Main Contact

• Michael Wise (ASTRON, wise@astron.nl),
• R.F.Pizzo (ASTRON, pizzo@astron.nl ),
• Susana Sanchez-Exposito (CSIC, sse@iaa.es)
• Daniele Lezzi (BSC, daniele.lezzi@bsc.es)
• Jose Sabater Montes (IAA, jsm@iaa.es)

Prior EGI Collaboration

EGI and LoFAR have been collaborating since Oct 2014 to integrate calibration, analysis and modelling pipelines of radio-astronomy data into a cloud infrastructure. It is developed jointly by users of the [www.lofar.org LOFAR] radio-telescope and members of the AMIGA4GAS project. https://wiki.egi.eu/wiki/FedCloudLOFAR

Science Viewpoint

Scientific challenges

• In most cases, ingest jobs by the Radio Observatory need to be monitored closely to verify that all files are ingested and to manually recover the situation after a failure. This causes quite some inconvenience for some users, who have to wait for several days to get their data. 

• Instability of the ingest system can cause long ingest queues and, inevitably, can make CEP2 very full. In extreme cases, the observing schedule needs to be rearranged because there is not enough disk space available on CEP2 to store more data till important ingest jobs are completed and the corresponding data can be removed from the cluster. This obviously limits the observing efficiency.
• Larger file number/size for staging required 

• Fully exploit processing resources offered by the LOFAR Long-Term Archive

Objectives

• Efficient user data retrieval.
  • Optimise the data staging, e.g., using pre-staging technology to move data from tapes to computing facilities to reduce the waiting for staging when a user requests to retrieve data
  • Allowing user to process large amount of data and retrieve results only to avoid downloading the data to their local computer.
• Elastic disk storage space to allow data ingest jobs to be smoothly executed when handling bust computation

User Stories

LOFAR requests a future system to efficiently support large volumes of data access and burst data access, in particularly, to support the following 2 scenarios:

• User wants to retrieve large volumes of data from LTA. He finds the desired datasets by using the searching facilities provided by LoFAR data portal. An optimisation mechanism is installed which accelerates data staging process. User also starts data processing/analysing service/application running at LTA HPC/Cloud which are near the datasets. The dataset is injected into the processing/analysing service/application, and produced the results. User examines the results using a visualisation service, and downloads the results on his local PC.

• LTA encounters a burst access from users, and existing disk space is too small to handle the requests. Since LTA is federated with EGI Cloud, additional resources are immediately assigned to LOFAR LTA to handle the burst access. After that, the additional resources from EGI FedCloud are released.